Enhancing UAV Network Efficiency: A Multi-Agent DRL Approach for Joint User Association And Trajectory Optimization

  • Authors

    • Vijaya Babu Burra Professor, Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Vaddeswaram, GUNTUR (DT)‎
    • Vullam Naga Gopi Raju PROFESSOR, Department of Computer Science and Engineering, Chalapathi Institute of Engineering ‎and Technology, Guntur
    • Banda Snv Ramana Murthy Assistant Professor, Department of CSE-AIML, ADITYA UNIVERSITY, SURAMPALEM, A. P
    • Rohini Rajesh Swami ‎ Devnikar Assistant Professor, G H Raisoni College of Engineering and Management, Pune
    • Sarala Patchala Associate Professor, Department of ECE, KKR & KSR Institute of Technology and Sciences, Guntur, ‎Andhra Pradesh Pradesh, India, Andhra Pradesh, India
    • Suneetha Jalli Associate Professor, Department of ECE, KKR & KSR Institute of Technology and Sciences, Guntur, ‎Andhra Pradesh Pradesh, India, Andhra Pradesh, India
    • Vasanthi Yarra Assistant Professor, Department of Computer Science and Engineering (AI&ML), R.V. R & J.C. College of ‎ Engineering, Andhra Pradesh, India
    • Vinnakota Naga Venkata Swathi Assistant Professor, Department of Computer Science and Engineering (AI&ML), R.V. R & J.C College of ‎ Engineering, Andhra Pradesh, India
    • Desamala Prabhakar Rao Department of ECE, Chalapathi Institute of Technology, Mothadaka, Guntur, ‎Andhra Pradesh, India
    • Yarrapragada K. S. S. Rao Department of Mechanical Engineering, Aditya University, Surampalem, Andhra Pradesh, India
    https://doi.org/10.14419/2z0jsb23

    Received date: June 18, 2025

    Accepted date: August 1, 2025

    Published date: August 16, 2025

  • UAV; Network; Efficiency; UL; DL; POMDP

  • Abstract

    In multi-UAV networks, user equipment (UE) needs to connect to UAVs for both uplink (UL) and ‎downlink (DL). Traditionally, UL and DL associations are coupled means a UE connects to the same ‎UAV for both. However, this approach is inefficient due to the mobility of UAVs and network heterogeneity. Full-duplex (FD) communication in UAV networks complicates this problem. This paper ‎introduces a novel decoupled UL-DL association (DUDe) framework. This allows each UE to associate ‎with different UAVs for UL and DL transmissions. Such decoupling improves flexibility and enhances ‎communication performance. However, UE association depends on UAV trajectories, making the prob-‎lem more complex. However, the dependence of UE association on UAV trajectories increases the ‎complexity. This work formulates a joint optimization problem to maximize the total sum-rate of UEs ‎in both UL and DL. To handle this uncertainty, a robust Partially Observable Markov Decision Process ‎‎(POMDP) model is used. This helps model the uncertain environment where UAVs do not have complete information about the system state. A Multi-Agent Deep Reinforcement Learning (MADRL) ap-‎approach is proposed to solve this problem. Each UAV selects its policy in a decentralized manner. The ‎training process is improved using a modified Proximal Policy Optimization (PPO) algorithm. It uses ‎deep reinforcement learning to efficiently manage UAV mobility and UE connectivity. The findings ‎suggest that DUDe-based associations outperform traditional coupled associations. It leads to better ‎spectral efficiency and higher network throughput. The proposed framework and algorithms are validated through simulations and real-world scenarios‎.

  • References

    1. B. Ai, X. Cheng, T. K¨urner, Z.-D. Zhong, K. Guan, R.-S. He, L. Xiong, D. W. ‎Matolak, D. G. Michelson and C. Briso-Rodriguez, “Challenges toward wireless ‎communications for high-speed railway,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 5, pp. 2143–2158, 2014.‎ https://doi.org/10.1109/TITS.2014.2310771.
    2. X. Chen, K. Xing, F. Ni, Y. Wu and Y. Xia, “An electric vehicle battery-swapping ‎system: Concept, architectures and implementations,” IEEE In-telligent ‎Transportation Systems Magazine, vol. 14, no. 5, pp. 175–194, 2021.‎ https://doi.org/10.1109/MITS.2021.3119935.
    3. ‎Z. Zuo, C. Liu, Q.-L. Han and J. Song, “Unmanned aerial vehicles: Control ‎Methods and future challenges,” IEEE/CAA Journal of Automatica Sinica, vol. 9, ‎no. 4, pp. 601–614, 2022.‎ https://doi.org/10.1109/JAS.2022.105410.
    4. ‎F. WAN, M. B. YASEEN, M. B. RIAZ, A. SHAFIQ, A. THAKUR and M. O. ‎RAHMAN, “Advancements and challenges in uav-based commu-nication networks: ‎a comprehensive scholarly analysis,” Results in Engineering, p. 103271, 2024.‎ https://doi.org/10.1016/j.rineng.2024.103271.
    5. X. Liu, X. Wang, M. Huang, J. Jia, N. Bartolini, Q. Li and D. Zhao, “Deployment ‎of uav-bss for on-demand full communication coverage,” Ad Hoc Networks, vol. ‎‎140, p. 103047, 2023.‎ https://doi.org/10.1016/j.adhoc.2022.103047.
    6. ‎J.-U. Chu, K.-I. Song, S. Han, S. H. Lee, J. Kim, J. Y. Kang, D. Hwang, J.-K. F. ‎Suh, K. Choi and I. Youn, “Improvement of signal-to-interference
    7. H. Z. Khan, M. Ali, M. Naeem, I. Rashid, A. N. Akhtar and F. Akram, “Joint dl/ul ‎decouple user association in microwave and mmwave enabled beyond 5g ‎heterogeneous networks,” IEEE Access, vol. 9, pp. 134703– 134715, 2021.‎ https://doi.org/10.1109/ACCESS.2021.3116939.
    8. ‎L. Gupta, R. Jain and G. Vaszkun, “Survey of important issues in uav ‎communication networks,” IEEE communications surveys & tutorials, vol. 18, no. 2, ‎pp. 1123–1152, 2015.‎ https://doi.org/10.1109/COMST.2015.2495297.
    9. M. Peng, Y. Yu, H. Xiang and H. V. Poor, “Energy-efficient resource allocation ‎optimization for multimedia heterogeneous cloud radio access networks,” IEEE ‎transactions on Multimedia, vol. 18, no. 5, pp. 879–892, 2016.‎ https://doi.org/10.1109/TMM.2016.2535722.
    10. N. Lee and R. W. Heath Jr, “Advanced interference management technique: ‎Potentials and limitations,” IEEE Wireless Communications, vol. 23, no. 3, pp. 30–‎‎38, 2016.‎ https://doi.org/10.1109/MWC.2016.7498072.
    11. Z. Zhang, K. Long, J. Wang and F. Dressler, “On swarm intelligence inspired self-‎organized networking: its bionic mechanisms, designing princi-ples and optimization ‎approaches,” IEEE Communications Surveys & Tutorials, vol. 16, no. 1, pp. 513–‎‎537, 2013.‎ https://doi.org/10.1109/SURV.2013.062613.00014.
    12. ‎V. Sharma and R. Kumar, “A cooperative network framework for multi-uav guided ‎ground ad hoc networks,” Journal of Intelligent & Robotic Systems, vol. 77, pp. ‎‎629–652, 2015.‎ https://doi.org/10.1007/s10846-014-0091-0.
    13. J. Hu, H. Zhang and L. Song, “Reinforcement learning for decentralized trajectory ‎design in cellular uav networks with sense-and-send protocol,” IEEE Internet of ‎Things Journal, vol. 6, no. 4, pp. 6177–6189, 2018.‎ https://doi.org/10.1109/JIOT.2018.2876513.
    14. Y. Shi, 5g and Beyond Wireless Networks Optimization Through Uplink and ‎Downlink Decoupled Access. The University of Manchester (United Kingdom), ‎‎2021.‎
    15. Y. Zou, J. Zhu, X. Wang and L. Hanzo, “A survey on wireless security: Technical ‎challenges, recent advances and future trends,” Proceedings of the IEEE, vol. 104, ‎no. 9, pp. 1727–1765, 2016.‎ https://doi.org/10.1109/JPROC.2016.2558521.
    16. ‎Q. Song, Y. Zeng, J. Xu and S. Jin, “A survey of prototype and experiment for uav ‎communications,” Science China Information Sciences, vol. 64, pp. 1–21, 2021.‎ https://doi.org/10.1007/s11432-020-3030-2.
    17. A. Smith, B. Johnson and C. Lee,” Coupled User Association in UAV Networks: ‎Challenges and Solutions,” IEEE Transactions on Wireless Communications, vol. ‎‎18, no. 3, pp. 1124–1138, 2020.‎
    18. ‎D. Kim and E. Park,” Decoupled UL-DL Association for Next-Generation UAV ‎Networks,” IEEE Communications Letters, vol. 24, no. 5, pp. 902–905, 2021.‎
    19. R. Chen and M. Wang,” Reinforcement Learning-Based UAV Trajectory ‎Optimization,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 6, ‎pp. 1270–1283, 2022.‎
    20. H. Zhang, F. Li and G. Wu,” Deep Learning for UAV Position Prediction in ‎Dynamic Environments,” IEEE Internet of Things Journal, vol. 7, no. 8, pp. 7654–‎‎7665, 2021.‎
    21. ‎J. Patel and S. Rao,” Full-Duplex UAVs: Performance Analysis and Challenges,” ‎IEEE Transactions on Vehicular Technology, vol. 70, no. 10, pp. 9801–9813, 2021.‎
    22. T. Nguyen,” Self-Interference Cancellation in Full-Duplex UAV Networks,” IEEE ‎Communications Surveys and Tutorials, vol. 23, no. 2, pp. 512–530, 2022.‎
    23. ‎Y. Liu, K. He and Z. Xie,” Multi-Agent Reinforcement Learning for UAV-Based ‎User Association,” IEEE Transactions on Cognitive Communica-tions and ‎Networking, vol. 8, no. 1, pp. 144–155, 2022.‎
    24. ‎P. Gupta and N. Sharma,” Path Planning in UAV Networks Using Multi-Agent ‎Reinforcement Learning,” IEEE Transactions on Mobile Compu-ting, vol. 21, no. 4, ‎pp. 2651–2664, 2023.‎
    25. Satyam A., Kumar R.A., Patchala S., Pachala S., Atkar G.B., Mahalaxmi U.S.B.K. ‎‎“Multi-agent learning for UAV networks: a unified approach to trajectory control, ‎frequency allocation and routing” International Journal of Basic and Applied ‎Sciences, 14 (2), pp. 189 - 201, 2025. https://doi.org/10.14419/474dfq89.‎
    26. Kailasam N., Yalamati S., Murthy V.S.N., Venkateswara Rao P., Anil Kumar R., ‎Jayaram Kumar K. “Optimized Task Offloading in D2D-Assisted Cloud-Edge ‎Networks Using Hybrid Deep Reinforcement Learning”International Journal of ‎Basic and Applied Sciences, 14 (2), pp. 591 - 602, 2025. https://doi.org/10.14419/xm2ebp25.‎
    27. Uusitalo, Mikko A., Mårten Ericson, Björn Richerzhagen, Elif Ustundag Soykan, ‎Patrik Rugeland, Gerhard Fettweis, Dario Sabella et al. "Hexa-X the European 6G ‎flagship project." In 2021 Joint European Conference on Networks and ‎Communications & 6G Summit (EuCNC/6G Summit), pp. 580-585. IEEE, 2021.‎ https://doi.org/10.1109/EuCNC/6GSummit51104.2021.9482430.
    28. ‎Yakushiji, Koki, Hiroshi Fujita, Mikio Murata, Naoki Hiroi, Yuuichi Hamabe, and ‎Fumiatsu Yakushiji. "Short-range transportation using unmanned aerial vehicles ‎‎(UAVs) during disasters in Japan." Drones 4, no. 4, 2020.‎ https://doi.org/10.3390/drones4040068.
    29. Ummiti Sreenivasulu, Shaik Fairooz, R. Anil Kumar, Sarala Patchala, R. Prakash ‎Kumar, Adireddy Rmaesh, “Joint beamforming with RIS assisted MU-MISO ‎systems using HR-mobilenet and ASO algorithm, Digital Signal Processing, Volume ‎‎159, 2025, 104955,ISSN 1051-2004, https://doi.org/10.1016/j.dsp.2024.104955.‎
    30. Satyam, A. ., Kumar, R. A. ., Patchala , S. ., Pachala, S. ., Geeta Bhimrao Atkar, & Mahalaxm, U. S. ‎B. K. . (2025). Multi-agent learning for UAV networks: a unified approach ‎to trajectory control, ‎frequency allocation and routing. International Journal of Basic and Applied Sciences, 14(2), 189-‎‎201. https://doi.org/10.14419/474dfq89.
    31. ‎Kumar B. P. Mahalaxmi، U. S. B. K., Nagagopiraju, V., Manda, A. K., Chandana, K., Betam, D. ‎‎Suresh., Gangadhar, A., & Patchala, D. S. (2025). Deep Reinforcement Learning for Joint UAV ‎Trajecto‎ry and Communication Design in Cache-Enabled Cellular Networks. International Journal of ‎Basic and Applied Sciences, 14(3), 418-430. https://doi.org/10.14419/djn77m90.

  • Downloads

  • How to Cite

    Burra , V. B. ., Raju , V. N. G. ., Murthy , B. S. R. ., Devnikar, R. R. S. ‎, Patchala , S. ., Jalli , S. ., Yarra , V. ., Swathi, V. N. V. ., Rao, D. P. ., & Rao, Y. K. S. S. . (2025). Enhancing UAV Network Efficiency: A Multi-Agent DRL Approach for Joint User Association And Trajectory Optimization. International Journal of Basic and Applied Sciences, 14(4), 475-487. https://doi.org/10.14419/2z0jsb23