Multi-agent learning for UAV networks: a unified approach ‎to trajectory control, frequency allocation and routing

  • Authors

    • Angara Satyam Department of Information Technology, Sagi Rama Krishnam Raju Engineering College(A), Bhimavaram, AP, India
    • R. Anil Kumar Department of Electronics and Communication Engineering, Aditya University, Surampalem, AP, India
    • Sarala Patchala Department of Electronics and Communication Engineering KKR & KSR Institute of Technology and Sciences, Guntur, AP, India
    • Sunitha Pachala Department of Artificial Intelligence and Data Science, Koneru Lakshmaiah Education Foundation, Guntur, AP, India
    • Geeta Bhimrao Atkar Department of Computer Science Engineering, G H Raisoni College of Engineering and Management, Pune, Maharashtra, India
    • U. S. B. K. Mahalaxm Department of Electronics and Communication Engineering, Aditya University, Surampalem, AP, India
    https://doi.org/10.14419/474dfq89

    Received date: May 6, 2025

    Accepted date: May 28, 2025

    Published date: June 11, 2025

  • UAV; Joint Optimization; LSTM Cell; JTFR Algorithm; Reinforcement Learning‎.

  • Abstract

    Unmanned aerial vehicle (UAV) swarm networks have various applications. They support surveillance and communication. However, these ‎networks face many challenges. High mobility causes frequent link breakages. Limited transmission range increases interference. Routing ‎becomes complex due to these factors. This paper proposes a new approach. It jointly optimizes trajectory control, frequency allocation and ‎routing. The proposed algorithm is called JTFR. It improves link utility in UAV swarm networks. Link utility depends on signal strength, ‎queue delay and residual energy. The proposed JTFR uses multi-agent reinforcement learning. The learning model considers both local and ‎neighbour information. This improves the stability of the network. A long short-term memory (LSTM) network is used. This helps in pre-‎dicting the best trajectory. A multi-head attention mechanism is also used. Each UAV adjusts its policy based on its neighbours. UAV ‎swarm networks are dynamic. The movement of UAVs changes frequently. This affects network connectivity. The routing decisions must ‎adapt to these changes. The proposed approach provides adaptive routing. JTFR is overall useful to improve UAV swarm communication. ‎It handles trajectory control, frequency allocation and routing in balancing. The method is dynamic in the sense that it adapts to dynamic ‎network conditions. The UAVs work well in complex environments‎.

  • References

    1. A. Khan, S. Gupta, and S. K. Gupta, “Emerging uav technology for disaster detection, mitigation, response, and preparedness,” Journal of Field Robotics, vol. 39, no. 6, pp. 905–955, 2022. https://doi.org/10.1002/rob.22075.
    2. Z. Feng, L. Ji, Q. Zhang, and W. Li, “Spectrum management for mmwave enabled uav swarm networks: Challenges and opportunities,” IEEE Communications Magazine, vol. 57, no. 1, pp. 146–153, 2018. https://doi.org/10.1109/MCOM.2018.1800087.
    3. W. Chen, J. Liu, H. Guo, and N. Kato, “Toward robust and intelligent drone swarm: Challenges and future directions,” IEEE Network, vol. 34, no. 4, pp. 278–283, 2020. https://doi.org/10.1109/MNET.001.1900521.
    4. L. Liu, S. Zhang, and R. Zhang, “Multi-beam uav communication in cellular uplink: Cooperative interference cancellation and sum-rate maximiza-tion,” IEEE Transactions on Wireless Communications, vol. 18, no. 10, pp. 4679–4691, 2019. https://doi.org/10.1109/TWC.2019.2926981.
    5. Z. Xiao, L. Zhu, L. Bai, and X.-G. Xia, Array beamforming enabled wireless communications. CRC Press, 2023. https://doi.org/10.1201/9781003366362.
    6. S. Javaid, N. Saeed, Z. Qadir, H. Fahim, B. He, H. Song, and M. Bilal, “Communication and control in collaborative uavs: Recent advances and future trends,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 6, pp. 5719–5739, 2023. https://doi.org/10.1109/TITS.2023.3248841.
    7. D.-G. Zhang, L. Chen, J. Zhang, J. Chen, T. Zhang, Y.-M. Tang, and J.N. Qiu, “A multi-path routing protocol based on link lifetime and energy consumption prediction for mobile edge computing,” IEEE Access, vol. 8, pp. 69058–69071, 2020. https://doi.org/10.1109/ACCESS.2020.2986078.
    8. X. Cao, B. Yang, K. Wang, X. Li, Z. Yu, C. Yuen, Y. Zhang, and Z. Han, “Ai-empowered multiple access for 6g: A survey of spectrum sensing, protocol designs, and optimizations,” Proceedings of the IEEE, 2024. https://doi.org/10.1109/JPROC.2024.3417332.
    9. Z. Lin, Z. Zeng, Y. Yu, Y. Ren, X. Qiu, and J. Chen, “Graph attention residual network based routing and fault-tolerant scheduling mechanism for data flow in power communication network.,” Computers, Materials & Continua, vol. 81, no. 1, 2024. https://doi.org/10.32604/cmc.2024.055802.
    10. L. Gupta, R. Jain, and G. Vaszkun, “Survey of important issues in uav communication networks,” IEEE communications surveys & tutorials, vol. 18, no. 2, pp. 1123–1152, 2015. https://doi.org/10.1109/COMST.2015.2495297.
    11. B. Chen, K. Jamieson, H. Balakrishnan, and R. Morris, “Span: An energy-efficient coordination algorithm for topology maintenance in ad hoc wire-less networks,” in Proceedings of the 7th annual international conference on Mobile computing and networking, pp. 85–96, 2001. https://doi.org/10.1145/381677.381686.
    12. P. Mishra and G. Singh, “Energy management systems in sustainable smart cities based on the internet of energy: A technical review,” Energies, vol. 16, no. 19, p. 6903, 2023. https://doi.org/10.3390/en16196903.
    13. M. M. Alam and S. Moh,” Joint Trajectory Control, Frequency Allocation, and Routing for UAV Swarm Networks: A Multi-Agent Deep Rein-forcement Learning Approach,” IEEE Transactions on Mobile Computing, vol. 23, no. 12, pp. 11989-12005, Dec. 2024. https://doi.org/10.1109/TMC.2024.3403890.
    14. J. Smith and K. Patel,” Behavior-Based UAV Trajectory Optimization,”IEEE Transactions on Wireless Communications, vol. 21, no. 6, pp. 3543-3556, Jun. 2023.
    15. L. Zhang, X. Li, and M. Chen,” Swarm Intelligence for UAV Networks,” IEEE Communications Letters, vol. 26, no. 8, pp. 1564-1568, Aug. 2023.
    16. H. Wang and Y. Zhao,” Dynamic Frequency Selection for UAV Communication,” IEEE Internet of Things Journal, vol. 10, no. 3, pp. 2156-2168, Mar. 2024.
    17. P. Kumar and S. Lee,” Adaptive Frequency Reuse for UAV Networks,” IEEE Transactions on Networking, vol. 32, no. 5, pp. 1457-1470, May 2023.
    18. A. Brown and T. Davis,” Machine Learning-Based Frequency Allocation for UAVs,” IEEE Transactions on Cognitive Communications, vol. 16, no.2, pp. 890-902, Apr. 2023.
    19. C. Wilson and M. Garcia,” Multi-Hop Routing in UAV Networks,” IEEE Transactions on Vehicular Technology, vol. 72, no. 4, pp. 3301-3315, Apr. 2024.
    20. N. Fernandez and R. Clark,” Position-Based Routing for UAVs,” IEEE Transactions on Communications, vol. 33, no. 6, pp. 987-1001, Jun. 2024.
    21. B. Johnson and H. Kim,” Predictive Routing Using Machine Learning in UAV Networks,” IEEE Transactions on Networking, vol. 29, no. 9, pp. 3564-3578, Sep. 2023.
    22. Vijay, S. T. Siddiqui, Ritu, R. A. Kumar, A. Kumar, S. Umamaheswararao, N. Dudiki, Y. V. Reddy, and K. Dekeba, “Intertwine connection-based routing path selection for data transmission in mobile cellular networks and wireless sensor networks,” Wireless Communications and Mobile Com-puting, vol. 2022, no. 1, p. 8398128, 2022. https://doi.org/10.1155/2022/8398128.
    23. E. Roberts and V. Carter,” Cross-Layer Optimization for UAV Networks,” IEEE Transactions on Wireless Communications, vol. 22, no. 7, pp. 4875-4888, Jul. 2023.
    24. R. K. Bharti, S. S, S. V. Sumant, C. A. D. Durai, R. A. Kumar, K. Singh, H. Palivela, B. R. Kumar, and B. Debtera, “Enhanced path routing with buffer allocation method using coupling node selection algorithm in MANET,” Wireless Communications and Mobile Computing, vol. 2022, no. 1, p. 1955290, 2022. https://doi.org/10.1155/2022/1955290.
    25. M. Thompson and L. White,” Deep Reinforcement Learning for UAV Optimization,” IEEE Transactions on Mobile Computing, vol. 20, no. 5, pp.2678-2692, May 2024.
    26. R. Gowtham, V. Anand, Y. V. Suresh, K. L. Narasimha, R. A. Kumar, and V. Saraswathi, “Enhancing incentive schemes in edge computing through hierarchical reinforcement learning,” Journal of Engineering and Technology for Industrial Applications, vol. 11, no. 52, p. 226 – 236, 2025. https://doi.org/10.5935/jetia.v11i52.1637.
    27. R. Hall and S. Young,” Collaborative Decision-Making in UAV Swarm Networks,” IEEE Transactions on Intelligent Systems, vol. 31, no. 3, pp. 1472-1485, Mar. 2024.
    28. F.-H. Tseng and Y.-J. Hsieh, “Delanalty minimization with reinforcement learning in uav-aided mobile network,” IEEE Transactions on Computa-tional Social Systems, vol. 11, no. 2, pp. 1991–2001, 2024. https://doi.org/10.1109/TCSS.2023.3283016.
    29. S. Javed, A. Hassan, R. Ahmad, W. Ahmed, R. Ahmed, A. Saadat, and M. Guizani, “State-of-the-art and future research challenges in UAV swarms,” IEEE Internet of Things Journal, vol. 11, no. 11, pp. 19023–19045, 2024. https://doi.org/10.1109/JIOT.2024.3364230.
    30. O. A. Amodu, C. Jarray, R. Azlina Raja Mahmood, H. Althumali, U. Ali Bukar, R. Nordin, N. F. Abdullah, and N. Cong Luong, “Deep reinforce-ment learning for aoi minimization in uav-aided data collection for WSN and IOT applications: A survey,” IEEE Access, vol. 12, pp. 108000–108040, 2024. https://doi.org/10.1109/ACCESS.2024.3425497.
    31. S. Lee, T.-W. Ban, and H. Lee, “Network-wide energy-efficiency maximization in UAV-aided IOT networks: Quasi-distributed deep reinforce-ment learning approach,” IEEE Internet of Things Journal, vol. 12, no. 11, pp. 15404– 15414, 2025. https://doi.org/10.1109/JIOT.2025.3532477.
    32. S. Fatima, M. Subba Raju, R. Anil Kumar, V. Anand, Y. K. S. S. Rao, D. Kishore, U. Mahalaxmi, and K. Kalyani, “A neural network approach to beam selection and power optimization in mm wave massive MIMO,” Journal of Mechanics of Continua and Mathematical Sciences, vol. 20, no. 4, p. 30–52, 2025. https://doi.org/10.26782/jmcms.2025.04.00003.

  • Downloads

  • How to Cite

    Satyam, A. ., Kumar, R. A. ., Patchala , S. ., Pachala, S. ., Geeta Bhimrao Atkar, & Mahalaxm, U. S. B. K. . (2025). Multi-agent learning for UAV networks: a unified approach ‎to trajectory control, frequency allocation and routing. International Journal of Basic and Applied Sciences, 14(2), 189-201. https://doi.org/10.14419/474dfq89