A Meta Multi-Objective Reinforcement Learning Framework For Non-Orthogonal, Age-Optimized Information Dissemination in Vehicular Networks
About this article
Abstract
This paper studies how to send fresh and efficient information in vehicular networks. A roadside unit (RSU) updates vehicles about different events. The goal is to reduce the delay in receiving information while also saving power during transmission. The system uses an innovative way of transmitting data. It sends multiple messages at the same time using superposition. Vehicles cancel unwanted signals with a technique called successive interference cancellation (SIC). This method helps to improve the efficiency of communication. The problem is complex because two objectives must be optimized. One is to minimize the delay in information updates, and the other is to minimize the power needed to send updates. This is a multi-objective problem that is difficult to solve using traditional methods. To address this challenge, the paper uses reinforcement learning (RL). A deep Q-network (DQN) decides the best way to decode messages, while a deep deterministic policy gradient (DDPG) model determines the optimal power allocation. Each learning model trains separately for different cases, which increases computational time and effort. Instead of training models separately, the paper proposes a meta-learning approach. This helps estimate good solutions quickly without the need for retraining every time. The meta-model adapts with small updates, saving significant time and computational resources. Simulation results demonstrate that the proposed method outperforms older approaches. It reduces training time while still achieving high efficiency. Moreover, it provides a better balance between formation freshness and power consumption. These improvements make it highly suitable for real-time data sharing in vehicular networks. This research has practical implications for enhancing road safety and smart transportation systems. By optimizing data dissemination it contributes to the development of more reliable and efficient vehicular communication networks.
Keywords
- Reinforcement
- Multi-Objective
- Vehicular Network
- Multi
- Meta Objective
References
[1] C. Zhang, X. Lin, R. Lu, and P.-H. Ho, “Raise: An efficient rsu-aided message authentication scheme in vehicular communication networks,” in 2008 IEEE international conference on communications, pp. 1451–1457, IEEE, 2008. https://doi.org/10.1109/ICC.2008.281.
[2] K. Fall, “A delay-tolerant network architecture for challenged internets,” in Proceedings of the 2003 conference on Applications, technologies, ar-chitectures, and protocols for computer communications, pp. 27–34, 2003. https://doi.org/10.1145/863955.863960.
[3] D. Chefrour, “One-way delay measurement from traditional networks to sdn: A survey,” ACM Computing Surveys (CSUR), vol. 54, no. 7, pp. 1–35, 2021. https://doi.org/10.1145/3466167.
[4] Y. Yuan, B. Yang, W. Su, H. Li, C. Wang, Q. Liu, and T. Taleb, “Aoi and energy-driven dynamic cache updates for wireless edge networks,” IEEE Internet of Things Journal, 2024. https://doi.org/10.1109/JIOT.2024.3470847.
[5] J. Jiang, W. Tu, H. Kong, W. Zeng, R. Zhang, and M. Konecny, “Large-scale urban multiple-modal transport evacuation model for mass gathering events considering pedestrian and public transit system,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 23059–23069, 2022. https://doi.org/10.1109/TITS.2022.3198178.
View more references (24)
[6] W. U. Khan, F. Jameel, N. Kumar, R. J¨antti, and M. Guizani, “Backscatter-enabled efficient v2x communication with non-orthogonal multiple ac-cess,” IEEE Transactions on Vehicular Technology, vol. 70, no. 2, pp. 1724–1735, 2021. https://doi.org/10.1109/TVT.2021.3056220.
[7] B. R. Sharan, S. Deshmukh, S. R. B. Pillai, and B. Beferull-Lozano, “Energy efficient aoi minimization in opportunistic noma/oma broadcast wire-less networks,” IEEE Transactions on Green Communications and Networking, vol. 6, no. 2, pp. 1009–1022, 2021. https://doi.org/10.1109/TGCN.2021.3135351.
[8] S. Wang, S. Zhang, J. Zhang, R. Hu, X. Li, T. Zhang, J. Li, F. Wu, G. Wang, and E. Hovy, “Reinforcement learning enhanced llms: A survey,” arXiv preprint arXiv:2412.10400, 2024.
[9] Y. Long, S. Zhao, S. Gong, B. Gu, D. Niyato, and X. Shen, “Aoi-aware sensing scheduling and trajectory optimization for multi-uav-assisted wire-less backscatter networks,” IEEE Transactions on Vehicular Technology, 2024. https://doi.org/10.1109/TVT.2024.3402740.
[10] A. K. Singh, P. Dziurzanski, H. R. Mendis, and L. S. Indrusiak, “A survey and comparative study of hard and soft real-time dynamic resource allo-cation strategies for multi-/many-core systems,” ACM Computing Surveys (CSUR), vol. 50, no. 2, pp. 1–40, 2017. https://doi.org/10.1145/3057267.
[11] X. Deng, Y. Liang, D. Luo, J. Wang, X. Yan, and J. Duan, “A multi-objective optimization model for rsu deployment in intelligent expressways based on traffic adaptability,” IET Intelligent Transport Systems, vol. 18, no. 11, pp. 2204–2223, 2024. https://doi.org/10.1049/itr2.12568.
[12] W. Xu, H. Zhou, N. Cheng, F. Lyu, W. Shi, J. Chen, and X. Shen, “Internet of vehicles in big data era,” IEEE/CAA Journal of Automatica Sinica, vol. 5, no. 1, pp. 19–35, 2017. https://doi.org/10.1109/JAS.2017.7510736.
[13] Y. Ni, L. Cai, and Y. Bo, “Vehicular beacon broadcast scheduling based on age of information (aoi),” China Communications, vol. 15, no. 7, pp. 67–76, 2018. https://doi.org/10.1109/CC.2018.8424604.
[14] M. I. Ashraf, C.-F. Liu, M. Bennis, W. Saad, and C. S. Hong, “Dynamic resource allocation for optimized latency and reliability in vehicular net-works,” IEEE Access, vol. 6, pp. 63843–63858, 2018. https://doi.org/10.1109/ACCESS.2018.2876548.
[15] A. A. Habob, H. Tabassum, and O. Waqar, “Non-orthogonal age-optimal information dissemination in vehicular networks: A meta multi-objective reinforcement learning approach,” arXiv preprint arXiv:2402.12260, 2024. https://doi.org/10.1109/TMC.2024.3367166.
[16] S. B. Tadele, B. Kar, F. G. Wakgra, and A. U. Khan, “Optimization of end-to-end aoi in edge-enabled vehicular fog systems: A dueling-dqn ap-proach,” IEEE Internet of Things Journal, 2024. https://doi.org/10.1109/JIOT.2024.3472026.
[17] M. Chen, Y. Xiao, Q. Li, and K.-c. Chen, “Minimizing age-of-information for fog computing-supported vehicular networks with deep q-learning,” in ICC 2020-2020 IEEE International Conference on Communications (ICC), pp. 1–6, IEEE, 2020. https://doi.org/10.1109/ICC40277.2020.9149054.
[18] Z. Li, L. Xiang, and X. Ge, “Age of information modeling and optimization for fast information dissemination in vehicular social networks,” IEEE Transactions on Vehicular Technology, vol. 71, no. 5, pp. 5445–5459, 2022. https://doi.org/10.1109/TVT.2022.3154766.
[19] L. Yang, J. Chen, Q. Ni, J. Shi, and X. Xue, “Noma-enabled cooperative unicast–multicast: Design and outage analysis,” IEEE Transactions on Wireless Communications, vol. 16, no. 12, pp. 7870–7889, 2017. https://doi.org/10.1109/TWC.2017.2754261.
[20] A. J. Kadhim and S. A. H. Seno, “Energy-efficient multicast routing protocol based on sdn and fog computing for vehicular networks,” Ad Hoc Networks, vol. 84, pp. 68–81, 2019. https://doi.org/10.1016/j.adhoc.2018.09.018.
[21] P. Scopelliti, G. Araniti, G.-M. Muntean, A. Molinaro, and A. Iera, “A hybrid unicast-multicast utility-based network selection algorithm,” in 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp. 1–6, IEEE, 2017. https://doi.org/10.1109/BMSB.2017.7986146.
[22] C.-Y. Lin and W. Liao, “Aoi-aware interference mitigation for task-oriented multicasting in multi-cell noma networks,” IEEE Transactions on Wire-less Communications, 2024. https://doi.org/10.1109/TWC.2024.3381480.
[23] C. Chen, W.-D. Zhong, and D. H. Wu, “Non-hermitian symmetry orthogonal frequency division multiplexing for multiple-input multiple-output visible light communications,” IEEE/OSA J. Opt. Commun. Netw., vol. 9, no. 1, pp. 36–44, Jan. 2017. https://doi.org/10.1364/JOCN.9.000036.
[24] W. Khalid, A.-A. A. Boulogeorgos, T. Van Chien, J. Lee, H. Lee and H. Yu, “Optimal Operation of Active RIS-Aided Wireless Powered Commu-nications in IoT Networks,” IEEE Internet of Things Journal, vol. 12, no. 1, pp. 390–401, 1 Jan. 2025. https://doi.org/10.1109/JIOT.2024.3461814.
[25] W. Khalid and H. Yu, “Security Improvement with QoS Provisioning Using Service Priority and Power Allocation for NOMA-IoT Networks,” IEEE Access, vol. 9, pp. 9937–9948, Jan. 2021. https://doi.org/10.1109/ACCESS.2021.3049258.
[26] W. Khalid, Z. Kaleem, R. Ullah, T. Van Chien, S. Noh and H. Yu, “Simultaneous Transmitting and Reflecting-Reconfigurable Intelligent Surface in 6G: Design Guidelines and Future Perspectives,” IEEE Network, vol. 37, no. 5, pp. 173–181, Sept. 2023. https://doi.org/10.1109/MNET.129.2200389.
[27] Ummiti Sreenivasulu, Shaik Fairooz, R. Anil Kumar, Sarala Patchala, R. Prakash Kumar, Adireddy Rmaesh, “Joint beamforming with RIS assisted MU-MISO systems using HR-mobilenet and ASO algorithm, Digital Signal Processing, Volume 159, 2025, 104955,ISSN 1051-2004, https://doi.org/10.1016/j.dsp.2024.104955. https://doi.org/10.1016/j.dsp.2024.104955.
[28] Satyam, A. ., Kumar, R. A. ., Patchala , S. ., Pachala, S. ., Geeta Bhimrao Atkar, & Mahalaxm, U. S. B. K. . (2025). Multi-agent learning for UAV networks: a unified approach to trajectory control, frequency allocation and routing. International Journal of Basic and Applied Sciences, 14(2), 189-201. https://doi.org/10.14419/474dfq89.
[29] Kumar , B. P. ., Mahalaxmi , U. S. B. K. ., Nagagopiraju, V. . ., Manda , A. K. ., Chandana , K. ., Betam, D. Suresh . ., Gangadhar , A. ., & Patchala, D. S. . . (2025). Deep Reinforcement Learning for Joint UAV Trajectory and Communication Design in Cache-Enabled Cellular Net-works. International Journal of Basic and Applied Sciences, 14(3), 418-430. https://doi.org/10.14419/djn77m90.