DRL-Based Adaptive Edge Caching for Enhanced Content Delivery

  • Authors

    • Dr. Akula Suneetha Associate Professor, Department of Computer Science and Engineering, KKR & KSR Institute of ‎ Technology and Sciences, Guntur
    • Dr. K. Ratna Babu Lecturer in Computer Engineering, M.B.T.S. Government Polytechnic, Guntur
    • Dr. Inakoti Ramesh Raja Department of ECE, Aditya University, Surampalem - 533437, AP, India
    • Dr. Vullam Nagagopiraju Professor, Department of Computer Science and Engineering, Chalapathi Institute of ‎Engineering and Technology, Guntur
    • Dr. Eppili Jaya Department of ECE, Aditya Institute of Technology and Management Tekkali, Srikakulam, Andhra Pradesh, India
    • Dr. K. B. Glory Assistant Professor, Engineering English, Koneru Lakshmaiah Education Foundation, Vaddeswaram, ‎ Andhra Pradesh, India
    • N Prasanna Lakshmi Assistant Professor, Computer Science and Engineering-AI & ML, RVR & JC College of Engineering
    https://doi.org/10.14419/w8hens76

    Received date: June 28, 2025

    Accepted date: August 18, 2025

    Published date: September 17, 2025

  • DRL; Edge; Recovery; ICE; UAV

  • Abstract

    The number of devices using the internet is growing fast. More people are using smartphones, smart‎watches, and other smart devices. These devices send and receive large amounts of data. This is causing ‎heavy traffic on mobile networks. Because of this, users sometimes experience slow responses and de-‎lays in getting content. To fix this problem, one idea is to use edge caching. When users request data, it ‎is delivered faster from nearby servers instead of faraway cloud centers. This helps improve user experience and reduce network load. This lead to low hit rates, which means users still need to fetch data from ‎far servers. We proposed a smart caching system. It is called ICE (Intelligent Caching at the Edge). ‎ICE uses a technique called Deep Reinforcement Learning (DRL). DRL is a form of machine learning. ‎It helps the system learn from past experiences and make better decisions over time. ICE decides ‎which content should be stored at the edge, and it is removed. ICE uses a special popularity model. It is ‎based on Newton's Law of Cooling. This model predicts how popular a piece of content will be over ‎time. The system uses a type of decision-making model called a Markov Decision Process (MDP). This ‎helps ICE choose the best action in each situation—whether to cache, replace, or skip content. The system is designed to improve the cache hit rate and reduce the time and energy needed to fetch data. The ‎paper proposes a second system called DCCC (Distributed Cooperative Caching with Coordination). ‎DCCC helps multiple edge nodes work together. It uses a two-layer caching setup and spreads content ‎requests across nodes. This improves caching efficiency and uses storage space better. We test ICE and ‎DCCC using many experiments. DCCC improves caching by using resources from many nodes. ICE ‎learns to store the right content using DRL. DCCC makes caching more efficient by coordinating ‎across many edge nodes‎.

  • References

    1. K. Shafique, B. A. Khawaja, F. Sabir, S. Qazi M. Mustaqim, “Internet of things (iot) ‎for next-generation smart systems: A review of current chal-lenges, future trends and ‎prospects for emerging 5g-iot scenarios,” IEEE access, vol. 8, pp. 23022–23040, ‎‎2020.‎ https://doi.org/10.1109/ACCESS.2020.2970118.
    2. Y. Zhang and A. ˚Arvidsson, “Understanding the characteristics of cellular data ‎traffic,” in Proceedings of the 2012 ACM SIGCOMM workshop on Cellular ‎networks: operations, challenges future design, pp. 13–18, 2012.‎ https://doi.org/10.1145/2342468.2342472.
    3. S. Crites, “The narrative quality of experience,” Journal of the American academy of ‎religion, vol. 39, no. 3, pp. 291–311, 1971.‎ https://doi.org/10.1093/jaarel/XXXIX.3.291.
    4. N. P. Lago and F. Kon, “The quest for low latency,” in ICMC, 2004.‎
    5. W. Yu, F. Liang, X. He, W. G. Hatcher, C. Lu, J. Lin X. Yang, “A survey on the ‎edge computing for the internet of things,” IEEE access, vol. 6, pp. 6900–6919, ‎‎2017.‎ https://doi.org/10.1109/ACCESS.2017.2778504.
    6. A. Liu, M. Huang, M. Zhao T. Wang, “A smart high-speed backbone path ‎construction approach for energy and delay optimization in wsns,” IEEE Access, vol. ‎‎6, pp. 13836–13854, 2018.‎ https://doi.org/10.1109/ACCESS.2018.2809556.
    7. T. Wang, Y. Deng, J. Mao, M. Chen, G. Liu, J. Di K. Li, “Towards intelligent ‎adaptive edge caching using deep reinforcement learning,” IEEE Transactions on ‎Mobile Computing, vol. 23, no. 10, pp. 9289–9303, 2024.‎ https://doi.org/10.1109/TMC.2024.3361083.
    8. A. Mondol, R. Gupta, S. Das T. Dutta, “An insight into newton’s cooling law using ‎fractional calculus,” Journal of Applied Physics, vol. 123, no. 6, 2018.‎ https://doi.org/10.1063/1.4998236.
    9. K.-W. Zhao, W.-H. Kao, K.-H. Wu Y.-J. Kao, “Generation of ice states through ‎deep reinforcement learning,” Physical Review E, vol. 99, no. 6, p. 062106, 2019.‎ https://doi.org/10.1103/PhysRevE.99.062106.
    10. W. K. Chai, D. He, I. Psaras G. Pavlou, “Cache “less for more” in information-‎centric networks (extended version),” Computer Communications, vol. 36, no. 7, pp. ‎‎758–770, 2013.‎ https://doi.org/10.1016/j.comcom.2013.01.007.
    11. X. Zhou, F. Shen, L. Liu, W. Liu, L. Nie, Y. Yang H. T. Shen, “Graph convolutional ‎network hashing,” IEEE transactions on cybernetics, vol. 50, no. 4, pp. 1460–1472, ‎‎2018.‎ https://doi.org/10.1109/TCYB.2018.2883970.
    12. R. Dadi, K. Meenakshy S. Damodaran, “A review on secondary control methods in ‎dc microgrid,” Journal of Operation and Automation in Power Engineering, vol. 11, ‎no. 2, pp. 105–112, 2023.‎
    13. X. Xia, F. Chen, Q. He, J. Grundy, M. Abdelrazek H. Jin, “Online collaborative data ‎caching in edge computing,” IEEE Transactions on Parallel and Distributed Systems, ‎vol. 32, no. 2, pp. 281–294, 2020.‎ https://doi.org/10.1109/TPDS.2020.3016344.
    14. Y. C. Hu, M. Patel, D. Sabella, N. Sprecher V. Young, “Mobile edge computing—a ‎key technology towards 5g,” ETSI white paper, vol. 11, no. 11, pp. 1–16, 2015.‎
    15. W.-X. Liu et al., “Content popularity prediction and caching for ICN: A deep ‎learning approach with SDN,” IEEE Access, vol. 6, pp. 5075–5089, 2017.‎ https://doi.org/10.1109/ACCESS.2017.2781716.
    16. Y. Im et al., “SNN-cache: A practical machine learning-based caching system ‎utilizing the inter-relationships of requests,” in Proc. CISS, 2018, pp. 1–6.‎ https://doi.org/10.1109/CISS.2018.8362281.
    17. K. C. Tsai et al., “Mobile social media networks caching with convolutional neural ‎network,” in Proc. IEEE WCNC Workshops, 2018, pp. 83–88.‎ https://doi.org/10.1109/WCNCW.2018.8368988.
    18. A. Lekharu et al., “Deep learning model for content aware caching at MEC servers,” ‎IEEE Trans. Netw. Service Manag., vol. 19, no. 2, pp. 1413–1425, 2022.‎ https://doi.org/10.1109/TNSM.2021.3136439.
    19. Y. Zhang et al., “PSAC: Proactive sequence-aware content caching via deep learning ‎at the network edge,” IEEE Trans. Netw. Sci. Eng., vol. 7, no. 4, pp. 2145–2154, ‎‎2020.‎ https://doi.org/10.1109/TNSE.2020.2990963.
    20. N. Kumar et al., “A novel distributed Q-learning based resource reservation ‎framework for facilitating D2D content access requests in LTE-a net-works,” IEEE ‎Trans. Netw. Service Manag., vol. 15, no. 2, pp. 718–731, 2018.‎ https://doi.org/10.1109/TNSM.2018.2807594.
    21. S. Liu et al., “Distributed reinforcement learning for privacy-preserving dynamic ‎edge caching,” IEEE J. Sel. Areas Commun., vol. 40, no. 3, pp. 749–760, 2022.‎ https://doi.org/10.1109/JSAC.2022.3142348.
    22. H. Zheng et al., “Reinforcement learning for energy-efficient edge caching in mobile ‎edge networks,” in Proc. IEEE INFOCOM Workshops, 2021, pp. 1–6.‎ https://doi.org/10.1109/INFOCOMWKSHPS51825.2021.9484635.
    23. R. Wang et al., “Cooperative caching strategy with content request prediction in ‎Internet of Vehicles,” IEEE Internet Things J., vol. 8, no. 11, pp. 8964–8975, 2021.‎ https://doi.org/10.1109/JIOT.2021.3056084.
    24. C. Zhong et al., “Deep reinforcement learning-based edge caching in wireless ‎networks,” IEEE Trans. Cogn. Commun. Netw., vol. 6, no. 1, pp. 48–61, 2020.‎ https://doi.org/10.1109/TCCN.2020.2968326.
    25. L. T. Tan and R. Q. Hu, “Mobility-aware edge caching and computing in vehicle ‎networks: A deep reinforcement learning,” IEEE Trans. Veh. Technol., vol. 69, no. ‎‎10, pp. 11303–11317, 2020.‎
    26. X. Chen et al., “Distributed edge caching for dynamic contents in wireless service,” ‎IEEE Trans. Wireless Commun., vol. 18, no. 1, pp. 518–531, 2019.‎
    27. L. Qian et al., “Collaborative edge caching with content placement in user-centric ‎networks,” IEEE Access, vol. 8, pp. 141512–141522, 2020.‎
    28. J. Shi et al., “Proactive cooperative caching in mobile edge networks: A deep ‎learning approach,” in Proc. IEEE GLOBECOM, 2020.‎
    29. B. He et al., “Multi-agent AC-based collaborative caching in vehicular networks,” ‎IEEE Internet Things J., vol. 7, no. 6, pp. 4817–4827, 2020.‎
    30. Q. Wang et al., “Online collaborative edge caching with Lyapunov optimization,” ‎IEEE Trans. Veh. Technol., vol. 69, no. 6, pp. 6822–6835, 2020.‎
    31. Satyam A., Kumar R.A., Patchala S., Pachala S., Atkar G.B., Mahalaxmi U.S.B.K. ‎‎“Multi-agent learning for UAV networks: a unified approach to trajectory control, ‎frequency allocation and routing” International Journal of Basic and Applied ‎Sciences, 14 (2), pp. 189 - 201, 2025. https://doi.org/10.14419/474dfq89.
    32. Kailasam N., Yalamati S., Murthy V.S.N., Venkateswara Rao P., Anil Kumar R., ‎Jayaram Kumar K. “Optimized Task Offloading in D2D-Assisted Cloud-Edge ‎Networks Using Hybrid Deep Reinforcement Learning”International Journal of ‎Basic and Applied Sciences, 14 (2), pp. 591 - 602, 2025. https://doi.org/10.14419/xm2ebp25.‎
    33. Uusitalo, Mikko A., Mårten Ericson, Björn Richerzhagen, Elif Ustundag Soykan, ‎Patrik Rugeland, Gerhard Fettweis, Dario Sabella et al. "Hexa-X the European 6G ‎flagship project." In 2021 Joint European Conference on Networks and ‎Communications & 6G Summit (EuCNC/6G Summit), pp. 580-585. IEEE, 2021.‎ https://doi.org/10.1109/EuCNC/6GSummit51104.2021.9482430.
    34. Yakushiji, Koki, Hiroshi Fujita, Mikio Murata, Naoki Hiroi, Yuuichi Hamabe, and ‎Fumiatsu Yakushiji. "Short-range transportation using unmanned aerial vehicles ‎‎(UAVs) during disasters in Japan." Drones 4, no. 4, 2020.‎ https://doi.org/10.3390/drones4040068.
    35. Ummiti Sreenivasulu, Shaik Fairooz, R. Anil Kumar, Sarala Patchala, R. Prakash ‎Kumar, Adireddy Rmaesh, “Joint beamforming with RIS assisted MU-MISO ‎systems using HR-mobilenet and ASO algorithm, Digital Signal Processing, Volume ‎‎159, 2025, 104955,ISSN 1051-2004, https://doi.org/10.1016/j.dsp.2024.104955.
    36. Satyam, A. ., Kumar, R. A. ., Patchala , S. ., Pachala, S. ., Geeta Bhimrao Atkar, & ‎Mahalaxm, U. S. B. K. . (2025). Multi-agent learning for UAV networks: a unified ‎approach ‎to trajectory control, frequency allocation and routing. International ‎Journal of Basic and Applied Sciences, 14(2), 189-‎‎201. https://doi.org/10.14419/474dfq89.
    37. Kumar , B. P. ., Mahalaxmi , U. S. B. K. ., Nagagopiraju, V. . ., Manda , A. K. ., ‎Chandana , K. ., Betam, D. ‎Suresh . ., Gangadhar , A. ., & Patchala, D. S. . . (2025). ‎Deep Reinforcement Learning for Joint UAV Trajecto‎ry and Communication Design ‎in Cache-Enabled Cellular Net-works. International Journal of Basic and Applied ‎Sciences, 14(3), 418-430. https://doi.org/10.14419/djn77m90.

  • Downloads

  • How to Cite

    Suneetha, D. A. ., Babu , D. K. R. ., Raja , D. I. R. ., Nagagopiraju , D. V. ., Jaya , D. E. ., Glory , D. K. B. ., & Lakshmi, N. P. . (2025). DRL-Based Adaptive Edge Caching for Enhanced Content Delivery. International Journal of Basic and Applied Sciences, 14(5), 603-616. https://doi.org/10.14419/w8hens76