Journal Article2025Cited by 6

A Graph-Assisted Digital-Twin-Driven Multiagent Shared Offloading for Internet of Vehicles

Md Zahangir Alam, Suryaia Rahman, Md Asif Bin Khaled, Ashraful Islam, Abbas Jamalipour

IEEE Internet of Things Journal

Institute of Electrical and Electronics Engineers (IEEE), Vol. 12, Issue 11, pp. 17349-17363, ISBN: 2327-4662

DOI: 10.1109/jiot.2025.3538657URL

Abstract

Vehicular edge computing (VEC) allows vehicles to process part of the tasks locally at the network edge while offloading the rest of the tasks to a centralized cloud server for processing. A massive volume of tasks generated by the Internet of Vehicles (IoV) leads to buffer overflow that causes higher latency. Elevating latency, in turn, can increase network energy consumption. Both higher latency and energy consumption lead to a degradation of network performance. Therefore, VEC design requires a balance between latency and energy consumption tradeoff. To reduce overwhelming amount of offloading to edge servers, a cooperative cluster-based shared offloading strategy has been proposed in this work. We use digital twin technology in VEC for managing and adapting to environmental dynamic changes. Then, we leverage Lyapunov (Ly) optimization to transform the stochastic offloading problem into a more manageable deterministic form. Finally, we present a decentralized coordination graph (CG)-driven Ly-based multiagent deep deterministic policy gradient (CG-LyMADDPG) algorithm that trains agents toward energy efficient optimal offloading policy while maintaining queue stability at a maximum delay constraint. The experimental result shows that the proposed learning significantly outperforms the baseline algorithms for energy savings while maintain queue stability.

Keywords

  • vehicular edge computing
  • Lyapunov optimization
  • digital twin
  • Internet of Vehicles
  • task offloading
  • queue stability
  • latency-energy tradeoff
  • multi-agent reinforcement learning
  • deep deterministic policy gradient
  • energy efficiency

CCDS Authors

MD Asif Bin Khaled
Md Asif Bin Khaled

MD Asif Bin Khaled

  • Co-Director, MIRA Wing

Published at

IEEE Internet of Things Journal

Ashraful Islam, PhD
Ashraful Islam

Ashraful Islam, PhD

  • Center Director, CCDS
  • Director, HCI Wing

Assistant Professor

Department of Computer Science and Engineering

Independent University, Bangladesh (IUB)

Published at

CHI, UIST, UbiComp, IJCNN, ICIP, HCII, ICDMW, BLP, IEEE Internet of Things Journal, JMIR, JMIR Human Factors, JMIR Formative Research, IEEE Access, Smart Agricultural Technology, Diabetes Research and Clinical Practice, PLOS ONE

Research Interests

Human-Computer Interaction, AI for Social Good, AI for Public Health, AI for Impact

References

  1. 1.Michael J. Neely. (2010). Stochastic Network Optimization with Application to Communication and Queueing Systems. Synthesis lectures on learning, networks, and algorithms[10.1007/978-3-031-79995-2]
  2. 2.Zhu Xiao, Jinmei Shu, Hongbo Jiang, Geyong Min, Hongyang Chen, Zhu Han. (2022). Perception Task Offloading With Collaborative Computation for Autonomous Driving. IEEE Journal on Selected Areas in Communications, 41(2), 457–473[10.1109/jsac.2022.3227027]
  3. 3.Suzhi Bi, Liang Huang, Hui Wang, Ying–Jun Angela Zhang. (2021). Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks. IEEE Transactions on Wireless Communications, 20(11), 7519–7537[10.1109/twc.2021.3085319]
  4. 4.Ke Zhang, Jiayu Cao, Yan Zhang. (2021). Adaptive Digital Twin and Multiagent Deep Reinforcement Learning for Vehicular Edge Computing and Networks. IEEE Transactions on Industrial Informatics, 18(2), 1405–1413[10.1109/tii.2021.3088407]
  5. 5.Lei Liu, Ming Zhao, Yu Miao, Mian Ahmad Jan, Dapeng Lan, Amirhosein Taherkordi. (2022). Mobility-Aware Multi-Hop Task Offloading for Autonomous Driving in Vehicular Edge Computing and Networks. IEEE Transactions on Intelligent Transportation Systems, 1–14[10.1109/tits.2022.3142566]
  6. 6.Yueyue Dai, Ke Zhang, Sabita Maharjan, Yan Zhang. (2020). Deep Reinforcement Learning for Stochastic Computation Offloading in Digital Twin Networks. IEEE Transactions on Industrial Informatics, 17(7), 4968–4977[10.1109/tii.2020.3016320]
  7. 7.Radhika Ranjan Roy. (2010). Handbook of Mobile Ad Hoc Networks for Mobility Models. [10.1007/978-1-4419-6050-4]
  8. 8.Huaming Wu, Junqi Chen, Tu N. Nguyen, Huijun Tang. (2022). Lyapunov-Guided Delay-Aware Energy Efficient Offloading in IIoT-MEC Systems. IEEE Transactions on Industrial Informatics, 19(2), 2117–2128[10.1109/tii.2022.3206787]
  9. 9.Liang Zhao, Zijia Zhao, Enchao Zhang, Ammar Hawbani, Ahmed Al‐Dubai, Zhiyuan Tan, Amir Hussain. (2023). A Digital Twin-Assisted Intelligent Partial Offloading Approach for Vehicular Edge Computing. IEEE Journal on Selected Areas in Communications, 41(11), 3386–3400[10.1109/jsac.2023.3310062]
  10. 10.M. Alam, Abbas Jamalipour. (2022). Multi-Agent DRL-Based Hungarian Algorithm (MADRLHA) for Task Offloading in Multi-Access Edge Computing Internet of Vehicles (IoVs). IEEE Transactions on Wireless Communications, 21(9), 7641–7652[10.1109/twc.2022.3160099]
  11. 11.Mohammad Parvini, Mohammad Reza Javan, Nader Mokari, Bijan Abbasi Arand, Eduard A. Jorswieck. (2023). AoI-Aware Resource Allocation for Platoon-Based C-V2X Networks via Multi-Agent Multi-Task Reinforcement Learning. IEEE Transactions on Vehicular Technology, 72(8), 9880–9896[10.1109/tvt.2023.3259688]
  12. 12.Yu Gong, Yifei Wei, Zhiyong Feng, F. Richard Yu, Yan Zhang. (2022). Resource Allocation for Integrated Sensing and Communication in Digital Twin Enabled Internet of Vehicles. IEEE Transactions on Vehicular Technology, 72(4), 4510–4524[10.1109/tvt.2022.3228583]
  13. 13.Peisong Li, Ziren Xiao, Xinheng Wang, Kaizhu Huang, Yi Huang, Honghao Gao. (2023). EPtask: Deep Reinforcement Learning Based Energy-Efficient and Priority-Aware Task Scheduling for Dynamic Vehicular Edge Computing. IEEE Transactions on Intelligent Vehicles, 9(1), 1830–1846[10.1109/tiv.2023.3321679]
  14. 14.Bishmita Hazarika, Keshav Singh, Chih–Peng Li, Anke Schmeink, Kim Fung Tsang. (2023). RADiT: Resource Allocation in Digital Twin-Driven UAV-Aided Internet of Vehicle Networks. IEEE Journal on Selected Areas in Communications, 41(11), 3369–3385[10.1109/jsac.2023.3310048]
  15. 15.Han Hu, Weiwei Song, Qun Wang, Rose Qingyang Hu, Hongbo Zhu. (2022). Energy Efficiency and Delay Tradeoff in an MEC-Enabled Mobile IoT Network. IEEE Internet of Things Journal, 9(17), 15942–15956[10.1109/jiot.2022.3153847]
  16. 16.Liwei Geng, Hongbo Zhao, Jiayue Wang, Aryan Kaushik, S. W. K. Yuan, Wenquan Feng. (2023). Deep-Reinforcement-Learning-Based Distributed Computation Offloading in Vehicular Edge Computing Networks. IEEE Internet of Things Journal, 10(14), 12416–12433[10.1109/jiot.2023.3247013]
  17. 17.Zhenchuan Sun, Yijun Mo, Yu Chen. (2021). Graph-Reinforcement-Learning-Based Task Offloading for Multiaccess Edge Computing. IEEE Internet of Things Journal, 10(4), 3138–3150[10.1109/jiot.2021.3123822]
  18. 18.Anitha Saravana Kumar, Lian Zhao, Xavier Fernando. (2023). Task Offloading and Resource Allocation in Vehicular Networks: A Lyapunov-Based Deep Reinforcement Learning Approach. IEEE Transactions on Vehicular Technology, 72(10), 13360–13373[10.1109/tvt.2023.3271613]
  19. 19.Yilin Xiao, Liang Xiao, Kunpeng Wan, Helin Yang, Yi Zhang, Yi Wu, Yanyong Zhang. (2022). Reinforcement Learning Based Energy-Efficient Collaborative Inference for Mobile Edge Computing. IEEE Transactions on Communications, 71(2), 864–876[10.1109/tcomm.2022.3229033]
  20. 20.Qi Guo, Fengxiao Tang, Nei Kato. (2023). Resource Allocation for Aerial Assisted Digital Twin Edge Mobile Network. IEEE Journal on Selected Areas in Communications, 41(10), 3070–3079[10.1109/jsac.2023.3310065]
  21. 21.Guifu Ma, Xiaowei Wang, Manjiang Hu, Wenjie Ouyang, Xiaolong Chen, Yang Li. (2022). DRL-Based Computation Offloading With Queue Stability for Vehicular-Cloud-Assisted Mobile Edge Computing Systems. IEEE Transactions on Intelligent Vehicles, 8(4), 2797–2809[10.1109/tiv.2022.3225147]
  22. 22.Qi Guo, Fengxiao Tang, Tiago Koketsu Rodrigues, Nei Kato. (2023). Five Disruptive Technologies in 6G to Support Digital Twin Networks. IEEE Wireless Communications, 31(1), 149–155[10.1109/mwc.013.2200296]
  23. 23.Xiao Du, Ting Wang, Qiang Feng, Chenhui Ye, Tao Tao, Lu Wang, Yuanming Shi, Mingsong Chen. (2022). Multi-Agent Reinforcement Learning for Dynamic Resource Management in 6G in-X Subnetworks. IEEE Transactions on Wireless Communications, 22(3), 1900–1914[10.1109/twc.2022.3207918]
  24. 24.Yueyue Dai, Jintang Zhao, Jing Zhang, Yan Zhang, Tao Jiang. (2024). Federated Deep Reinforcement Learning for Task Offloading in Digital Twin Edge Networks. IEEE Transactions on Network Science and Engineering, 11(3), 2849–2863[10.1109/tnse.2024.3350710]
  25. 25.Ta Huu Binh, Do Bao Son, Hiep Vo, Binh Minh Nguyen, Huỳnh Thị Thanh Bình. (2023). Reinforcement Learning for Optimizing Delay-Sensitive Task Offloading in Vehicular Edge–Cloud Computing. IEEE Internet of Things Journal, 11(2), 2058–2069[10.1109/jiot.2023.3292591]
  26. 26.Xin Peng, Zhengke Han, Wenwu Xie, Chao Yu, Peng Zhu, Jian Xiao, Jinxia Yang. (2022). Deep Reinforcement Learning for Shared Offloading Strategy in Vehicle Edge Computing. IEEE Systems Journal, 17(2), 2089–2100[10.1109/jsyst.2022.3190926]
  27. 27.Homa Maleki, Mehmet Başaran, Lütfiye Durak-Ata. (2023). Handover-Enabled Dynamic Computation Offloading for Vehicular Edge Computing Networks. IEEE Transactions on Vehicular Technology, 72(7), 9394–9405[10.1109/tvt.2023.3247889]
  28. 28.Bomin Mao, Xueming Zhou, Jiajia Liu, Nei Kato. (2023). Digital Twin Satellite Networks Toward 6G: Motivations, Challenges, and Future Perspectives. IEEE Network, 38(1), 54–60[10.1109/mnet.2023.3332895]
  29. 29.Xiangjun Zhang, Weiguo Wu, Zhihe Zhao, Jinyu Wang, Song Liu. (2023). RMDDQN-Learning: Computation Offloading Algorithm Based on Dynamic Adaptive Multi -Objective Reinforcement Learning in Internet of Vehicles. IEEE Transactions on Vehicular Technology, 72(9), 11374–11388[10.1109/tvt.2023.3270967]
  30. 30.Liwen Niu, Xianfu Chen, Ning Zhang, Yongdong Zhu, Rui Yin, Celimuge Wu, Yangjie Cao. (2023). Multiagent Meta-Reinforcement Learning for Optimized Task Scheduling in Heterogeneous Edge Computing Systems. IEEE Internet of Things Journal, 10(12), 10519–10531[10.1109/jiot.2023.3241222]
  31. 31.Bishmita Hazarika, Keshav Singh, Anal Paul, Trung Q. Duong. (2023). Hybrid Machine Learning Approach for Resource Allocation of Digital Twin in UAV-Aided Internet-of-Vehicles Networks. IEEE Transactions on Intelligent Vehicles, 9(1), 2923–2939[10.1109/tiv.2023.3335277]
  32. 32.Yunlong Lu, Bo Ai, Zhangdui Zhong, Yan Zhang. (2022). Energy-Efficient Task Transfer in Wireless Computing Power Networks. IEEE Internet of Things Journal, 10(11), 9353–9365[10.1109/jiot.2022.3223690]
  33. 33.Xuehan Chen, Linfeng Luo, Fengxiao Tang, Ming Zhao, Nei Kato. (2024). AIGC-Based Evolvable Digital Twin Networks: A Road to the Intelligent Metaverse. IEEE Network, 38(6), 370–379[10.1109/mnet.2024.3411008]
  34. 34.Lun Tang, Zhangchao Cheng, Jun Dai, Hongpeng Zhang, Qianbin Chen. (2024). Joint Optimization of Vehicular Sensing and Vehicle Digital Twins Deployment for DT-Assisted IoVs. IEEE Transactions on Vehicular Technology, 73(8), 11834–11847[10.1109/tvt.2024.3373175]
  35. 35.Binbin Lu, Bo Fan, Yuan Wu, Liping Qian, Haixia Zhang, Rongxing Lu. (2023). Predictive Computation Offloading and Resource Allocation in DT-Empowered Vehicular Networks. IEEE Transactions on Intelligent Transportation Systems, 25(6), 5474–5487[10.1109/tits.2023.3331885]
  36. 36.Nan Geng, Qinbo Bai, Chenyi Liu, Tian Lan, Vaneet Aggarwal, Yuan Yang, Mingwei Xu. (2023). A Reinforcement Learning Framework for Vehicular Network Routing Under Peak and Average Constraints. IEEE Transactions on Vehicular Technology, 72(5), 6753–6764[10.1109/tvt.2023.3235946]
  37. 37.Lei Zhu, Zhizhong Zhang, Lilan Liu, Linlin Feng, Peng Lin, Yu Zhang. (2023). Online Distributed Learning-Based Load-Aware Heterogeneous Vehicular Edge Computing. IEEE Sensors Journal, 23(15), 17350–17365[10.1109/jsen.2023.3283413]
  38. 38.M. Alam, Komal S. Khan, Abbas Jamalipour. (2023). Multiagent Best Routing in High-Mobility Digital-Twin-Driven Internet of Vehicles (IoV). IEEE Internet of Things Journal, 11(8), 13708–13721[10.1109/jiot.2023.3338020]
  39. 39.Lei Wang, Hongbin Liang, Dongmei Zhao. (2023). Deep-Reinforcement-Learning-Based Computation Offloading and Power Allocation Within Dynamic Platoon Network. IEEE Internet of Things Journal, 11(6), 10500–10512[10.1109/jiot.2023.3327712]

Cited By

  1. 1.Mitra Valizadeh, Miadreza Shafie-khah, Iman Pishkar, Reza Kia, Pegah Mahoor. (2026). Digital twins for energy management: bridging virtual intelligence and physical systems. Applied Energy, 417, 128050[10.1016/j.apenergy.2026.128050]
  2. 2.Lin Kang, Yaqi Wei, Mingxi Ma, Yang Guo, Wenjing Qiao. (2025). Lyapunov-enabled attention-inspired DRL approach for R2X offloading in vehicular edge network. Computer Networks, 274, 111852[10.1016/j.comnet.2025.111852]
  3. 3.Amir Alavi, Hamidreza Alavi. (2026). Digital Twin-Enabled Vehicle Routing in Logistics and Transportation. Sensors, 26(19), 5986[10.3390/s26195986]
  4. 4.Pan Zhao, Liuyuan Chen, Shaoyong Guo, Lei Feng, Wenjing Li, Peng Yu, Weiliang Han. (2026). A Bipartite Graph Reinforcement Learning Approach for Digital Twin-Assisted D2D–LEO Collaborative Computing. IEEE Internet of Things Journal, 13(18), 41755–41771[10.1109/jiot.2026.3705727]
  5. 5.Hao Yang, Li Pan, Shijun Liu. (2026). Learning reward functions via GNNs for multi-agent task placement in edge–cloud LLM services. Journal of Network and Computer Applications, 249, 104457[10.1016/j.jnca.2026.104457]
  6. 6.Hao Yang, Li Pan, Shijun Liu. (2025). Is a Reward Function Necessary? A GNN-Based Distributed Multi-Agent Task Placement Strategy for Edge-Cloud-Based LLM Service. SSRN Electronic Journal[10.2139/ssrn.5472895]
← All publications