Optimization for Master-UAV-Powered Auxiliary-Aerial-IRS-Assisted IoT Networks: An Option-Based Multi-Agent Hierarchical Deep Reinforcement Learning Approach
Page view(s)
58
Checked on Aug 10, 2025
Optimization for Master-UAV-Powered Auxiliary-Aerial-IRS-Assisted IoT Networks: An Option-Based Multi-Agent Hierarchical Deep Reinforcement Learning Approach
Optimization for Master-UAV-Powered Auxiliary-Aerial-IRS-Assisted IoT Networks: An Option-Based Multi-Agent Hierarchical Deep Reinforcement Learning Approach
Xu, J., Kang, X., Zhang, R., Liang, Y.-C., & Sun, S. (2022). Optimization for Master-UAV-Powered Auxiliary-Aerial-IRS-Assisted IoT Networks: An Option-Based Multi-Agent Hierarchical Deep Reinforcement Learning Approach. IEEE Internet of Things Journal, 9(22), 22887–22902. https://doi.org/10.1109/jiot.2022.3185799
Abstract:
This article investigates a master unmanned aerial vehicle (MUAV)-powered Internet of Things (IoT) network, in which we propose using a rechargeable auxiliary UAV (AUAV) equipped with an intelligent reflecting surface (IRS) to enhance the communication signals from the MUAV and also leverage the MUAV as a recharging power source. Under the proposed model, we investigate the optimal collaboration strategy of these energy-limited UAVs to maximize the accumulated throughput of the IoT network. Depending on whether there is charging between the two UAVs, two optimization problems are formulated. To solve them, two multi-agent deep reinforcement learning (DRL) approaches are proposed, which are centralized training multi-agent deep deterministic policy gradient (CT-MADDPG) and multi-agent deep deterministic policy option critic (MADDPOC). It is shown that the CT-MADDPG can greatly reduce the complexity of optimization, and the proposed MADDPOC is able to support low-level multi-agent cooperative learning in the continuous action domains, which has great advantages over the existing option-based hierarchical DRL that only supports single-agent learning and discrete actions.
License type:
Publisher Copyright
Funding Info:
This research is supported by core funding from: I2R
Grant Reference no. : NA
This work was supported in part by the National Natural Science Foundation of China under Grants U1801261, by the National Key R&D Program of China under Grant 2018YFB1801105, by the Key Areas of Research and Development Program of Guangdong Province, China, under Grant 2018B010114001, by the Science and Technology Development Fund, Macau SAR, under Grant 0009/2020/A1, by the Central Universities under Grant ZYGX2019Z022 and by the 111 Project under Grant B20064.