On Optimal Power Control for URLLC over a Non-stationary Wireless Channel using Contextual Reinforcement Learning

Page view(s)
73
Checked on Nov 29, 2024
On Optimal Power Control for URLLC over a Non-stationary Wireless Channel using Contextual Reinforcement Learning
Title:
On Optimal Power Control for URLLC over a Non-stationary Wireless Channel using Contextual Reinforcement Learning
Journal Title:
ICC 2022 - IEEE International Conference on Communications
Keywords:
Publication Date:
11 August 2022
Citation:
Sharma, M. K., Sun, S., Kurniawan, E., & Hui Tan, P. (2022). On Optimal Power Control for URLLC over a Non-stationary Wireless Channel using Contextual Reinforcement Learning. ICC 2022 - IEEE International Conference on Communications. https://doi.org/10.1109/icc45855.2022.9839177
Abstract:
In this work we investigate the design of energy optimal policies for ultra-reliable low-latency communications (URLLC) over a non-stationary wireless channel, using a contextual reinforcement learning (RL) framework. We consider a point-to-point communication system over a piece-wise stationary wireless channel where the Doppler frequency of the channel switches between two distinct values, depending on the underlying state of the channel. To benchmark the performance, first we consider an oracle agent which has a perfect but causal information about the switching instants, and consists of two deep RL (DRL) agents each of which is tasked with optimal decision making in a unique partially stationary environment. Comparing the performance of the oracle agent with the conventional DRL reveals that the performance gain obtained using oracle agent depends on the dynamics of the non-stationary channel. In particular, for a non-stationary channel with faster switching rate the oracle agent results in approximately 15-20% less energy consumption. In contrast, for a channel with slower switching rate the performance of the oracle agent is similar to the conventional DRL agent. Next, for a more realistic scenario when the information about the switching instants for the Doppler frequency of the underlying channel is not available, we model the non-stationary channel as a regime switching process modulated by a Markov process, and adapt the oracle agent by aiding a state tracking algorithm proposed for the regime switching process. Our simulation results show that the proposed algorithm yields a better performance compared to the conventional DRL agent.
License type:
Publisher Copyright
Funding Info:
This research / project is supported by the A*STAR - 5G-AMSUS-WP1
Grant Reference no. : NA
Description:
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
ISBN:

Files uploaded:

File Size Format Action
sh-non-stationary-rl-urllc-harq-camready-v2.pdf 317.58 KB PDF Open