On Optimal Power Control for URLLC over a Non-stationary Wireless Channel using Contextual Reinforcement Learning

On Optimal Power Control for URLLC over a Non-stationary Wireless Channel using Contextual Reinforcement Learning

ICC 2022 - IEEE International Conference on Communications

10.1109/ICC45855.2022.9839177

http://dx.doi.org/10.1109/icc45855.2022.9839177

Mohit K. Sharma, Sumei Sun, Ernest Kurniawan, Peng Hui Tan

11 August 2022

Sharma, M. K., Sun, S., Kurniawan, E., & Hui Tan, P. (2022). On Optimal Power Control for URLLC over a Non-stationary Wireless Channel using Contextual Reinforcement Learning. ICC 2022 - IEEE International Conference on Communications. https://doi.org/10.1109/icc45855.2022.9839177

In this work we investigate the design of energy optimal policies for ultra-reliable low-latency communications (URLLC) over a non-stationary wireless channel, using a contextual reinforcement learning (RL) framework. We consider a point-to-point communication system over a piece-wise stationary wireless channel where the Doppler frequency of the channel switches between two distinct values, depending on the underlying state of the channel. To benchmark the performance, first we consider an oracle agent which has a perfect but causal information about the switching instants, and consists of two deep RL (DRL) agents each of which is tasked with optimal decision making in a unique partially stationary environment. Comparing the performance of the oracle agent with the conventional DRL reveals that the performance gain obtained using oracle agent depends on the dynamics of the non-stationary channel. In particular, for a non-stationary channel with faster switching rate the oracle agent results in approximately 15-20% less energy consumption. In contrast, for a channel with slower switching rate the performance of the oracle agent is similar to the conventional DRL agent. Next, for a more realistic scenario when the information about the switching instants for the Doppler frequency of the underlying channel is not available, we model the non-stationary channel as a regime switching process modulated by a Markov process, and adapt the oracle agent by aiding a state tracking algorithm proposed for the regime switching process. Our simulation results show that the proposed algorithm yields a better performance compared to the conventional DRL agent.

Publisher Copyright

This research / project is supported by the A*STAR - 5G-AMSUS-WP1
Grant Reference no. : NA

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

https://oar.a-star.edu.sg/communities-collections/articles/18506

Institute for Infocomm Research