The use of autoregressive models based on Vector Quantized Variational AutoEncoders (VQ-VAE) recently achieved comparable results versus state-of-the-art Generative Adversarial Networks (GANs). However, autoregressive models have several limitations such as exposure bias and their training objective does not guarantee visual fidelity. To address these limitations, we propose to use Reinforced Adversarial Learning (RAL) based on policy gradient optimization for autoregressive models. By applying RAL, we enable a similar process for training and testing which addresses the exposure bias issue. In addition, visual fidelity has been further optimized with adversarial loss inspired by their strong counterparts: GANs. Due to the slow sampling speed of autoregressive models, we propose partial generation for faster training. RAL also empowers the collaboration between different modules of the VQ-VAE framework. To our best knowledge, the proposed framework is first to enable adversarial learning in autoregressive models. Experiments on synthetic and real-world datasets show improvements over the MLE trained models. The proposed method improves both negative log-likelihood (NLL) and Fréchet Inception Distance (FID) which indicates improvements in terms of visual quality and diversity. The proposed method achieves state-of-the-art results on Celeba for 64x64 image resolution, showing promise for large scale image generation.