Learning to Prune Deep Neural Networks via Reinforcement Learning

Learning to Prune Deep Neural Networks via Reinforcement Learning
Title:
Learning to Prune Deep Neural Networks via Reinforcement Learning
Other Titles:
7th ICML Workshop on Automated Machine Learning
DOI:
Keywords:
Publication Date:
18 July 2020
Citation:
Abstract:
This paper proposes PuRL - a deep reinforcement learning (RL) based algorithm for pruning neural networks. Unlike current RL based model compression approaches where feedback is given only at the end of each episode to the agent, PuRL provides rewards at every pruning step. This enables PuRL to achieve sparsity and accuracy comparable to current state-of-the-art methods, while having a much shorter training cycle. PuRL achieves more than 80% sparsity on the ResNet-50 model while retaining a Top-1 accuracy of 75.37% on the ImageNet dataset. Through our experiments we show that PuRL is also able to sparsify already efficient architectures like MobileNet-V2. In addition to performance characterisation experiments, we also provide a discussion and analysis of the various RL design choices that went into the tuning of the Markov Decision Process underlying PuRL. Lastly, we point out that PuRL is simple to use and can be easily adapted for various architectures.
License type:
http://creativecommons.org/licenses/by-nc-nd/4.0/
Funding Info:
Agency for Science, Technology and Research (A*STAR) under its AME Programmatic Funds (Project No.A1892b0026 and No.A19E3b0099)
Description:
ISBN:

Files uploaded:

File Size Format Action
learning-to-prune-deep-neural-networks.pdf 235.68 KB PDF Open