Filter Pruning by Switching to Neighboring CNNs With Good Attributes

Page view(s)
98
Checked on Jul 19, 2024
Filter Pruning by Switching to Neighboring CNNs With Good Attributes
Title:
Filter Pruning by Switching to Neighboring CNNs With Good Attributes
Journal Title:
IEEE Transactions on Neural Networks and Learning Systems
Publication Date:
18 February 2022
Citation:
He, Y., Liu, P., Zhu, L., & Yang, Y. (2022). Filter Pruning by Switching to Neighboring CNNs With Good Attributes. IEEE Transactions on Neural Networks and Learning Systems, 1–13. https://doi.org/10.1109/tnnls.2022.3149332
Abstract:
Filter pruning is effective to reduce the computational costs of neural networks. Existing methods show that updating the previous pruned filter would enable large model capacity and achieve better performance. However, during the iterative pruning process, even if the network weights are updated to new values, the pruning criterion remains the same. In addition, when evaluating the filter importance, only the magnitude information of the filters is considered. However, in neural networks, filters do not work individually, but they would affect other filters. As a result, the magnitude information of each filter, which merely reflects the information of an individual filter itself, is not enough to judge the filter importance. To solve the above problems, we propose Meta-attribute-based Filter Pruning (MFP). First, to expand the existing magnitude information based pruning criteria, we introduce a new set of criteria to consider the geometric distance of filters. Additionally, to explicitly assess the current state of the network, we adaptively select the most suitable criteria for pruning via a meta-attribute, a property of the neural network at the current state. Experiments on two image classification benchmarks validate our method. For ResNet-50 on ILSVRC-2012, we could reduce more than 50% FLOPs with only 0.44% top-5 accuracy loss.
License type:
Publisher Copyright
Funding Info:
This research is supported by core funding from: A*STAR SERC Central Research Fund
Grant Reference no. : NA

This work was supported by the Australian Research Council (ARC) under Grant DP200100938.
Description:
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works
ISSN:
2162-2388
2162-237X
Files uploaded:

File Size Format Action
tnnls-2020-p-13243-upd.pdf 2.21 MB PDF Open