Adaptive Convolutional Neural Networks Architecture Search with Filter Shape Pruning

Abstract

This thesis presents a Filter Shape Pruning (FSP) method, which prunes the net- works using the kernel shape obtained by Stripe-Wise Pruning (SWP). FSP prunes the network while preserving the receptive field of the kernel shape, and the effect of this key concept is demonstrated in an ablation study. To obtain an architecture that satisfies the target FLOPs with the FSP method, this thesis proposes the Adaptive Architecture Search (AAS) framework. The AAS framework adaptively searches for the architecture that satisfies the target FLOPs with the layer-wise threshold. The layer-wise threshold is calculated at each iteration using the metric that reflects the filter’s influence on accuracy and FLOPs together. The proposed metric can find filters that generate many FLOPs while less sensitive in accuracy. Comprehensive experimental results demonstrate that the FSP can achieve a higher compression ratio with an acceptable reduction in accuracy. FSP is the only one that reduces FLOPs by more than 80% in VGG-16 on Cifar-10 and more than 60% in ResNet-18 on ImageNet with little accuracy loss.