Stepwise Resolution Scaling for Low-Cost Convolutional Neural Network Training

Abstract

With the development of artificial intelligence, convolutional neural networks (CNNs) are now widely used in mobile systems such as home appliances and smartphones. However, CNNs have high computational costs for training and inference. Thus, they are typically used on cloud servers or specially designed deep learning accelerators with pre-trained parameters. To enable on-device learning, it is important to reduce the computational costs and memory usage. In this thesis, a stepwise resolution scaling technique is presented to address this problem. It starts by training with a low resolution, and then increasing the resolution, in a stepwise manner, after a predetermined number of epochs. This technique is further improved by using stepwise resolution scaling with blockwise layer freezing. With blockwise layer freezing, the number of active layer blocks is decreased as the training resolution is increased. It is experimentally shown that stepwise resolution scaling with blockwise layer freezing reduces the required computational costs and memory usage with practically no accuracy drop.