AutoRelax: HW-SW Co-Optimization for Efficient SpGEMM Operations with Automated Relaxation in Deep Learning

Abstract

IEEEWe propose a HW-SW co-optimization technique to perform energy-efficient spGEMM operations for deep learning. First, we present an automated pruning algorithm, named AutoRelax, that allows some level of relaxation to achieve higher compression ratio. Since the benefit of the proposed pruning algorithm may be limited by the sparsity level of a given weight matrix, we present additional steps to further improve its efficiency. Along with the software approach, we also present a hardware architecture for processing sparse GEMM operations to maximize the benefit of the proposed pruning algorithm and sparse matrix format. To validate the efficiency of our co-optimization methodology, we evaluated the proposed method on three benchmarks, language modeling, speech recognition and image classification. As a result, our approach improved on-chip performance of spGEMM operations by 9.5027.57% and achieved energy reductions of 15.3533.28% considering DRAM accesses over other sparse accelerators.