An efficient parallel block backpropagation learning algorithm in transputer-based mesh-connected parallel computers

Abstract

Learning process is essential for good performance when a neural network is applied to a practical application. The backpropagation algorithm [1] is a well-known learning method widely used in most neural networks. However. since the backpropagation algorithm is time-consuming, much research have been done to speed up the process. The block backpropagation algorithm. which seems to be more efficient than the backpropagation, is recently proposed by Coetzee in [2]. In this paper, we propose an efficient parallel algorithm fur the block backpropagation method and its performance model in mesh-connected parallel computer systems. The proposed algorithm adopts master-slave model for weight broadcasting and data parallelism for computation of weights. In order to validate our performance model. a neural network is implemented for printed character recognition application in the TiME [3] which is a prototype parallel machine consisting of 32 transputers connected in mesh topology. It is shown that speedup by our performance model is very close to that by experiments.