GENERALIZATION IN A PERCEPTRON WITH A SIGMOID TRANSFER-FUNCTION

Abstract

Learning of layed neural networks is studied using the methods of statistical mechanics. Networks are trained from examples using the Gibbs algorithm. We consider perceptron learning with a sigmoid transfer function. Ising perceptrons, with weights constrained to be discrete, exhibit sudden learning at low temperatures within the annealed approximation. There is a first order transition from a state of poor generalization to a state of perfect generalization. The transition becomes continuous at high temperatures. The analytic results show a good agreement with the computer simulations