Storage capacity of a fully-connected parity machine with continuous weights

Abstract

We study a fully-connected parity machine with K hidden units for continuous weights. The geometrical structure of the weight space of this model is analysed in terms of the volumes associated with the internal representations of the training set. By examining the asymptotic behaviour of order parameters in the large K limit, we find the maximum number ru,, the storage capacity, of patterns per input unit to be K In K/ln2 up to leading order, which saturates the mathematical bound given by Mitchison and Durbin. Unlike the committee machine, the storage capacity per weight remains unchanged compared with the corresponding tree-like architecture.