A Frozen Neural Network

The neural network is a feedforward neural network with full layer-wise connectivity. The hidden layers of the neural network are composed of frozen weights. The network is unfrozen using parametric activation functions as the adjustable part.

The result is speed improvements of 1000 or more scaling with width. Together with reduced parameter cost.