Viewpoint 1
The output of a neuron in an ordinary neural networks connects to n weights in the next layer and the pattern in those weights is projected with intensity x (the output value of the single neuron) into the next layer.
In a fast transform if a single input is made non-zero the basis vector it corresponds to is projected onto the output with intensity x, x being the value of the single non-zero input. And that pattern is typically a highly ordered sine wave or cosine wave or something like that. The parametric activation function then controls the intensity the highly ordered pattern is projected with.
Viewpoint 2:
A random projection maps an input vector to a vector pointing in a different (random) direction.
If you make the random projection adjustable then you can start to map an input set of vectors to a wanted set of output vectors.
If you include adjustable non-linear functions you can perform a sequence of adjustable mapping using the random projections and there is no real difference with a normal neural network.
Using randomly assigned random sign flips before a fast Walsh Hadamard transform results in a random projection.
Altering those sign flips makes the (random) projection adjustable. Since multiplying by a randomly assigned pattern of +1's and -1's is one method of applying sign flips you can make the system more adjustable by multiplying by any numbers between say -1 and +1 for example.
You can introduce non-linear behavior by switching in different "flip" values depending on some condition. This allows you form a sequence of adjustable projections using switching and the fast Walsh Hadamard transform culminating in a neural network.