A computational representation of a biological neuron

Neurons in neural network were inspired by neurons in the human brain. Here is a diagram of the anatomy of a brain neuron [1]:

Let’s walk through this diagram step-by-step.

As you can see, neurons in a deep learning model are capable of having synapses that connect to more than one neuron in the preceding layer. Each synapse has an associated weight, which impacts the preceding neuron’s importance in the overall neural network.

Weights are a very important topic in the field of deep learning because adjusting a model’s weights is the primary way through which deep learning models are trained. You’ll see this in practice later on when we build our first neural networks from scratch.

Once a neuron receives its inputs from the neurons in the preceding layer of the model, it adds up each signal multiplied by its corresponding weight and passes them on to an activation function, like this [1]:

Translating this into a mathematical prescription, one assigns to the input cells the numbers x1, x2, ..., xn (input data point). The strength of the synaptic connections is controlled with the weights wi. Then the combined signal is defined as the weighted sum [2]:

When the combined signal s is larger than the bias (threshold), the nucleus fires. i.e. the signal passed along the axon is 1. in the opposite case, the generated signal value is 0 (no firing). This is precisely what we need to mimic the biological prototype.

For more details on creating and manipulating activation functions, please see [2, 3]. An example of activation function is given in [4] and is as th next figure. Observe that it resembles to the curve provided by the Logistic Regression.

Combinations of neurons and neural network topologies

As described in [5] neurons could be combined to create a neural network as shown in the next figure:

As shown by [8], a variety of neural network topologies are possible as shown in the next figure.

A small numerical example

Neural networks could be created from scratch as done in [9], but a much more simpler alternative is to use the Python Package Keras [10]. The next numerical example employs this package in a numerical example of binary classification that had been already addressed in this track.

The next link provides a Python code to read the data and could be used as an initial base to create neural networks tests:

https://colab.research.google.com/drive/1f8_LkwYJ6TwJfGo8m62jDbAAke76dcS0?usp=sharing 

The previous code could be added with the following code to create and configure a neural network in which has input layer has 60 neurons, the hidden layer has 32 neurons, in both layers all neurons employ a 'relu' activation function, and the output layer has just 1 neuron with a 'sigmoid' activation function which is the most appropriate for binary classification. For a problem with multiple labels, the variable could be transformed into an array of binary values as described in [6] and in the next figure.