A numerical example 

Let's start to show how Garson's algorithm could be employed in a neural network. First, load all the code available in the following Google Colab (click on the link):

https://colab.research.google.com/drive/1ANZ6eRdPCr7gCGmg4tkgLyFwG_pwLymJ?usp=sharing

Then insert the code related with the Garson's algoritm:

import keras

import numpy as np

# Taken from https://csiu.github.io/blog/update/2017/03/29/day33.html

def garson(A, B):

    """

    Computes Garson's algorithm

    A = matrix of weights of input-hidden layer (rows=input & cols=hidden)

    B = vector of weights of hidden-output layer

    """

    B = np.diag(B)

    # connection weight through the different hidden node

    cw = np.dot(A, B)

    # weight through node (axis=0 is column; sum per input feature)

    cw_h = abs(cw).sum(axis=0)

    # relative contribution of input neuron to outgoing signal of each hidden neuron

    # sum to find relative contribution of input neuron

    rc = np.divide(abs(cw), abs(cw_h))

    rc = rc.sum(axis=1)

    # normalize to 100% for relative importance

    ri = rc / rc.sum()

    return(ri)

# Adapted from https://csiu.github.io/blog/update/2017/03/29/day33.html

def VarImpGarson(model):

  #Computes Garson's algorithm

  #A = matrix of weights of input-hidden layer (rows=input & cols=hidden)

  #B = vector of weights of hidden-output layer

  A = model.layers[0].get_weights()[0]

  B = model.layers[len(model.layers)-1].get_weights()[0]

  varScores = 0

  for i in range(B.shape[1]):

    varScores += garson(A, np.transpose(B)[i])

    print("Importance of input variables: ",

    np.array(varScores).argsort()[-10:][::-1])

  return varScores

Now, it is time to test the functions created.

garson_metrics = VarImpGarson(model)

print(garson_metrics)

Importance of input variables: 

[0 1] 

[0.56102777 0.4389723 ] 

The previous code indicates that the input variable x0 has an importance of 56% and x1 has an importance of 44% in the final output of the neural network.