Neural Network TransferFunctions
Post date: Jan 20, 2016 3:05:55 PM
the neural network is a tool used for classification and has currently become a hot topic as it has accomplished some advanced classification issues, yet it can be said that a neural net is only as good as its transfer functions!!
A few transfer functions been created in this class which i have created.
Some of the functions have derivatives and some do not.
The class itself doesn't need to be instantiated the EvaluateTransferFunct function allows for the input to return the result to the caller.
Code
Enum TransferFunctionType
none
sigmoid
HyperbolTangent
BinaryThreshold
RectifiedLinear
Logistic
StochasticBinary
Gaussian
Signum
End Enum
This Enum / Type needs to be created as it holds the names of the transfer function types needed by the class.
I have found that this method of creating a type list enables for the select case statement to become and effective tool and a type of decision tree process can be created.
The file which has been uploaded is the class file i have added the exstension .FILE to the end which needs to be removed from the file name to be added to the vb project of your own!
There are many more functions in the class including Gaussian, Rectified and more.
This class can be extended for better gradient descent models and stochastic methods .
Sigmoid
Public Shared Function Sigmoid(ByRef Value As Integer) As Double
'z = Bias + (Input*Weight)
'Output = 1/1+e**z
Return 1 / (1 + Math.Exp(-Value))
End Function
Public Shared Function SigmoidDerivitive(ByRef Value As Integer) As Double
Return Sigmoid(Value) * (1 - Sigmoid(Value))
End Function
TanH
Public Shared Function HyperbolicTangent(ByRef Value As Double) As Double
' TanH(x) = (Math.Exp(x) - Math.Exp(-x)) / (Math.Exp(x) + Math.Exp(-x))
Return Math.Tanh(Value)
End Function
Public Shared Function HyperbolicTangentDerivative(ByRef Value As Double) As Double
HyperbolicTangentDerivative = 1 - (HyperbolicTangent(Value) * HyperbolicTangent(Value)) * Value
End Function