Training Neural Networks

In this page you can find the instructions on how to train the neural networks to learn to classify partially symbolic structures as well as the network precision and recall performing such a task. No the that the folder jpf-symbc-bounded/neuralnetwork/classifiers/keras already contains the trained and evaluated models from which the experiments on the paper were computed.

Training Data

First, download the training sets of the case study you want to analyze:

• • AvlTree

  • BinTree

  • BinomialHeap

  • TreeSet

Then uncompress the downloaded file in the folder jpf-symbc-bounded/neuralnetwork/instances. If you download the training set for a case study, once you uncompress the file, for each scope k you will find the instances produced by each instance generation mechanism. For instance, for the AvlTree case study you will find:

• avltree-scopek-LI.data: contains the instances produced with symbolically running the repOK method using LI as basis.

• avltree-scopek-BB.data: contains the instances produced with the builders based method.

Training Neural Networks

Assuming that you want to generate a model for the case AvlTree using a scope of 8, the instructions are as follows:

• 1. Go to the folder jpf-symbc-bounded/neuralnetwork

  2. Run the gen-keras-model script which takes the dataset, the output file name and the epochs for training:

     • python3 gen-keras-model.py instances/avltree-scope8-LI.data classifiers/keras/ avltree-scope8-LI 100 (for LI based instances)

     • python3 gen-keras-model.py instances/avltree-scope8-BB.data classifiers/keras/ avltree-scope8-BB 100 (for builders based instances)

Go the page training results to see detailed results of the training phase.

Models Evaluation with 5-fold Cross Validation

In order to perform 5-fold cross validation as described in the paper, you will need to go to the folder jpf-symbc-bounded/neuralnetwork and run the script model-cross-validation.py which takes a dataset, a model and the epochs number. For instance, for the AvlTree case study with scope 3, run the following command:

• python3 model-cross-validation.py instances/avltree-scope3-LI.data classifiers/keras/avltree-scope3-LI.h5 100

which will produce the following output containing the mean values for the considered metrics:

Cross Validation Report

-----------------------

Accuracy mean: 99.98% - Std: (+/- 0.02%)

Precision mean: 100.00% - Std: (+/- 0.00%)

Recall mean: 99.82% - Std: (+/- 0.26%)

F1-score mean: 1.00 - Std: (+/- 0.00%)

Total K-fold time: 538.21

Notice that the input dataset and the input model must be files of the same scope k.