Embedded Files
This website presented our study results about the test oracle for deep learning application
1. Automated Test Oracle Generation Approaches
1. Automated Test Oracle Generation Approaches
1.1. External-Reference-Based Test Oracle
1.1. External-Reference-Based Test Oracle
Differential Testing Approaches and Criterion of Oracle's Fault Judgement.
- Reject Differential (RejDiff): Any model has a different output with the model to be tested.
- Majority Voting (MajVot): Less than half of the models in the set predict the same output as the tested model.
- Plurality Voting (PluVot): The prediction result of the tested model is not the same as the most occurrence result in the set.
- Weighted Voting (WeightVot): Weighted voting result is not the same with the tested model.
Differential Testing Configuration.
- SDS: Sample at the different training stages from the same model.
- SAT: Train multiple models with the same architecture.
- DAT: Train multiple models with the different architecture
- MM: Mutate the testing model followed the instructions from DeepMutation.
- AD: Activate the dropout layer randomly as the new model
Differential architecture used for image classification
Differential architecture used for natural language processing
1.2. Internal-Attribute-Based Test Oracle
1.2. Internal-Attribute-Based Test Oracle
2. Evaluation Metrics
2. Evaluation Metrics
2.1. Effectiveness Metrics
2.1. Effectiveness Metrics
- Positive Predictive Value Increase(PPV):
- True Positive Rate Increase(TPR):
- True Negative Rate Increase(TNR):
- Negative Predictive Value Increase(NPV):
- Accuracy Increase(A):
2.2. Robustness Metrics
2.2. Robustness Metrics
3. Deep Learning Application
3. Deep Learning Application
3.1. Image Classification
3.1. Image Classification
We choose CIFAR-10 as our study application. The experiment results are showed in Image Classification.
3.2. Natural Language Processing
3.2. Natural Language Processing
We choose IMDB as our study application. The experiment results are showed in Natural Language Processing
Page updated
Google Sites
Report abuse