Embedded Files
In the Evaluating Machine Learning (ML) Models activity you will have the chance to explore the various performance metrics used in evaluating machine learning models for their applicability towards specific tasks and situations. The purpose of this activity is to provide you with a basic understanding of using performance metrics to make determinations about the ‘best’ model to use in various applications.
This Evaluating Machine Learning Models Activity will help you:
This Evaluating Machine Learning Models Activity will help you:
List some of the most commonly used performance metrics for classification machine learning models.
Explain the mathematical equations behind obtaining an array of ML model performance metrics.
Apply multiple ML model algorithms on the same image dataset to compare and contrast the performance of each model.
Understand how performance metrics can be used to determine an optimal ML model for specific applications.
Required Materials/Resources:
Required Materials/Resources:
Access to a computer and large screen (if you want share with others)
A Google account to access Google Colab
Keywords:
Keywords:
Performance - In most applications, performance refers to how well an action completes a set task. In machine learning, the performance of a model refers to how well the model makes predictions for the intended task at hand. The concept of performance in ML relies heavily on the task and the chosen measurements/metrics. There is not one single go-to performance metric used in ML applications, so metrics must be chosen to best suit the application.
Metric - A quantitative measurement of the ability of a machine learning model to complete its given task in regards to specific parameters. A performance metric is not the same as the loss function used in training a model. Loss functions refer to the ‘error’ present at the end of each layer of a model. Instead, performance metrics refer to the ability of the overall model to perform to specific standards.
Background:
Background:
Learning how to evaluate a machine learning model is important since making sure our model is outputting accurate data is important, since if our model is outputting an incorrect answer or bad data it could lead to many other issues that rely on that information it outputs. We can also need to be able to determine if our model is actually learning from its mistakes and the information we provide or only stating the same answer it knows to be correct for the dataset it learned from but is wrong for any new information we show to it, otherwise known as overfitting the model.
In this activity we will be learning how to properly evaluate machine learning models, how to tell if a model is learning from its mistakes by reviewing metrics such as accuracy, the loss function, and other metrics we create that are more specific to the model we are working with to help better analyze our models performance on its given task.
Activity Layout & Instructions
Activity Layout & Instructions
- Preparation
Not applicable for this activity.
- Activity
Read and work through the Evaluating Machine Learning Models - Handout and the corresponding Evaluating Machine Learning Models - Colab Notebook.
Hypothesize which model would be best applicable to a variety of applications and justify this hypothesis with your knowledge of model performance metrics and the metrics obtained in running through the Evaluating Machine Learning Models - Colab Notebook.
Downloadable Content & Links
Downloadable Content & Links
Further Learning Resources
Further Learning Resources
https://www.geeksforgeeks.org/machine-learning/metrics-for-machine-learning-model/
https://medium.com/@mlmind/evaluation-metrics-for-classification-fc770511052d
https://neptune.ai/blog/performance-metrics-in-machine-learning-complete-guide
https://scikit-learn.org/stable/auto_examples/model_selection/plot_precision_recall.html
Page updated
Report abuse