Selecting right machine learning algorithm for your problem

Introduction

Laymen explanation

We have to choose suitable Machine Learning algorithm depending on the problem statement and the dataset and no model is better algorithm compared to another in absolute sense. If you are interested to know selection approach, then this document helps

Technical explanation

Machine learning is part art and part science. When you look at machine learning algorithms, there is no one solution or one approach that fits all. There are several factors that can affect your decision to choose a machine learning algorithm.

Brute force approach for ML algorithm selection

How do you know what machine learning algorithm to choose for your problem? Why don’t we try all the machine learning algorithms or some of the algorithms which we consider will give good accuracy. If we apply each and every algorithm it will take a lot of time. So, it is better to apply a technique to identify the algorithm that can be used.

Factors affecting ML algorithm decision

• • Type of problem - For example, forecasting, classification

  • Domain of problem - For example speech recognition domain

  • Dataset availability - For example, is labelled data available, but not in enough size

  • Dataset visualisation - For example, visualisation of Inter class distribution

  • Real life cost of error - For example, cancer detection system has high cost of misclassification. For example, auto-pilot vehicle needs very high accuracy.

  • Your finance to afford costly resources - For example, computing resource

  • Time to market - For example, long training time is not desirable for your business

Useful steps for deciding ML algorithm

Identify type of problem

It can be

• • Regression problem

  • Classification problem

  • Clustering problem

For example, Some ML algorithms (such as Random Forest classifiers or naive Bayes classifiers) are capable of handling multiple classes directly and Others (such as Support Vector Machine classifiers or Linear classifiers) are strictly binary classifiers. Refer here for the detail.

Look at problem domain

For example speech recognition domain problem should look for NLP based algorithm (for example BERT)

Look at available data

It can be

• • Labelled data is not available

  • Enough Labelled Data is available

  • Not enough labelled data is available

Visualize data

Interclass data distribution helps to identify choice of ML model. Below example dataset can't be classified by any linear ML model and so, non-linear ML models are needed.

Look at Real life cost of error

Theoretically, best accuracy is good. However, due to tradeoff related to resources and time to market, one need to consider the real nature of problem at hand. If tradeoff allows to meet these constraints at little less accuracy (of-course upto allowed threshold of error cost), then sub-optimal solution is allowed.

A nice example is the travelling salesman problem. Here sub-optimal solution is very popular.

Look at computing resources

It can be

• • Good amount of processing with GPU/DPU availability

  • Computing resource availability is moderate

Look at business need

This is also a constraint. However, this constraint must be applied considering long run vision of the solution not the short term need. Also Choosing the right ML model can help to address it without compromising business needs.

Clustering criteria

This is applicable for clustering problem.

Given the dataset, it is possible to create more than one cluster. So, based on clustering criteria, right ML algorithm should be selected. Note that a clustering algorithm may not be have capability to generate all types clusters.

Factors affecting training time

• • More the number of features, more training time

  • Hyperparameter tuning helps to reduce training time

  • Data scaling helps to reduce training time

  • More Non-linearity of problem, more training time

  • Statistical based training is relatively faster than neural networks based training. However accuracy is relatively better for neural networks for many cases (for example, image classification)

Point to remember

Use non-linear ML algorithm only if it can't be solved using linear ML algorithm. Note that non-linear algorithms has lots of cost involved (for example, training time, resource need, difficult to understand by newcomers).

Reference

https://intellipaat.com/community/49991/how-will-you-select-suitable-machine-learning-algorithm-for-a-problem-statement

https://hackernoon.com/choosing-the-right-machine-learning-algorithm-68126944ce1f

https://analyticsindiamag.com/how-to-choose-the-best-machine-learning-algorithm-for-a-particular-problem/

https://www.kdnuggets.com/2020/05/guide-choose-right-machine-learning-algorithm.html

https://medium.com/@aravanshad/how-to-choose-machine-learning-algorithms-9a92a448e0df

https://www.geeksforgeeks.org/choosing-a-suitable-machine-learning-algorithm/

https://youtu.be/LxjL_yLaFS8?t=1514

https://images.app.goo.gl/kyDXbRUkBLQgmvrt9

https://coursera.org/share/7f8a9758962ed466607e29fd080794e1

https://www.amazon.in/Hands-Machine-Learning-Scikit-Learn-TensorFlow/dp/1491962291

https://sites.google.com/site/jbsakabffoi12449ujkn/home/machine-intelligence/handling-multi-class-classification-in-machine-learning