[TBD]Role of linear regression in current data science

Introduction

Laymen explanation

Linear regression is perhaps one of the most well known and well understood algorithms in statistics and machine learning.

Technical explanation

Linear regression is a linear model, e.g. a model that assumes a linear relationship between the input variables (x) and the single output variable (y). More specifically, that y can be calculated from a linear combination of the input variables (x).

Mathematical interpretation

When there is a single input variable (x), the method is referred to as simple linear regression. When there are multiple input variables, literature from statistics often refers to the method as multiple linear regression or multivariate linear regression.

Advantages

It is easy to implement and understand

??

Varieties of regression models

Linear regression

Logistic regression

Why it is linear model(verify...)https://stats.stackexchange.com/questions/93569/why-is-logistic-regression-a-linear-classifier??

Support vector machine

SVM or Support Vector Machine is a linear model for classification and regression problems. It can solve linear and non-linear problems and work well for many practical problems(Refer here)

Polynomial regression

Why it is linear model(verify...)https://online.stat.psu.edu/stat462/node/158/??

Linear regression role for classification problem

Linear regression is not safe to apply for classification problem. Its due to its linearity nature. In the available data, result may be good, however it often lack generalization capability and so, with new data it can misclassify.

Can we use vertical line for above classification problem?

Can logistic regression solves it?

What if data-points are aligned over a vertical line

What to do?

How to know if dataset can be modelled using linear regression

If you can draw a line or hyperplane that can separate those points into two classes, then the data is separable. If not, then it may be separated by a hyperplane in higher dimensions. Still if any of the hyperplanes could not seperate them, then the data is termed as non linearly separable data.

If the data have multiple variables, it's impossible to visualize on a plot. In this case, convex hull algorithm can help(verify it)

Role in machine learning

Use in neural networks

Point to remember

Generally speaking, in Machine Learning and before running any type of classifier, it is important to understand the data we are dealing with to determine which algorithm to start with, and which parameters we need to adjust that are suitable for the task. This brings us to the topic of linear separability and understanding if our problem is linear or non-linear.

Reference

https://machinelearningmastery.com/linear-regression-for-machine-learning/

https://en.wikipedia.org/wiki/Linear_regression

https://youtu.be/JgAcjo8cPLs?t=302

https://stats.stackexchange.com/questions/175771/overfitting-with-linear-classifiers

https://stats.stackexchange.com/questions/57685/line-of-best-fit-linear-regression-over-vertical-line

https://stats.stackexchange.com/questions/22381/why-not-approach-classification-through-regression

https://towardsdatascience.com/https-medium-com-pupalerushikesh-svm-f4b42800e989

https://sebastianraschka.com/faq/docs/logistic_regression_linear.html

https://towardsdatascience.com/https-medium-com-pupalerushikesh-svm-f4b42800e989

https://www.quora.com/How-can-I-know-whether-my-data-is-linearly-separable

https://www.tarekatwan.com/index.php/2017/12/methods-for-testing-linear-separability-in-python/

https://www.reddit.com/r/learnmachinelearning/comments/ce7pfb/how_could_i_know_if_data_is_linear_separable/

https://youtu.be/c8AvwQeSRdQ