Regression

REGRESSION OVERVIEW

Regression is a type of supervised machine learning used when the output (target) is a continuous value, such as predicting a person’s weight, house price, or anxiety severity score. The goal of a regression algorithm is to find a mathematical relationship between the input features (like age, sleep hours, stress level) and the target variable.

The most commonly used regression model is linear regression, which assumes that the relationship between inputs and the output can be represented by a straight line (or a flat plane in higher dimensions). It learns this line by adjusting the feature weights to minimize the error specifically, the difference between predicted and actual values using a method called Ordinary Least Squares (OLS).

Once trained, the regression model can take new input data and use the learned equation to predict a continuous numeric outcome. It’s widely used for forecasting, trend analysis, and risk modeling in fields like finance, healthcare, and social sciences.

QUESTIONS

Both linear and logistic regression involve computing a weighted sum of input features and coefficients. They use similar model structures and are trained using optimization techniques. However, linear regression predicts continuous values, while logistic regression predicts probabilities for categorical outcomes. Linear regression uses the least squares method, while logistic regression uses maximum likelihood estimation. Logistic regression also includes a non-linear transformation (sigmoid) to constrain outputs between 0 and 1.

(d) Does logistic regression use the Sigmoid function? Explain.

Yes, logistic regression relies on the sigmoid function to convert raw linear scores into probabilities. The sigmoid function maps any real-valued number into a range between 0 and 1, which is suitable for binary classification tasks. It creates an S-shaped curve that helps model the probability of the positive class. This transformation is essential because it makes the model’s outputs interpretable as probabilities. Without the sigmoid function, logistic regression would behave like linear regression and predict unbounded values.

(a) Define and explain linear regression.

Linear regression is a supervised machine learning algorithm used to model the relationship between one or more independent variables and a continuous dependent variable. It assumes this relationship is linear meaning the change in output is proportional to the change in input. The model tries to fit a straight line (or plane in multiple dimensions) to minimize the residual errors between predicted and actual values. It uses a technique called Ordinary Least Squares (OLS) to minimize the sum of squared errors. Linear regression is simple, interpretable, and widely used in predictive analytics.

(b) Define and explain logistic regression.

Logistic regression is a classification algorithm used to predict binary outcomes (such as yes/no, 0/1). Instead of predicting a continuous value, it estimates the probability that a given input belongs to a particular class. It does this by applying a sigmoid (logistic) function to the output of a linear model, which squashes values into a range between 0 and 1. The decision boundary is usually set at 0.5 values above are classified as 1, below as 0. Logistic regression is commonly used in fields like healthcare, marketing, and social sciences for risk prediction or binary outcomes.

(e) Explain how maximum likelihood is connected to logistic regression.

Logistic regression is trained using a method called maximum likelihood estimation (MLE). Instead of minimizing squared errors like linear regression, it tries to find the set of parameters (weights) that maximize the probability of correctly predicting the observed labels. This is done by treating the output of the logistic (sigmoid) function as a probability and calculating the likelihood of all training examples. The model then adjusts its weights to maximize this likelihood. MLE provides a principled, probabilistic way of fitting the logistic regression model to the data. It ensures that the predicted probabilities match the actual distribution of outcomes as closely as possible.

Data (LR)

Page updated

Google Sites

Report abuse