After completing this lesson, you will be able to:
Machine Learning enables computers to learn patterns from historical data and make predictions on new data. One of the most fundamental supervised learning techniques is Regression, which is used to predict numerical values such as sales, prices, demand, temperature, and revenue.
Regression models help organizations understand relationships between variables and estimate future outcomes based on historical observations. Businesses use regression models for forecasting, budgeting, pricing strategies, financial planning, and risk analysis.
Among all regression algorithms, Linear Regression and Logistic Regression are the most widely used. Although both contain the word “Regression,” they solve different business problems. Linear Regression predicts continuous numerical values, whereas Logistic Regression predicts the probability of a categorical outcome.
In this lesson, you will first learn the fundamentals of Linear Regression before exploring Logistic Regression in the next part.
Regression is a supervised Machine Learning technique that predicts a continuous numerical value based on one or more input variables.
Regression algorithms learn the relationship between independent variables (features) and a dependent variable (target). Once trained, the model can estimate values for new observations.
Regression is commonly used when the target variable is a number rather than a category.
Examples include:
Regression and Classification are the two major categories of supervised learning.
| Regression | Classification |
|---|---|
| Predicts numerical values. | Predicts categories or classes. |
| Output is continuous. | Output is discrete. |
| House Price Prediction | Email Spam Detection |
| Sales Forecasting | Fraud Detection |
| Revenue Prediction | Customer Churn Prediction |
Linear Regression is one of the simplest and most widely used Machine Learning algorithms. It models the relationship between an independent variable (X) and a dependent variable (Y) using a straight line.
The objective of Linear Regression is to find the best-fitting line that minimizes the difference between actual values and predicted values.
Once the relationship has been learned, the model can predict future numerical values.
Simple Linear Regression uses one independent variable to predict one dependent variable.
Example:
Multiple Linear Regression uses two or more independent variables to predict a dependent variable.
Example:
Predicting house prices using:
Using multiple features often improves prediction accuracy because the model considers more information.
The mathematical equation of Linear Regression is:
Y = β₀ + β₁X + ε
Where:
The slope indicates how much the target variable changes for every one-unit increase in the independent variable.
The algorithm attempts to draw the best possible straight line through the data points.
The objective is to minimize the prediction error between the actual values and the predicted values.
The difference between the actual value and the predicted value is called the Residual Error.
Linear Regression finds the line with the smallest total residual error using the Least Squares Method.
Linear Regression performs best when certain assumptions are satisfied.
The relationship between the independent and dependent variables should be approximately linear.
Each observation should be independent of the others.
The variance of residual errors should remain approximately constant across all predicted values.
The residual errors should follow a normal distribution.
Independent variables should not be highly correlated with one another in Multiple Linear Regression.
The following example predicts sales based on advertising expenditure.
import pandas as pd
from sklearn.linear_model import LinearRegression
data = {
'Advertising':[10,20,30,40,50],
'Sales':[25,40,55,70,85]
}
df = pd.DataFrame(data)
X = df[['Advertising']]
y = df['Sales']
model = LinearRegression()
model.fit(X, y)
print("Slope:", model.coef_[0])
print("Intercept:", model.intercept_)
prediction = model.predict([[35]])
print("Predicted Sales:", prediction)
The model learns the relationship between advertising expenditure and sales and predicts future sales for new advertising budgets.
A retail company wants to estimate monthly sales based on advertising expenditure.
The Data Analytics team collects historical data containing advertising budgets and monthly sales figures.
Using Linear Regression, the model identifies the relationship between advertising investment and revenue. Management uses these predictions to allocate marketing budgets more effectively and improve sales forecasting.
Continue to Part 2, where you will learn Logistic Regression, the Sigmoid Function, Binary Classification, Decision Boundaries, Python implementation using Scikit-learn, and the differences between Linear and Logistic Regression.
Logistic Regression is a supervised Machine Learning algorithm used for classification problems. Unlike Linear Regression, which predicts continuous numerical values, Logistic Regression predicts the probability that an observation belongs to a particular class.
The predicted probability ranges from 0 to 1. This probability is then converted into a class label such as Yes/No, True/False, or 1/0.
Although its name contains the word “Regression”, Logistic Regression is actually a classification algorithm.
Logistic Regression is appropriate when the target variable has two possible outcomes.
Common examples include:
Logistic Regression first calculates a weighted combination of the input features, similar to Linear Regression.
Instead of producing any numerical value, it passes the result through a mathematical function called the Sigmoid Function.
The Sigmoid Function converts any real number into a probability between 0 and 1.
The Sigmoid Function is represented by the following equation:
P(Y=1)=1 / (1 + e-(β₀ + β₁X))
Where:
The output is always between 0 and 1.
Logistic Regression predicts probabilities rather than exact categories.
For example:
| Predicted Probability | Interpretation |
|---|---|
| 0.10 | 10% chance of purchase |
| 0.45 | 45% chance of purchase |
| 0.82 | 82% chance of purchase |
| 0.98 | 98% chance of purchase |
After predicting probabilities, Logistic Regression converts them into class labels using a decision threshold.
The most common threshold is 0.50.
| Probability | Prediction |
|---|---|
| 0.82 | Yes (Class 1) |
| 0.70 | Yes (Class 1) |
| 0.48 | No (Class 0) |
| 0.15 | No (Class 0) |
If the predicted probability is greater than or equal to 0.50, the model predicts the positive class. Otherwise, it predicts the negative class.
Logistic Regression is based on the concepts of Odds and Log Odds (Logit).
Odds compare the probability that an event occurs to the probability that it does not occur.
Odds = P / (1 − P)
The logarithm of the odds is called the Logit Function, which enables Logistic Regression to model binary outcomes effectively.
import pandas as pd
from sklearn.linear_model import LogisticRegression
data = {
'Age':[22,25,30,35,40,45],
'Purchased':[0,0,0,1,1,1]
}
df = pd.DataFrame(data)
X = df[['Age']]
y = df['Purchased']
model = LogisticRegression()
model.fit(X,y)
prediction = model.predict([[32]])
probability = model.predict_proba([[32]])
print("Prediction:", prediction)
print("Probability:", probability)
This model predicts whether a customer is likely to purchase a product based on age.
| Feature | Linear Regression | Logistic Regression |
|---|---|---|
| Purpose | Predict numerical values | Predict categories |
| Problem Type | Regression | Classification |
| Output | Continuous Numbers | Probability (0–1) |
| Equation | Straight Line | Sigmoid Curve |
| Examples | Sales Forecasting | Spam Detection |
| Prediction | Revenue, Price | Yes/No |
A telecommunications company wants to reduce customer churn.
The Data Science team trains a Logistic Regression model using customer demographics, monthly usage, billing history, and customer support interactions.
The model predicts the probability that each customer will cancel their subscription.
Customers with a high churn probability receive personalized retention offers, helping the company reduce customer loss and improve long-term profitability.
Continue to Part 3, where you will learn how to evaluate Regression and Classification models using MAE, MSE, RMSE, R² Score, Confusion Matrix, Accuracy, Precision, Recall, F1 Score, along with a complete case study, FAQs, lesson summary, and key takeaways.
After training a Linear Regression model, it is important to measure how accurately it predicts numerical values. Model evaluation helps determine whether the model is suitable for solving real-world business problems.
Several evaluation metrics are commonly used for regression models.
Mean Absolute Error (MAE) measures the average absolute difference between actual values and predicted values.
Formula:
MAE = Σ |Actual − Predicted| / n
A lower MAE indicates better prediction accuracy.
Business Example:
If a sales forecasting model has an MAE of ₹500, the model’s predictions differ from actual sales by an average of ₹500.
Mean Squared Error (MSE) measures the average squared prediction error.
Formula:
MSE = Σ (Actual − Predicted)² / n
Because the errors are squared, larger prediction errors receive greater penalties.
Root Mean Squared Error (RMSE) is the square root of MSE.
Formula:
RMSE = √MSE
RMSE is expressed in the same units as the target variable, making it easier to interpret.
R² Score measures how well the independent variables explain the variation in the dependent variable.
The value ranges from 0 to 1.
| R² Value | Interpretation |
|---|---|
| 1.00 | Perfect Prediction |
| 0.90 | Excellent Model |
| 0.75 | Good Model |
| 0.50 | Moderate Model |
| 0.00 | No Predictive Power |
A higher R² score generally indicates that the model explains more of the variation in the target variable.
Unlike Linear Regression, Logistic Regression predicts categories rather than numerical values. Therefore, different evaluation metrics are required.
A Confusion Matrix summarizes the prediction results of a classification model.
| Predicted Positive | Predicted Negative | |
|---|---|---|
| Actual Positive | True Positive (TP) | False Negative (FN) |
| Actual Negative | False Positive (FP) | True Negative (TN) |
The Confusion Matrix forms the basis for several important classification metrics.
Accuracy measures the proportion of correct predictions.
Formula:
Accuracy = (TP + TN) / Total Predictions
Accuracy is useful when classes are balanced.
Precision measures how many predicted positive cases are actually positive.
Formula:
Precision = TP / (TP + FP)
Precision is important when false positives are costly.
Example: Fraud Detection.
Recall measures how many actual positive cases are correctly identified.
Formula:
Recall = TP / (TP + FN)
Recall is important when missing positive cases is expensive.
Example: Cancer Detection.
F1 Score combines Precision and Recall into a single evaluation metric.
Formula:
F1 Score = 2 × (Precision × Recall)
-------------------------
Precision + Recall
F1 Score is particularly useful when dealing with imbalanced datasets.
| Problem Type | Common Metrics |
|---|---|
| Linear Regression | MAE, MSE, RMSE, R² |
| Logistic Regression | Accuracy, Precision, Recall, F1 Score, Confusion Matrix |
An online retail company wants to improve both sales forecasting and customer retention.
The Data Analytics team develops two Machine Learning models.
Linear Regression Model
Logistic Regression Model
By combining both regression models, the company improves revenue forecasting, reduces customer churn, and increases profitability through data-driven decision-making.
Linear Regression and Logistic Regression are two of the most fundamental algorithms in supervised Machine Learning. Linear Regression is designed to predict continuous numerical values such as sales, prices, and demand, while Logistic Regression predicts categorical outcomes such as customer churn, fraud, or loan approval. Understanding these algorithms, their assumptions, evaluation metrics, and practical business applications provides a strong foundation for solving predictive analytics problems. These techniques are widely used across finance, healthcare, marketing, retail, manufacturing, and many other industries.
Linear Regression predicts continuous numerical values, whereas Logistic Regression predicts categorical outcomes by estimating probabilities.
Linear Regression should be used when the target variable is continuous, such as predicting sales, revenue, prices, or demand.
Logistic Regression is suitable for binary classification problems such as spam detection, fraud detection, customer churn prediction, and loan approval.
Common metrics include Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared Error (RMSE), and R-squared (R²).
Classification models are commonly evaluated using Accuracy, Precision, Recall, F1 Score, and the Confusion Matrix.
In the next lesson, you will learn Decision Trees and Random Forest, two powerful Machine Learning algorithms used for both classification and regression. You will explore how decision trees split data, how random forests improve prediction accuracy, and how these models are applied to solve complex business problems.