Introduction
Logistic Regression is a type of supervised learning algorithm used for classification problems. Unlike linear regression which predicts continuous values, logistic regression is used to estimate the probability that a given input belongs to a certain category.
What is Logistic Regression?
Logistic regression is used when the dependent variable is categorical (usually binary — 0 or 1, Yes or No, True or False). The algorithm uses a logistic function (also called the sigmoid function) to model the probability of a binary outcome:
Sigmoid function: P(y=1) = 1 / (1 + e-(β0 + β1x1 + β2x2 + ... + βnxn))
The output of the sigmoid function is always between 0 and 1, representing the probability of the positive class.
How it Works
- Collect input features (independent variables)
- Apply weights to each input
- Compute the weighted sum (linear combination)
- Pass the sum through the sigmoid function to get probability
- Classify the output based on a threshold (commonly 0.5)
Example
Let’s consider a case where we want to predict whether a student will pass or fail an exam based on hours studied.
Dataset:
| Hours Studied | Pass (1) / Fail (0) |
|---|---|
| 1 | 0 |
| 2 | 0 |
| 3 | 0 |
| 4 | 1 |
| 5 | 1 |
| 6 | 1 |
We train a logistic regression model on this data. Suppose the model learns the function:
P(pass) = 1 / (1 + e-( -4 + 1.5 × hours_studied ))
Now, if a student studies for 3.5 hours:
P(pass) = 1 / (1 + e- ( -4 + 1.5 × 3.5 ))
= 1 / (1 + e-1.25) ≈ 0.777
Since 0.777 > 0.5, the model predicts that the student will pass.
Applications
- Email spam detection
- Credit scoring
- Disease prediction (e.g., cancer diagnosis)
- Marketing response prediction
Advantages
- Simple to implement
- Works well with linearly separable data
- Probabilistic interpretation of outputs
Conclusion
Logistic regression is one of the foundational classification algorithms in machine learning. It is effective, easy to interpret, and widely used for binary classification tasks. With the ability to model probabilities, it gives more insight than just class labels, making it a powerful tool for prediction and decision-making.