Logistic Regression
Python · Machine Learning (Scikit-Learn)
Theory
Despite its name, logistic regression is a classification model. It computes a weighted sum of the features like linear regression, then passes it through the sigmoid function, which squeezes any number into a probability between 0 and 1. predict_proba(X) returns those probabilities, one column per class, and predict(X) returns class 1 whenever the probability is at least 0.5.
0.5 is only a default threshold. If missing a positive is costly (a fraudulent payment, a sick patient), lower it to catch more positives; if false alarms are costly, raise it. Apply your own with (model.predict_proba(X)[:, 1] >= 0.3).astype(int). The sign of each coefficient shows whether a feature pushes the probability up or down. C controls regularisation (smaller C means a simpler model), and scaling the features helps the solver converge.
Why we use it
It gives well-behaved probabilities, trains fast and is easy to explain, which makes it the standard baseline for yes/no problems such as churn, default or pass/fail, and a common choice in regulated industries.
When to use it
Use it for binary or multi-class classification when a mostly linear boundary is reasonable and you want probabilities. Tune the threshold to the cost of errors. For complex non-linear patterns, try random forests or gradient boosting.
How it works
- Features X and 0/1 labels y
- Weighted sum z = coef x X + intercept
- Sigmoid turns z into a probability from 0 to 1
- Probability >= threshold? Yes: Predict class 1. No: Predict class 0.
- Tune the threshold to the cost of each mistake
- Class predictions with probabilities
Syntax
from sklearn.linear_model import LogisticRegression
model = LogisticRegression(C=1.0, max_iter=1000).fit(X_train, y_train)
model.predict_proba(X_test)[:, 1]
(model.predict_proba(X_test)[:, 1] >= 0.3).astype(int)Example
model = LogisticRegression().fit(X_train, y_train)
prob = model.predict_proba(X_test)[:, 1]
print((prob >= 0.3).astype(int))Key points
- A classifier: outputs probabilities via the sigmoid
- predict uses a 0.5 threshold by default
- Lower the threshold to catch more positives (higher recall)
- Positive coefficients push the probability up
- Scale features; smaller C means stronger regularisation