Linear Regression
Python · Machine Learning (Scikit-Learn)
Theory
Linear regression predicts a number as a weighted sum of the features plus a constant: price = coef x area + intercept. fit finds the coefficients that minimise the sum of squared errors on the training data. Each coefficient (model.coef_) says how much the prediction changes when that feature rises by one unit with the others held fixed, and model.intercept_ is the prediction when all features are 0.
Regression is judged on held-out data with error metrics. MAE (mean absolute error) is the average size of a miss in the target's units, such as 1.1 lakh. RMSE (root mean squared error) squares errors before averaging, so large misses count more; compute it as np.sqrt(mean_squared_error(y, pred)). R-squared (r2_score) is the share of variance explained: 1 is perfect, 0 is no better than predicting the mean, and it can go negative.
Why we use it
It is fast, needs little data and is easy to explain: every coefficient has a direct meaning. It is the baseline every regression problem should start with, and its metrics tell you how far off predictions typically are.
When to use it
Use it for numeric targets with roughly straight-line relationships, such as price from area or sales from ad spend. For curved relationships or interactions try trees or random forests; with many correlated features use Ridge or Lasso.
How it works
- Numeric target y and features X
- Split into train and test
- LinearRegression().fit(X_train, y_train) minimises squared error
- Read coef_ and intercept_
- predict(X_test) and compute MAE, RMSE, R-squared
- Errors acceptable and residuals patternless? Yes: Use or deploy the model. No: Try features, Ridge or tree models.
- Numeric predictions
Syntax
from sklearn.linear_model import LinearRegression
model = LinearRegression().fit(X_train, y_train)
model.coef_, model.intercept_
pred = model.predict(X_test)
mean_absolute_error(y_test, pred) r2_score(y_test, pred)
np.sqrt(mean_squared_error(y_test, pred))Example
model = LinearRegression().fit(X_train, y_train)
pred = model.predict(X_test)
print(mean_absolute_error(y_test, pred), r2_score(y_test, pred))Key points
- Prediction = sum of coef x feature + intercept
- coef_ is the change per one unit of a feature
- MAE is the typical miss in the target's own units
- RMSE punishes large errors more than MAE
- R-squared: 1 is perfect, 0 is no better than the mean