Feature Scaling
Python · Machine Learning (Scikit-Learn)
Theory
Features often sit on very different scales: age runs 25-53 while income in rupees runs into lakhs. Models that use distances or gradients, such as KNN, KMeans, SVM, PCA and regularised linear or logistic regression, are dominated by the column with the biggest numbers unless you rescale. StandardScaler subtracts each column's mean and divides by its standard deviation, so every column has mean 0 and standard deviation 1 (a z-score). MinMaxScaler maps each column to the range 0 to 1 with (x - min) / (max - min).
Scalers learn their numbers (mean, std, min, max) in fit, so call fit or fit_transform on the training set only and then transform the test set with the same values. A test value outside the training range simply goes below 0 or above 1. Tree-based models (decision trees, random forests) split on thresholds and do not need scaling.
Why we use it
Without scaling, a column measured in large units silently gets more weight in distance and gradient-based models. Scaling makes features comparable, helps solvers converge and often improves accuracy noticeably.
When to use it
Use StandardScaler as the default for linear models, KNN, SVM, PCA and KMeans; MinMaxScaler when you need a fixed 0-1 range. Skip scaling for tree models. Always put the scaler inside a Pipeline so it is fitted on training folds only.
How it works
- Numeric features on different scales
- Is the model tree-based? Yes: No scaling needed. No: Choose a scaler.
- Need a fixed 0-1 range? Yes: MinMaxScaler. No: StandardScaler (default).
- scaler.fit(X_train) learns mean/std or min/max
- transform X_train and X_test with the same numbers
- Comparable features for the model
Syntax
from sklearn.preprocessing import StandardScaler, MinMaxScaler
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test) # never fit on testExample
scaler = StandardScaler()
X_train_s = scaler.fit_transform(X_train)
X_test_s = scaler.transform(X_test)
print(scaler.mean_, scaler.scale_)Key points
- StandardScaler: (x - mean) / std, giving mean 0 and std 1
- MinMaxScaler: (x - min) / (max - min), giving 0 to 1
- fit on training data only; transform train and test
- Needed for KNN, SVM, KMeans, PCA, regularised linear models
- Tree-based models do not need scaling