Feature Scaling
Python · Advanced Python & AI
Theory
Feature scaling puts numeric features on comparable ranges. Without it, a feature measured in lakhs (salary) can overwhelm one measured in single digits (experience) simply because its numbers are bigger. StandardScaler subtracts each column's mean and divides by its standard deviation, so every column ends up with mean 0 and standard deviation 1 (a z-score). MinMaxScaler instead squeezes each column into the range 0 to 1.
In the diagram, experience [1, 3, 5] and salary [40000, 60000, 80000] have very different magnitudes, but after StandardScaler both become [-1.22, 0, 1.22]. The key rule is to fit the scaler on training data only - scaler.fit_transform(X_train) - and then use scaler.transform(X_test), so information from the test set never leaks into training. A Pipeline does this automatically.
Why we use it
Distance-based and gradient-based models treat bigger numbers as more important. Scaling gives every feature a fair say and helps optimisers converge faster and more reliably.
When to use it
Scale for KNN, SVM, K-Means, PCA, neural networks and regularised linear or logistic regression. Tree-based models (decision trees, random forests) do not need it, because splits depend only on order.
How it works
- Split data into train and test first
- scaler = StandardScaler()
- scaler.fit(X_train) learns each column's mean and std
- Transform X_train and X_test with the same scaler
- Is the model tree-based? Yes: Scaling is optional. No: Train on the scaled features.
- Model sees features on a common scale
Syntax
from sklearn.preprocessing import StandardScaler
X_scaled = StandardScaler().fit_transform(X)Example
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)Key points
- Essential for KNN, SVM, and gradient-descent-based models
- StandardScaler centers to mean 0, std 1; MinMaxScaler scales to [0,1]
- Fit on training data only, then transform the test data
- Tree-based models do not need scaling
- Wrap scaler and model in a Pipeline to avoid leakage