Encoding and Imputation
Python · Machine Learning (Scikit-Learn)
Theory
Models need numbers with no gaps, but real tables contain text categories and missing values. OneHotEncoder turns a category column into one 0/1 column per value: city with Jaipur, Pune and Delhi becomes city_Delhi, city_Jaipur and city_Pune, with a single 1 in each row. handle_unknown='ignore' makes a city never seen in training encode as all zeros instead of raising an error. OrdinalEncoder maps ordered categories such as S < M < L to 0, 1, 2 when the order is meaningful.
SimpleImputer fills missing values with a statistic learned from the training data: strategy='mean' or 'median' for numbers, 'most_frequent' for categories, or 'constant' with fill_value. Like scalers, encoders and imputers learn in fit, so fit them on training data only. add_indicator=True also adds a column flagging which values were missing, which can itself be useful.
Why we use it
Almost every real dataset has text columns and gaps, and scikit-learn models reject both. Encoders and imputers convert them consistently, so the same mapping is applied to training data, test data and future predictions.
When to use it
Use OneHotEncoder for unordered categories with a manageable number of values, OrdinalEncoder for ordered ones or for tree models. Use the median for skewed numbers and most_frequent for categories. For many columns, combine them with ColumnTransformer.
How it works
- Raw columns with text and gaps
- Are there missing values? Yes: SimpleImputer with mean/median/most_frequent. No: Continue.
- Is the column categorical? Yes: Encode it. No: Leave numbers as they are (or scale).
- Do categories have a natural order? Yes: OrdinalEncoder. No: OneHotEncoder(handle_unknown='ignore').
- Fit on train, transform train and test
- All-numeric, gap-free feature matrix
Syntax
from sklearn.preprocessing import OneHotEncoder, OrdinalEncoder
from sklearn.impute import SimpleImputer
OneHotEncoder(handle_unknown='ignore', sparse_output=False)
OrdinalEncoder(categories=[['S', 'M', 'L']])
SimpleImputer(strategy='median')Example
ohe = OneHotEncoder(handle_unknown='ignore', sparse_output=False)
X_city = ohe.fit_transform(df[['city']])
print(ohe.get_feature_names_out())Key points
- One-hot: one 0/1 column per category value
- handle_unknown='ignore' avoids errors on unseen categories
- OrdinalEncoder only when the order means something
- SimpleImputer: mean/median for numbers, most_frequent for text
- Fit encoders and imputers on training data only