Cross-Validation
Python · Machine Learning (Scikit-Learn)
Theory
A single train/test split gives one score that depends on which rows happened to land in the test set. K-fold cross-validation repeats the evaluation: the data is cut into k folds, the model is trained on k-1 folds and scored on the remaining one, and this rotates until every fold has been the test fold once. You get k scores; their mean is a more reliable estimate and their spread shows how stable the model is.
cross_val_score(model, X, y, cv=5) does it in one line and returns the five scores. For classifiers an integer cv uses StratifiedKFold, keeping class proportions in every fold; pass KFold(n_splits=5, shuffle=True, random_state=0) for explicit control, and scoring='f1' or 'neg_mean_absolute_error' to choose the metric. cross_validate can return several metrics at once. Put preprocessing in a Pipeline so it is refitted inside each fold.
Why we use it
It gives a more trustworthy performance estimate than one split, uses all the data for both training and testing and shows whether a score is stable or a lucky draw, which matters most for small datasets.
When to use it
Use cross-validation to compare models and tune hyperparameters, keeping a final test set aside for one last check. Use 5 or 10 folds; StratifiedKFold for classification and TimeSeriesSplit for time-ordered data, never a shuffled KFold.
How it works
- Training data and a model (or pipeline)
- Split rows into k folds
- Train on k-1 folds, score on the held-out fold
- Has every fold been the test fold? Yes: Average the k scores. No: Rotate to the next fold.
- Compare models by mean score and spread
- Reliable performance estimate
Syntax
from sklearn.model_selection import cross_val_score, KFold
cv = KFold(n_splits=5, shuffle=True, random_state=0)
scores = cross_val_score(model, X, y, cv=cv, scoring='accuracy')
scores.mean(), scores.std()Example
scores = cross_val_score(model, X, y, cv=5)
print(scores.round(3), scores.mean().round(3))Key points
- Every row is used for testing exactly once
- Report the mean and the spread of the fold scores
- Integer cv on a classifier uses StratifiedKFold
- Wrap preprocessing in a Pipeline to avoid leakage
- Use TimeSeriesSplit for time-ordered data