Machine Learning Workflow
Python · Machine Learning (Scikit-Learn)
Theory
Machine learning means fitting a model that learns patterns from past examples so it can make predictions on new ones. Almost every project follows the same steps. Frame the problem: what to predict (the target y) from which inputs (the features X), and how success is measured. Collect and explore the data, then split it into training and test sets before learning anything from it.
Next, preprocess (fill gaps, encode categories, scale numbers), fit a model on the training set, and evaluate it on the test set with a suitable metric. Improve by trying other models, tuning hyperparameters with cross-validation and engineering better features, and only then check the final model once on the test set. Finally, save and deploy it, and monitor it, because real data drifts over time. In scikit-learn, every model shares the same API: fit, predict and score.
Why we use it
A fixed workflow stops the classic mistakes: testing on training data, leaking test information into preprocessing, or tuning until the test score looks good by luck. It also makes projects repeatable and easy to explain to others.
When to use it
Follow it for every predictive task, from a quick experiment to production. For one-off analysis that only describes past data (averages, trends), you do not need a model at all; plain pandas and charts are enough.
How it works
- Business question: what to predict and why
- Collect data; choose features X and target y
- train_test_split before learning anything
- Preprocess and fit the model on the training set
- Good enough on validation / cross-validation? Yes: Final check on the test set. No: Try features, models, tuning.
- Save, deploy and monitor
- Predictions on new data
Syntax
X, y = df[features], df[target]
X_train, X_test, y_train, y_test = train_test_split(X, y)
model.fit(X_train, y_train)
model.predict(X_test)
model.score(X_test, y_test)Example
model = make_pipeline(StandardScaler(), LogisticRegression())
model.fit(X_train, y_train)
print(model.score(X_test, y_test))Key points
- Split before any step that learns from the data
- Every estimator shares fit / predict / score
- Choose the metric from the business cost of mistakes
- Tune with cross-validation; touch the test set once
- Monitor deployed models; data drifts over time