Guides ยท Science
Machine Learning Evaluation Plan Basics
Plan ML evaluations that hold up
This guide outlines a practical ML evaluation plan: pick fit-for-purpose metrics, set train/validation/test splits or CV, define baselines, avoid leakage, and pre-commit decision criteria before training.
- ml evaluation
- metrics
- train test split
- baseline
- leakage
Pick the right metrics
Match metrics to the problem (classification vs regression) and business goals.
Set splits and CV
Use train/validation/test or cross-validation; respect time/ID leakage constraints.
Define baselines
Compare to simple models or heuristics to ensure improvement is real.
Pre-commit decisions
Decide success thresholds and stopping rules before training to reduce p-hacking.