Why one train-test split is not enough
With a single split, your score depends partly on luck: which rows happened to land in the test set. Run the same code with a different random split and the accuracy can move by several points, especially on small datasets.
Cross-validation reduces that luck. Every row is used for testing exactly once, and you get both an average score and a sense of how much it varies, which is far more honest than one number.
How k-fold cross-validation works
Five folds (k = 5) is a common default.
Split the data into k folds
Shuffle the rows (unless order matters) and divide them into k roughly equal parts.
Train on k−1 folds
Fit the model on four of the five parts.
Test on the remaining fold
Score the model on the part it has not seen.
Repeat k times
Rotate so each fold is the test set once, giving k scores.
Average and check the spread
Report the mean score and the standard deviation. A large spread means the model is unstable or the data is small.
Types of cross-validation and when to use them
Choose the split to match your data.
K-fold
The standard choice for most tabular problems where rows are independent.
Stratified k-fold
Keeps the same class balance in every fold. Use it for classification, especially with imbalanced classes such as fraud or churn.
Time-series split
Always trains on the past and tests on the future. Use it for forecasting, where shuffling would let the model see the future.
Group k-fold
Keeps all rows from the same customer, patient or device in one fold, so the model is not tested on someone it has already seen.
Cross-validation mistakes that inflate scores
These make results look better than they really are:
- Scaling, encoding or filling missing values on the whole dataset before splitting, which leaks test information into training. Put preprocessing inside a scikit-learn Pipeline instead.
- Shuffling time-series data, so the model learns from the future.
- Tuning hyperparameters on the same folds you report, then presenting that score as final. Keep a separate hold-out test set or use nested cross-validation.
- Using plain k-fold on imbalanced classes, so some folds have almost no positive cases.
- Letting the same person or entity appear in both training and test folds.
Quick answers about cross-validation
Short answers to the questions learners search for most.
What is cross-validation in simple words?
It is testing a model several times on different parts of the data and averaging the results, so the score does not depend on one lucky or unlucky split.
What is the difference between train-test split and cross-validation?
A train-test split evaluates the model once on one held-out part. Cross-validation repeats training and testing across several splits and averages the scores, giving a more reliable estimate.
How many folds should I use?
Five or ten folds are common defaults. More folds use more data for training each time but take longer to run.
How do I do cross-validation in Python?
In scikit-learn, use cross_val_score or cross_validate with a model or Pipeline and a splitter such as KFold, StratifiedKFold or TimeSeriesSplit.
Do I still need a test set if I use cross-validation?
It is good practice. Use cross-validation to choose and tune models, then confirm the final choice once on a separate test set the process never touched.
Go further with model evaluation
See where evaluation sits in the syllabus, or read the guides that build on it.
Compare one split with five folds
Train the same model with a single train-test split five times using different random seeds, then run 5-fold cross-validation once. Seeing how much the single-split score jumps around is the fastest way to understand why cross-validation matters.

