Why this is the first problem every ML learner meets
The goal of a model is not to remember the examples it was trained on. It is to make good predictions on data it has never seen. A model that memorises its training set can look perfect in your notebook and still fail the moment real users send it new inputs.
That is why interviewers so often ask about overfitting. Being able to read a training curve, explain what is going wrong and choose a sensible fix shows you understand how models actually behave, not just how to call fit() and predict().
How to tell overfitting from underfitting
Compare the score on training data with the score on validation data.
Underfitting
Training score is low and validation score is also low. The model has not learned the pattern, for example a straight line fitted to clearly curved data.
Good fit
Training and validation scores are both good and close to each other. This is what you are aiming for.
Overfitting
Training score is very high but validation score is much lower. The model has memorised details that do not generalise, such as a very deep decision tree.
Learning curves
Plot both scores as training data or training epochs grow. A widening gap points to overfitting; two low, flat lines point to underfitting.
The bias-variance idea in plain words
Overfitting and underfitting are two sides of the same trade-off:
- Bias – error from a model that is too simple and makes strong assumptions. High bias causes underfitting.
- Variance – error from a model that is too sensitive to the exact training rows. High variance causes overfitting.
- Making a model more complex usually lowers bias but raises variance, and simplifying it does the reverse.
- The aim is a model complex enough to learn the pattern but constrained enough to ignore the noise.
How to fix overfitting, step by step
Try these roughly in order, checking the validation score after each change.
Check for data leakage first
A suspiciously high score can mean test information leaked into training. Make sure preprocessing sits inside a Pipeline and fits on training data only.
Get more or better data
More varied examples make it harder to memorise noise. For images and text, data augmentation can help.
Simplify the model
Limit tree depth, reduce the number of features or use fewer layers and units in a neural network.
Add regularisation
Use L1 or L2 penalties (Lasso, Ridge, the C parameter in logistic regression) or dropout and weight decay in deep learning.
Stop training earlier
For neural networks and boosting, use early stopping so training ends when the validation score stops improving.
Validate with cross-validation
Confirm the improvement holds across several folds, not just one lucky split.
How to fix underfitting
If both scores are low, the model needs more capacity or better inputs:
- Use a more flexible model, for example a random forest or gradient boosting instead of a linear model.
- Add or engineer more informative features from the raw data.
- Reduce regularisation that is holding the model back too much.
- Train for longer if a neural network is still improving.
- Check the data itself: wrong labels or missing key columns can make any model look weak.
Quick answers about overfitting and underfitting
Short answers to the questions AI & ML learners search for most.
What is overfitting in simple words?
Overfitting is when a model memorises its training data, including random noise, so it performs very well on that data but badly on new data it has not seen.
What is underfitting in machine learning?
Underfitting is when a model is too simple to learn the real pattern in the data, so it performs poorly on both training data and new data.
How do I know if my model is overfitting?
Compare training and validation scores. A high training score with a much lower validation score, or a gap that keeps widening on a learning curve, is the usual sign.
Does regularisation prevent overfitting?
It helps. Regularisation such as L1, L2, dropout or weight decay penalises overly complex models, which usually narrows the gap between training and validation scores.
Can more data fix overfitting?
Often, yes. More varied training data makes it harder for the model to memorise noise, though it will not fix underfitting or data leakage on its own.
Go further with model training
See where model building sits in the syllabus, or read the guides that build on it.
Make a model overfit on purpose
Train a decision tree on a small dataset with no depth limit, note the training and validation scores, then set max_depth to 3 and compare again. Watching the gap close is the quickest way to understand overfitting well enough to explain it in an interview.

