Welcome back! In the last lesson, you learned how to control the complexity of decision trees by tuning their depth. You saw how a tree that is too deep can overfit the training data, while a shallow tree might underfit. Now, let’s build on that knowledge and look at a different way to prevent overfitting, but this time in boosted tree models like XGBoost.

XGBoost is a popular machine learning library that builds models by combining many decision trees. While this approach can be very powerful, it also increases the risk of overfitting, especially if you let the model train for too many rounds. To address this, XGBoost offers a feature called early stopping. Early stopping is a practical tool that helps you stop training at the right time — when the model’s performance on new, unseen data stops improving. This lesson will show you how to use early stopping in XGBoost to get the best results without overfitting.

Early stopping monitors your model’s performance on a validation set during training. If the performance metric does not improve for a set number of rounds, training halts automatically to prevent overfitting.

To use early stopping in XGBoost correctly, you should use the lower-level xgb.train() API instead of XGBClassifier.fit(), since some early stopping arguments are not supported in older versions of the classifier API.

Here’s what you need to set:

• evals: a list of datasets to evaluate performance on (typically your validation/test set),
• eval_metric: the metric to track, such as 'logloss',
• early_stopping_rounds: how many rounds to wait before stopping when no improvement is seen.

Let’s see how this works in practice. First, we’ll prepare the data and convert it to the format XGBoost expects:

Now we train the model using early stopping:

In this example:

• The model is allowed up to 500 boosting rounds.
• Validation performance is evaluated using 'logloss' on the test set.
• Training stops early if performance does not improve for 10 rounds.

After training, you can evaluate the model’s performance:

You’ll see output like this during training:

The line Stopping. Best iteration: [27] tells you that XGBoost stopped training after 27 rounds because it did not see improvement for 10 consecutive rounds. This helps prevent the model from overfitting to the training data. The reported test accuracy shows how well the model generalizes to new, unseen examples.

In this lesson, you learned how to properly use early stopping in XGBoost by switching to the xgb.train() interface and passing the correct parameters. You saw how early stopping can monitor validation performance and halt training automatically when improvement stalls.

Next, you’ll practice using early stopping in your own models. You’ll try different early_stopping_rounds values, observe training behavior, and learn how to strike the right balance between underfitting and overfitting.