Gradient Boosting Trees for Classification: A Beginner’s Guide

What are Ensemble Techniques ?

Fig 1. Bagging (independent predictors) vs. Boosting (sequential predictors)
Fig 2. Gradient Boosting Algorithm

What is Gradient Boosting ?

Fig 3. Sample dataset
Fig 4. Sample dataset with calculated residuals
Fig 5. Decision Tree — While constructing the tree, its maximum depth has been limited to 2

How do we calculate the predicted residuals in each leaf?

Fig 6. Modified Decision Tree

Why do we need this learning rate?

Fig 7. Sample dataset with residuals calculated using new predicted probabilities
where subscript of predicted residual denotes the i-th tree where i = 1,2,3,…

Important Parameters







Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store


Affine is a provider of analytics solutions, working with global organizations solving their strategic and day to day business problems