Boosting
Also: gradient boosting, AdaBoost, XGBoost, gradient-boosted trees, GBM
Weak learners, usually shallow trees, are added one at a time: AdaBoost reweights misclassified examples, while gradient boosting fits each new tree to the residuals (negative gradient of the loss). It mainly reduces bias and is a top performer on tabular data.
F_m(x) = F_{m−1}(x) + η · h_m(x)
Example
The target is 10 and the current ensemble predicts 7; the next tree is fit to the residual 3, and with learning rate η = 0.1 the prediction moves to 7.3.