Bagging

Also: bootstrap aggregating, bootstrap aggregation

Bootstrap aggregating: draw B bootstrap samples (n rows sampled with replacement), fit one model to each, then average (regression) or vote (classification). Averaging reduces variance while leaving bias about the same, so it suits unstable models like deep trees.

Var(mean of B independent models) = σ² / B

Example

Averaging 10 independent trees that each have prediction variance σ² would cut it to σ²/10; real bootstrap trees are correlated, so the reduction is smaller but in the same direction.

See it in the lesson: Random Forest →

Related terms

More in Classical ML