Decision tree

Also: decision trees, CART, classification tree, regression tree

Built greedily: at each node, pick the feature and threshold whose split most reduces impurity (Gini or entropy for classification, variance for regression), and recurse until a stopping rule. Easy to read, but deep trees overfit badly.

Gini = 1 − Σₖ pₖ²

A decision tree splits on one feature at a time until it reaches a leaf

Example

A node with 4 samples of class A and 4 of class B has Gini 1 − (0.5² + 0.5²) = 0.5; a split into {4 A} and {4 B} gives two pure children, a gain of 0.5.

See it in the lesson: Decision Trees →

Related terms

More in Classical ML