Concept of Decision Trees

General idea of decision trees

• Gready decomposition of feature space by recursive binary splitting
• Stop splitting according to criterion, e.g.
• minimal numbers of instances in region
• max depth of tree
• minimal performance gain

Finding splits

• Search through all features
• For each feature: consider all split values
• Take "best" (feature, splitval) combination

For each box

• Take mean, in case of regression
• Take majority class, in case of classification

California Housing

Decision Tree

Prediction of an individual Instance

Contributions

Recursive Binary Splitting

Split 1

Split 2

Split 3

Split 4

Split 5

MSE Value of Region

Mathematical Definition

Data

Calculation

Split Criterion Regression

Iris Plants

Decision Tree

Prediction of an individual Instance

Gini Value of Region

Mathematical Definition

Data

Calculation

Split Criterion Classification