Concept of Decision Trees
General idea of decision trees
- Gready decomposition of feature space by recursive binary splitting
- Stop splitting according to criterion, e.g.
- minimal numbers of instances in region
- max depth of tree
- minimal performance gain
Finding splits
- Search through all features
- For each feature: consider all split values
- Take "best" (feature, splitval) combination
For each box
- Take mean, in case of regression
- Take majority class, in case of classification
California Housing
Decision Tree
Prediction of an individual Instance
Contributions
Recursive Binary Splitting
Split 1
Split 2
Split 3
Split 4
Split 5
MSE Value of Region
Mathematical Definition
Data
Calculation
Split Criterion Regression
Iris Plants
Decision Tree
Prediction of an individual Instance
Gini Value of Region
Mathematical Definition
Data
Calculation
Split Criterion Classification