Decision Tree
About this app
Axis-parallel splits by information gain: ΔI for every candidate threshold, tree diagram linked to the plane, depth, minimum leaf size and pruning, train against test accuracy over the depth.
How a decision tree learns
A decision tree cuts the feature space into rectangles with axis-parallel splits such as x₁ ≤ θ and predicts the majority class in each leaf. The CART algorithm checks every threshold of both features at each node and picks the one with the largest information gain ΔI = I(parent) − N_l/N · I(left) − N_r/N · I(right). The impurity I is Gini or entropy. Weighting by N_l/N keeps tiny pure leaves from winning. Without limits the tree grows until every leaf is pure and also fits the noise.
What you see
On the left the feature space with the tree’s rectangles, on the right the same tree as a set of rules, and the two are linked. Hovering a node highlights its rectangle; hovering a point shows its decision path in the tree. Strips along the edges show ΔI for each candidate threshold of the selected node. A split panel works through the gain, and a chart plots training and test accuracy against depth.
Things to try
Load “circle”, “diagonal” or “XOR” and let the tree grow. Add noise in the form of mislabelled points and raise the maximum depth: training accuracy climbs to 100 % while test accuracy falls again. Then limit growth with a minimum leaf size or pruning α. On “diagonal”, notice how many steps axis-parallel cuts need to approximate a slanted boundary.
Frequently asked questions
- How does a decision tree choose its splits?
- At each node, every possible threshold of every feature is evaluated. The tree picks the split that reduces the weighted impurity of the two child nodes the most, that is, the one with the largest information gain. This repeats recursively until a stopping rule applies.
- What is the difference between Gini impurity and entropy?
- Both measure how mixed the classes in a node are and are zero for a pure node. For two classes with share p, Gini impurity is 2p(1−p) and entropy is −p·log₂p − (1−p)·log₂(1−p). In practice both usually produce similar trees.
- What is pruning in decision trees?
- Pruning removes branches that add little accuracy, to reduce overfitting. In cost-complexity pruning a branch is kept only if each extra leaf saves at least an error share of α. A larger α gives smaller, more robust trees.
- What can I do in this app?
- You choose Gini or entropy, limit depth, leaf size and pruning α, place your own points and add mislabelled ones. The app links the tree diagram to the feature space, shows ΔI for each candidate threshold and compares training and test accuracy across depths.
Subject: Machine Learning
More from Machine Learning
- AdaBoost
- Anatomy of a prompt
- Data privacy traffic lights
- Ensemble: Bagging vs. Boosting
- Finding faces: masks versus learning from examples
- k-Means
- k-Nearest Neighbors (kNN)
- AI theatre
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps