Naïve Bayes classifier
About this app
Prior times per-feature likelihoods: posterior field with marginal densities, axis-aligned ellipses and a quadratic boundary; correlated data show what the independence assumption misses; plus a spam mode with log-odds.
The naive assumption
Naive Bayes applies Bayes’ theorem: P(c | x) ∝ P(c) · ∏ p(xᵢ | c). For each class, the prior P(c) is multiplied by the likelihoods of the individual features, and the larger product wins. The “naive” part is assuming that features are independent within a class. The likelihood then factors into a product of marginal distributions, which makes the model simple and fast. With Gaussian features this gives axis-aligned ellipses and usually a quadratically curved boundary. When features are correlated, the model misses that structure.
What you see
In “measurements” mode a field shows the posterior P(A | x), with the marginal densities of both classes along the axes. A table works through prior, both likelihoods and their product for the query point. Optionally the app compares with a full-covariance model and hatches the areas where the two decide differently. In “text (spam)” mode a spam filter classifies a message made of words, and bars show each word’s log-odds.
Things to try
Drag the query point across the field and shift the prior P(A): the boundary moves. Load “correlated” and turn on the full-covariance comparison; look at the hatched areas. In spam mode, add words to the message. Choose “unseen word”: without Laplace smoothing one product becomes 0, with smoothing (+1) a decision is still possible.
Frequently asked questions
- How does a Naive Bayes classifier work?
- Naive Bayes computes, for each class, the product of the prior and the likelihoods of the individual features, and picks the class with the largest product. It is based on Bayes’ theorem. The simplification is to treat features as independent within each class.
- Why is Naive Bayes called “naive”?
- It assumes that all features are independent of each other given the class. In real data this is rarely true, for example when two measurements are strongly correlated. Naive Bayes still often classifies well, because the decision only depends on which class comes out ahead.
- What is Laplace smoothing in a spam filter?
- If a word never appears in one class’s training data, its probability would be 0 and would set the whole product to 0. Laplace smoothing counts every word one extra time: P(w|c) = (n_w,c + 1) / (N_c + |V|). Every word then gets a small positive probability.
- What can I do in this app?
- In “measurements” mode you drag a query point, choose datasets such as “overlapping” or “correlated”, set the prior and compare with full covariance. In “text (spam)” mode you build a message from words, see the log-odds per word and switch Laplace smoothing on and off.
Subject: Machine Learning
More from Machine Learning
- AdaBoost
- Anatomy of a prompt
- Data privacy traffic lights
- Decision Tree
- Ensemble: Bagging vs. Boosting
- Finding faces: masks versus learning from examples
- k-Means
- k-Nearest Neighbors (kNN)
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps