Metrics Lab: confusion matrix & metrics
About this app
Slide a classifier's threshold and watch cases move between TP, FP, FN and TN — accuracy, precision, recall, specificity and F1 with their formulas, the precision–recall curve and the accuracy paradox with rare positives.
Four cells, many metrics
A classifier gives each case a score, and a threshold t decides between “positive” and “negative”. Compared with the truth, every case lands in one of the four cells of the confusion matrix: true positive (TP), false positive (FP), false negative (FN), true negative (TN). Accuracy = (TP + TN) / N, precision = TP / (TP + FP), recall = TP / (TP + FN), specificity = TN / (TN + FP), and F1 is the harmonic mean of precision and recall. All are ratios of the same four numbers, yet they answer different questions.
What you see
Each dot is a case, with actual positives above the axis and negatives below, sorted by score. Filled dots are predicted positive, rings predicted negative. Alongside are the confusion matrix, bars for accuracy, precision, recall, specificity and F1, and a precision-recall curve across all thresholds. Hovering a metric highlights its numerator and denominator in the chart. Optionally you can set a cost per FP and per FN.
Things to try
Choose “rare positives (1 %)” and turn on “predict everything negative”: accuracy is 99 % and recall 0 %. Then move the threshold and watch more recall cost precision. Compare the “spam filter (costly FP)” and “cancer screening (costly FN)” scenarios and move the threshold to the cost minimum. Watch how far it shifts between the two.
Frequently asked questions
- What is the difference between precision and recall?
- Precision is the share of cases predicted positive that really are positive; it drops with every false alarm. Recall is the share of all actual positives that the model finds; it drops with every missed case. Lowering the threshold usually gains recall and loses precision.
- Why is accuracy misleading on imbalanced data?
- If only 1 % of cases are positive, a model that always says “negative” reaches 99 % accuracy but finds no positive case at all. For rare classes, recall, precision, F1 or the precision-recall curve are more informative.
- What is the F1 score?
- The F1 score is the harmonic mean of precision and recall: F1 = 2·P·R / (P + R). It is only high when both are high. True negatives do not enter it.
- What can I do in this app?
- You choose a scenario, set prevalence, separation, population and threshold, and see how the confusion matrix, the metrics and the point on the precision-recall curve change. With error costs for FP and FN you can find the cheapest threshold.
Subject: Machine Learning
More from Machine Learning
- AdaBoost
- Anatomy of a prompt
- Data privacy traffic lights
- Decision Tree
- Ensemble: Bagging vs. Boosting
- Finding faces: masks versus learning from examples
- k-Means
- k-Nearest Neighbors (kNN)
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps