Skip to content
All apps

ROC curve & AUC

About this app

Two score distributions, the threshold sweeps: the ROC curve is traced point by point, AUC as the share of correctly ranked pairs, ROC stays the same at any prevalence, two models with crossing curves compared.

What the ROC curve measures

A classifier gives each case a score, and a threshold t decides what counts as positive. Each threshold yields one point made of the false positive rate FPR = FP / (FP + TN) and the true positive rate TPR = TP / (TP + FN). Sweeping the threshold over all values traces the ROC curve. The diagonal corresponds to guessing. The area under the curve, AUC, equals the probability that a random positive scores higher than a random negative. Because TPR and FPR are each computed within one class, the ROC curve does not depend on prevalence.

What you see

On the left, scores of negatives and positives as bell curves and as individual dots, with a draggable threshold; on the right, the ROC curve with the diagonal. A confusion matrix and metrics such as sensitivity, specificity, precision and accuracy follow the threshold. A pairs mode draws a random positive and negative and marks whether they are correctly ordered. Optionally, model B appears with its own separation and spread.

Things to try

Let the curve be drawn and watch every positive lift it by 1/P and every negative push it right by 1/N. Draw 50 pairs and compare the share of correctly ordered pairs with the AUC. Lower the prevalence: the ROC curve stays put while precision drops. Show model B and look for settings where the two curves cross.

Frequently asked questions

What is an ROC curve?
An ROC curve plots a classifier’s true positive rate against its false positive rate for every possible threshold. It shows how many false alarms you must accept to catch a given share of positives. The closer the curve gets to the top-left corner, the better the model separates the classes.
What does the AUC value mean?
AUC is the area under the ROC curve and lies between 0 and 1. It equals the probability that a randomly chosen positive receives a higher score than a randomly chosen negative. 0.5 corresponds to guessing and 1 to a perfect ranking.
Why does the ROC curve not change with prevalence?
TPR is computed only among positives and FPR only among negatives. If the class ratio changes, both rates stay the same at a given threshold. Precision mixes both classes and drops when positives become rarer.
What can I do in this app?
You set separation, threshold, prevalence and number of cases, let the ROC curve be drawn and draw single pairs or 50 at once to estimate AUC as the share of correctly ordered pairs. A second model can be shown to compare crossing ROC curves.

Subject: Machine Learning

More from Machine Learning

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps