Skip to content
All apps

ROC curves and discrimination

About this app

Adjust default and non-default score distributions, sample size and cutoffs. See the ROC curve, AUC and Gini update live.

Discriminatory power of a rating

A rating or scoring model discriminates well when borrowers who later default receive clearly worse scores than those who do not. Each cut-off yields a hit rate (TPR, share of defaults caught) and a false alarm rate (FPR, share of non-defaults wrongly flagged). The ROC curve plots TPR against FPR across all cut-offs. The area under it, the AUC, is the probability that a random default scores worse than a random non-default: 0.5 is a coin toss, 1 is perfect separation. Banks usually report the Gini coefficient (accuracy ratio): Gini = 2 · AUC − 1.

What you see

The left card shows the score distributions of non-defaults, N(55, 15), and defaults, N(55 − d·15, 15), with a draggable cut-off line; hits and false alarms are shaded. The right card builds the ROC staircase from a drawn sample, together with the random diagonal and the perfect-model corner. A marker shows the current cut-off. The values panel lists TPR, FPR, AUC and Gini; AUC and Gini are estimated from the sample via the Mann–Whitney statistic.

Things to try

Raise the separation d from 0 to 3: the distributions move apart, the ROC curve bulges towards the top left and the Gini rises. Drag the cut-off in the left card and follow the marker along the curve; every extra hit costs additional false alarms. Choose a small sample and a low default rate, then press “Redraw sample” a few times: the estimated Gini moves noticeably although the underlying model stays the same.

Frequently asked questions

What is the difference between AUC and the Gini coefficient?
Both measure the same discriminatory power on different scales. The AUC is the area under the ROC curve and lies between 0.5 and 1 for any useful model. The Gini coefficient, called accuracy ratio in credit risk, equals 2 · AUC − 1 and therefore runs from 0 (random) to 1 (perfect separation).
How do you read a ROC curve?
Each point on the ROC curve belongs to one cut-off and shows the share of defaults caught (y-axis) against the share of good borrowers wrongly rejected (x-axis). The more the curve bends towards the top left, the better the model separates. The diagonal corresponds to a model with no discriminatory power.
Why does the Gini fluctuate in small samples?
The Gini is estimated from the defaults and non-defaults actually observed. With few defaults the result depends strongly on which cases happened to land in the sample. That is why rating model validation looks at the Gini over several years and with confidence intervals.
What can I adjust in the ROC app?
You set separation, default rate, sample size and cut-off, and “Redraw sample” rolls a new sample. The app shows the score distributions, the ROC staircase, hit rate, false alarm rate, AUC and Gini.

Subject: Financial Risk & Analytics

More from Financial Risk & Analytics

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps