Skip to content
All apps

ROC model comparison

About this app

Compare two scoring models at a chosen acceptance rate. Inspect how many additional defaults are caught and good customers retained.

The question

A scoring model ranks credit applications from “worst” to “best”. The bank fixes an acceptance rate and rejects the worst scores. How many future defaults end up rejected, and how many good clients are lost along the way, depends on discriminatory power, measured by the Gini coefficient. At the same acceptance rate the more discriminating model rejects more bads and fewer goods. That difference is the business value of a model overhaul.

What you see

The chart holds two ROC curves (old and new model) with the cumulative share of rejected goods on the x-axis and the cumulative share of rejected bads on the y-axis, plus the random diagonal. For each model a vertical line marks the cutoff for the chosen acceptance rate. A result table lists accepted and rejected goods and bads, the bad rate of accepted cases and the marginal bad rate at the cutoff. A gain block shows defaults additionally caught and good clients additionally kept. The curves are smooth model curves, not real score data.

Things to try

Set “Gini new” above “Gini old” and watch the gain block grow. At a fixed acceptance rate every extra default caught is also a good client who stays in the book, which is why both figures are identical. Change the acceptance rate, the number of cases N and the default rate to see where better discrimination pays off most. Under “Calibration” you can move the anchor point (x₀, y₀) each curve passes through.

Frequently asked questions

What is the acceptance rate in credit scoring?
The acceptance rate is the share of applications a lender approves; the cases with the worst scores are declined. It determines the cutoff score. A lower acceptance rate reduces defaults in the book but also turns away good clients.
How does a higher Gini affect a lending business?
A higher Gini means the model separates defaulters from non-defaulters better. At the same acceptance rate, more future defaults fall among the rejections and more good clients among the approvals. The bad rate in the book drops without shrinking volume.
What is the marginal bad rate at the cutoff?
The marginal bad rate is the share of bad cases right at the cutoff, among the applications that are only just accepted. It is usually much higher than the average bad rate of all accepted cases. It helps judge whether moving the cutoff is worthwhile.
What does the ROC model comparison app show?
It compares an old and a new scoring model with adjustable Gini values at the same acceptance rate. It draws both ROC curves with cutoff lines and calculates in a table how many additional defaults the new model catches. The curves are a teaching model, not derived from real score data.

Subject: Financial Risk & Analytics

More from Financial Risk & Analytics

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps