PD calibration and reliability
About this app
Compare predicted default probabilities with realised rates across twenty rating grades, including confidence bands, portfolio size and systematic bias.
What calibration means
A rating system assigns each grade a probability of default (PD), fixed in its master scale. The system is calibrated when observed default rates match these forecasts. Because a portfolio is finite, the number of defaults in a grade with nᵢ obligors and probability p scatters binomially around nᵢ · p. A deviation alone is therefore not a model error. Each grade is checked with a binomial test: does the realised rate fall inside a confidence band?
What you see
On a logarithmic axis the forecast PD of the 20 grades rises geometrically from 0.01 % to about 30 % and appears as a straight line. Around it lie the 95 % and 99 % confidence bands, whose width depends on how many obligors each grade holds. The realised default rate is drawn as its own line. Below each grade a traffic light shows the test result: green inside the 95 % band, yellow between 95 and 99 %, red outside 99 %. An overall verdict sums up. Quantiles use a normal or Poisson approximation and defaults are treated as independent.
Noise or real error
Turn the portfolio size down (logarithmic from 5,000 to 2,000,000 obligors): the bands widen and almost everything turns green, because little data means little test power. Press “Re-roll” to see that with 20 grades one grade occasionally leaves the 95 % band by chance. Then dial in a systematic bias: the realised line breaches the bands grade after grade in the same direction. That pattern points to a genuine calibration error.
Frequently asked questions
- What is PD calibration in credit risk?
- PD calibration means that the default probabilities forecast by a rating system match the default rates actually observed. A model can rank borrowers well (high discriminatory power) and still be miscalibrated if its PD level is too high or too low. Banks test calibration regularly as part of model validation.
- How does the binomial test for rating grades work?
- For a grade with n obligors and forecast PD p, the number of defaults follows a binomial distribution if defaults are independent. That distribution gives bounds that are exceeded with only 5 % or 1 % probability. If the observed number of defaults falls outside, the deviation is statistically significant.
- Why is a single yellow rating grade no cause for alarm?
- With 20 grades and a 95 % band, about one grade is expected to fall outside the band by pure chance. It becomes a concern when several grades deviate in the same direction, which suggests a systematic error. Real defaults are also correlated, which adds further year-to-year variation.
- What can I change in the PD calibration app?
- You set the portfolio size and a systematic bias between true and forecast PD, and you can re-roll the realisation. The app shows forecast, realised default rate, confidence bands, a traffic light per grade and an overall verdict for 20 rating grades.
Subject: Financial Risk & Analytics
More from Financial Risk & Analytics
- Asset correlation in 3D
- Risk-bearing capacity: loss distribution
- ROC curves and discrimination
- ROC model comparison
A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps