Are the Probabilities Right? New Multiperiod Calibration Tests

Abstract

A credit rating system in which the number of observed defaults aligns well with the number of defaults expected by the system demonstrates good calibration. The author derives new goodness-of-fit statistics to test the calibration hypothesis over several observation periods, and he therefore solves the multiple testing problem. The new test statistics are more powerful than existing ones: they are unbiased even under sparse default observations, and they take default correlations explicitly into account. Yet, the tests can be implemented with straightforward spreadsheet calculations. Consequently, compared with Monte Carlo simulations, the new method is computationally far less intensive.

TOPICS: Information providers/credit ratings, statistical methods