One model to rule them all

One model to rule them all

First there was Basel II, with it’s subsequent years of interpretation, model development, review, documentation, and refinement. In retrospect, it’s surprising how much effort went into estimating 12-month forward default rates under an average economy scenario. Then came stress testing: CCAR (Comprehensive Capital Analysis and Review) and DFAST (Dodd-Frank Act Stress Testing) required banks to look further into the future (9 quarters) under alternate economic scenarios. This is a fundamentally more challenging task than Basel II, because one must quantify and incorporate economic sensitivities, adjust for attrition / pay-off, and quantify not just the probability of default, but also the timing of default.

As always happens with new regulations, lenders and regulators both evolve their knowledge and expectations of what should be considered standard practice. For CCAR, lenders consistently report that the Federal Reserve examiners want the best possible models and at the loan-level. However, for the banks with assets less than $50 billion but greater than $10 billion that must only comply with DFAST, OCC and FDIC examiners are reported to have made comments such as, “If you build a more complex model, you will receive increased scrutiny.” Thus, the path of least resistance for banks is simple time series models, even though they fail to capture key portfolio dynamics and do not prepare banks for crossing the CCAR threshold.

Now we have CECL (the Financial Accounting Standards Board’s rules for Current Expected Credit Loss) and IFRS9 (the International Accounting Standards Board’s rules for International Financial Reporting Standards, latest revision), and both banks and examiners are groaning. Please, not another parallel model with different requirements and another cycle of interpretation, development, review, documentation, and refinement!

Although CECL and IFRS9 have some differences, both approaches require lifetime loss forecasting for loans of any age, considering current economic conditions but relaxing onto long-run averages beyond the foreseeable macroeconomic horizon. Although the regulations claim that any approach works so long as it pays homage to the original goals, CECL and IFRS9 modeling needs actually come closest to CCAR. Lenders of all sizes and business models, if they want models that are defendable to auditors and examiners, will need models with economic sensitivity, competing risks of attrition and default, and monthly or quarterly forecasting. The best models will be loan-level so that they consider current loan conditions and integrate with accounting systems.

Screen Shot 2016-09-22 at 17.42.45

How do we bring unity to such differing requirements? With a few nudges, it’s quite possible. Basel and DFAST are the outliers. If we start from Basel models, the gap to CCAR or CECL / IFRS9 is quite wide. A 12-month average Basel PD could be viewed as an input to a more complete model, but the simplest approach is to start from CCAR or CECL / IFRS9. A best-in-class CCAR model would provide loan-level, monthly default rate forecasts with adjustment for attrition and economic scenarios. To satisfy Basel, one need only use a through-the-cycle economic scenario and aggregate the first 12-months of the forecast.

Clearly, a large bank building a CCAR-compliant model can use the same model for DFAST, but if we start from one of those commonly-built DFAST time series models, life is not easy. A simple time series model will not work for CECL or IFRS9, because it does not capture any lifetime loss aspects. In fact, it’s not even clear how it can comply with DFAST given how much of the portfolio’s key dynamics are lost, but so far they have been accepted. For DFAST lenders that are not large enough for CCAR, satisfying CECL or IFRS9 may push them to create CCAR-class models anyway, and their DFAST submissions will be much better because of it.

Thinking through the similarities and differences between all these requirements leads us to the follow commonalities.

  • Economic sensitivity (to model different economic scenarios)
  • Monthly or quarterly forecasting (to incorporate economic scenarios)
  • Competing risks of default and attrition / pay-off (for long-range forecasting)
  • Lifecycles in loss and attrition / pay-off timing (for long-range forecasting)
  • Loan-level (for CCAR and best-practice CECL / IFRS9 and Basel)

In most general terms, two primary classes of model can satisfy these needs: state transition models and survival models.State transition models are just another name for Markov models. State transition models are just another name for Markov models. The states can be risk grades, delinquency buckets, or anything else where a given account can be described as having a unique state membership in a given month. To meet the above goals, the transitions need to depend upon economic factors, account performance factors, and the age of the loan. Terminal states should include charge-off and attrition / pay-off at a minimum. For long-range forecasting, the distribution function for a single account would rapidly become too complex to estimate, so a Monte Carlo approach is used instead. At each time step, each account is randomly assigned to one of the states according to the predicted transition probabilities until the end of the forecast horizon is reached.

Survival models is a general term for a broad collection of models. Cox proportional hazards models, age-period-cohort (APC) models plus scoring, and panel data models with age as a factor are the most common examples. In all cases, macroeconomic factors and account performance factors can be incorporated along with the obvious lifecycle (hazard) function versus the age of the account. These models fit naturally with a competing risks framework, but usually simplify the problem to exclude modeling intermediate steps and focus instead on the end states. Forecasts are a simple loan-level expectation value of conditional probabilities of charge-off or attrition / pay-off at each month in the forecast horizon.

With sufficient data and careful treatment of multicolinearity among the factors, the above two classes actually become indistinguishable other than the choice of simulation versus expectation. Fortunately, state transition and survival models are known technologies. Enhancements will continue to be made, but either type can be used today to obtain universal applicability, so long as the validators, examiners, and auditors are prepared to accept them.