Stress testing validation: Effective review and challenge framework

Stress testing validation: Effective review and challenge framework

By Mabel Wong, SVP, Head of Model Risk Operations Assurance and Management, Citizens Bank.

Ahead of the Stress Testing USA: CCAR & DFAST Congress 2017, Mabel provided us with an overview into executing an effective review and challenge framework for stress testing validation. 

Mabel, can you please tell the Risk Insights readers a little bit about yourself, your experiences and what your current professional focus is?

I’ve been with Citizens for 32 years and accumulated a wide range of institutional knowledge and connections in a management capacity across areas of model risk, retail, commercial, portfolio management. I joined the risk organization 6 years ago and recently assumed the role of the Head of Model Risk Governance.

Since 2014, I’ve been actively involved in the model risk management group in applying and implementing the framework for model risk to meet the OCC’s and Federal Reserve’s 2011-12 supervisory guidance on Model Risk Management. I’ve led a number of large validation initiatives supporting the Bank’s CCAR and DFAST model validation efforts. I’ve built processes and tracking tools to enable effective resource management and a review and challenge framework for model validation.

At the Stress Testing USA: CCAR & DFAST Congress, you spoke on your insight regarding executing an effective review and challenges framework for stress testing validation. Why do you believe this is a key talking point and what do you think are the crucial things to consider?

Effective review and challenge is a guiding principle for managing model risk. This theme is omnipresent in every stage and turn of testing and validating models for its suitability for use. While we all conduct due diligence in developing and vetting the approach and outcomes before applying the models to our forecasts, effective documentation and transparency is as crucial as the the quality of the models.There needs to be back up for rationale to be repeatable, defensible and a basis to springboard to future enhancements.

In your experience, what are the key considerations that need to be made when executing an effective challenge framework?

The key to executing an effective challenge framework is to assemble a wide range of experts to ensure a broad representation of knowledge and viewpoints. The experts must be qualified and have the technical and practical basis to provide a valid, educated view on the reasonability of the approach and outcomes.

Can you please give a brief overview of the challenge in stages of stress testing?

The first challenge is to develop customized menus of review and approval elements for each stage of the model lifecycle and to have enough checkpoints for thorough, meaningful, transparent discussions on options and selections. The second is to ensure robust participation and engagement from stakeholders from across all disciplines to be the sounding board for options, voice biased and unbiased opinions on advantages and disadvantages for options on the table to come to a concensus. The final challenge is to have consistent documentation to artifact the formal vetting that was accomplished. The artifacts would also serve as basis for future model improvements and options.

How can institutions ensure their internal review processes are adequate? And what are the implications if they are not?

One indication of an adequate review process is having had robust discussion, debate and compromise, clearly identified responsibilities, for robust discussions covering every milestone of the model cycle. The other key indication is to have a repeatable process with sufficient documentation. The impact of not having a well-vetted model is finding out during implementation and application that it doesn’t work.

What, in your opinion does the future hold for stress testing professionals, and how can they keep up with the increasing changes in the industry?

Change and varying interpretations of guidance is constant. To keep up and evolve, the obvious tools are to stay current with the publications, attend conferences. However, maintaining open lines with Regulatory partners and conducting a regular gap analysis to ensure alignment with current guidance is key because there is not a day where there isn’t something more to learn and new perspectives to apply and improve on our processes.