By Heather Russell, Director Model Risk Management, Bank of America
What, for you, are the benefits of attending the ‘Risk Americas Convention’ and what have attendees learnt from your session?
The Risk Americas Convention offers a great chance to build your professional network and broaden your perspective on best practices, challenges, and trends in financial risk management. Even if you do not come with a specific agenda, the contacts and ideas that you take away from the conference could be of great benefit.
In my session, I shared my stories and thoughts on Model Risk Management. Attendees experienced more of a conversation than a lecture.
In your opinion, how can the SR11-7 rule ensure consistent approaches?
SR 11-7 touches on all major areas of model risk management. Beyond providing a view of what regulators are looking for, it also summarizes best practices in a principles-based way. While it is the individual bank’s policy that ensures consistency with SR 11-7, it is worth rereading SR 11-7 from time to time for a clear view of what the underlying principles are.
What could be done to further increase the accuracy of model validation?
- Risk tiering – Materiality should drive the allocation of time and energy, both between different models and across different aspects of a particular model. For example, if the existence of an extra spline knot has much less potential impact than inadequate data quality assurance procedures, data quality investigation deserves a bigger share of energy than the spline knot.
- Model use focus– The purpose of model validation is ultimately to understand the risk of using the model under investigation. Having a solid understanding of how the model is used and relating validation findings back to model use risk are essential.
- Context-driven analysis – Automated and standardized testing can save time and provide valuable testing coverage. But, there is often no substitute for custom-designed testing to investigate specific questions. Identification of assumptions is similar in that it may save time to focus on generic assumptions that always apply, such as “The model is well-specified.” But, the assumptions that exist because of how the model is being used are often where the hidden risk lies. For example, a larger credit line is generally predictive of safer obligors. But, if a credit line variable is included in a model, it would be important to identify whether the associated model use surreptitiously assumes that increasing credit lines makes obligors safer.
- To think deeply of simple things – Investigating the simplest thing that is unclear is a good antidote to “a little knowledge.” Moreover, simple investigations, such as looking at data before imposing assumptions or looking at how a model works in specific examples, can be quite powerful.
In your opinion, what are some of the challenges associated with validation of models and assumptions?
- Getting the full story – Ideally, all the materials needed to validate a model would be provided in the initial model submission. In reality, some of the most critical information is obtained by asking questions.
- Honing conclusions – Initial conclusions on model risks and their mitigation often do not survive a round of proper scrutiny. Carefully weighing evidence, identifying the range of options, and comparing relative merits of possible approaches are essential to effective model validation. When there are multiple validators on a project, Conway’s Law becomes apparent; the coherence of the project’s conclusions mirrors the communication patterns of the team.
- Communicating results – Once conclusions from a validation activity are rock-solid, reaping the benefits depends on getting key messages through to the relevant stakeholders. Anything buried in a report has a good chance of being overlooked. Direct in-person communication has a much higher chance of success.
What challenges do you see ahead for the future of model risk management?
Now that a decade has passed since the last financial crisis and regulatory scrutiny is beginning to wane, complacency around risk management poses an increasing danger. This makes it important to express the value of sound model risk management practices, above and beyond regulatory compliance. Other challenges ahead include addressing new emerging risks such as climate change and new model frameworks such as those based on Machine Learning and Artificial Intelligence.
