Assessment of model risk in the aggregate

Assessment of model risk in the aggregate

By Liming Brotcke, Quantitative Manager, Federal Reserve Bank of Chicago

About Liming:

I lead the Risk Modeling and Assessment team of the Supervision and Regulation department at the Federal Reserve Bank of Chicago.  I started my career at the Chicago Fed as a model risk management specialist in 2013. During the past five year I have conducted numerous in-depth quantitative and qualitative reviews of models developed by large banks for their capital and liquidity stress testing as well as key lines of business decision making across the Federal Reserve System. Those models cover areas such as retail, wholesale, market risk, interest rate risk, derivatives, and operational. I am a veteran CCAR examiner and have been co-leading the quantitative review of credit cards loss estimation approaches. Recently, I am serving on the national supervision team such as the LISCC Program focusing on consumer lending, CECL methodologies, and machines learning algorithmsused for anti-money launder, alternative lending and trading. I also conduct research and analytics related to model risk management and consumer lending.
Prior to joining the Fed, I was with Citi Group managing risk of a $2-billion portfolio.  Before that I worked at Discover Financial Services as a modeler and developed models to assist various business operations including underwriting, marketing, loss forecasting, and deposits. I hold a doctoral degree in Economics from the University of Illinois at Chicago.

What, for you, are the benefits of attending a Course like ‘Model Risk Management’? What can attendees expect to learn from your session?

Large and mid-sized banks have increased reliance on models for various operations of their banking business. It is important for Model Risk Management professionals to meet regularly and discuss challenges encountered and accomplishments achieved.

We will walk through our peers a practical approach we developed to assess aggregate model risk leveraging model statistics.  While the approach we propose has its limitations, it addresses some common caveats of the industry common practice related to measuring model risk in the aggregate.

Can you provide our readers with some insight into your research on quantitative model risk in the aggregate?

While most large institutions have established a qualitative assessment framework combined with a suite of metrics to evaluate model risk, the active search for transparent and easy-to-implement approaches to measuring aggregate model risk at the firm level eludes practitioners. In this study, we introduce a quantitative element to the assessment framework leveraging various model statistics from model development and performance monitoring periods.

 In your opinion, what further progress could be made across the industry?

Transparency, consistency, along with explainability and interpretability are the key elements for sound and sustainable MRM practice.

Without giving too much away, can you provide an example of adding some quantitative elements to assess model risk?

Think from the model life cycle concept for a second. Risk associated with newly developed models can be assessed by measuring various model statistics related to model fit and robustness upon development completion.

How do you see the model risk landscape evolving over the next 6-12 months?

Machines learning algorithms will continue to attract attention of MRM professionals both on the model development and validation fronts as banks continue to search alternative ways to gain efficiency.

Hear insights like this and more at the Model Risk Management Course…
Make your free account