PLEASE SELECT A DAY FROM THE BELOW DROPDOWN TO VIEW THE AGENDA:

8:00 Registration and breakfast

8:50 Chair’s opening remarks

Day 1 Moderator: Jonas Jacobi, CEO & Co-founder, ValidMind

REGULATION

9:00 Reviewing the global regulatory agenda and future updates to include additional scope

Session details 

  • Reviewing regulatory expectations for model risk teams
  • Expectations across different sized organizations
  • Uses of automation, AI and machine learning
  • Global regulatory requirements and lessons learnt across jurisdictions
  • Revisions to model risk guidelines to include AI and machine learning
  • Application of SR11-7 to evolving model risk programs

David Palmer, Senior Supervisory Financial Analyst, Federal Reserve Board

INVENTORY – PANEL DISCUSSION

9:35 Managing evolution of inventory and expanded definition of model beyond traditional concepts

Session details 

  • Explosion of model inventories and new methodologies
  • Managing data limitations
  • Best practice to manage evolution of inventory
  • Methodologies and controls variations across inventory
  • Inclusion of non-quantitative models
  • Model components across entire risk appetite
    • Moving beyond financial models
  • Updates and modifications to policies and procedures to accommodate change

Alexandre Maurel, Head of US CIB Model Validation Team, BNP Paribas

Irfan Kazi, Managing Director, US Credit Risk Management, CIBC

Javier Calvo, Partner, Management Solutions 

Nikolai Kukharkin, Managing Director, Head of Quantitative Risk Control, MUFG Bank

10:20 Morning refreshment break and networking

OPTIMIZATION

10:50 Enhancing and optimizing model risk management programs in line with internal business objectives

Session details 

      • Maturity of model risk function
      • Automating model risk practices
      • Automating manual tasks and processes
      • Ensuring quality control and consistency
      • Maintaining standards to do more with less
      • Revising model risk frameworks to include AI and machine learning
      • Scaling up model risk in a fast-paced deployment environment
      • Speed and agility of model risk management

Mazin Joumaa, VP, Head of Model Risk Management,  Navy Federal Credit Union

NFR

11:25 Inclusion of operational risks under model risk including cyber risk and fraud models

Session details 

    • Cybersecurity validation requirements
      • Threat identification and mitigation
      • Categorization criteria for models
    • OCC booklet inclusion of cyber security model component
    • Expansion of areas leveraging AI and machine learning
    • Technical expertise and talent
    • Understanding nuances and requirements across domains

Chris Smigielski, Model Risk Director,  Arvest Bank

AI/ML

12:00 Implementing a dynamic model risk management framework for oversight of AI and machine learning

Session details 

    • Success stories implementing an AI or machine learning approach
    • Managing model input and interpreting output
    • Transparency of uses of AI and machine learning
    • Regulatory appetite and expectations
    • Aligning with fair lending requirements
    • Understanding model risk requirements using AI and machine learning
    • Updating on a continuous and ongoing manner
    • Enhancing data governance programs for use of AI
      • Increased volume and complexity of data
    • Risk mitigants on AI and machine learning
    • Managing black box models with no access to data

Rafic Fahs, Chief Model Risk Officer, Fifth Third Bank  

12:35 Lunch break and networking


BLACK BOX MODELS – PANEL DISCUSSION

1:35 Reviewing expectations and management of vendor/black box models with limited control and visibility

Session details 

  • Vendor compliance with model risk regulations
  • Governance and oversight of vendor models
    • Transparency of AI models
    • Documentation for effective validation
  • Visibility and transparency challenges
  • Explainability of black box models
  • Determining assurance and control processes
  • Identifying use cases for black box models

Steve Zhou, MD, Model Risk Management, Webster Bank

Seyhun Hepdogan, Director, Model Risk Management, Discover Financial

Kamil Kluza, CPO, Climate X

Rafic Fahs, Chief Model Risk Officer, Fifth Third Bank

EXPLAINABILITY

2:20 Enhancing methods for explainability of AI and machine learning models

Session details 

  • Explaining decisions from input to output
  • Privacy concerns with excess data collection
  • Data requirements with fragmented programs
  • Understanding which inputs drive certain outputs
  • Development of AI best practice document – PPI/ Policy institute??
  • Ensuring compliance with consumer laws and regulations
    • Integrating compliance teams
  • Customer requirements for explainable AI
  • Countering misconceptions on explainability
  • Defining and understanding the difference between interpretable and explainable from an AI and machine learning perspective
  • Separating explainability and fairness

Kiran Yalavarthy, EVP, Head of Risk Modeling Group, Wells Fargo

EXPLAINABILITY & FAIRNESS

2:45 Understanding ethics and fairness considerations and approaches for effective measurement

Session details 

  • Integrating fairness into credit models
  • Model development perspectives to consider fairness from outset
  • Managing Headline and reputation risk
  • Determining appropriate metrics
  • Collecting data for fairness testing
  • Managing algorithmic fairness in uncertainty
  • Understanding ethics and fairness of AI results

 

Raghu Kulkarni, Head of Model Development, Discover Financial

Kiran Yalavarthy, Head of Risk and Finance Model Development, Wells Fargo

Harsh Singhal, C3.AI

Moderated by Agus Sudjianto, EVP, Head of Corporate Model Risk, Wells Fargo

3:30 Afternoon refreshment break and networking

4:00 Best practices for model governance and documentation

Session details 

Session details to come

Jos Gheerardyn, Co-Founder & CEO, Yields.io

4:35 Benchmarking machine learning algorithms in the PD and retail credit space

Session details 

        • Adversarial testing
        • Empirical examples

Imir Arifi, Head of Methodologies & Models, Americas, UBS

5:10 Chair’s closing remarks
5:20 End of day 1 and networking drinks reception

8:00 Registration and breakfast

8:50 Chair’s opening remarks

Day 2 Moderator: Michael Jacobs Jr, SVP – Lead Quantitative Analytics and Modeling Expert, PNC Financial Services Group

LARGE LANGUAGE MODELS

9:00 A deep dive into large language models

Session details 

  • Overview of Large Language Models
  • Pros and cons of Large Language Models
  • Use cases for LLMs in Model Risk Management

Roderick A. Powell, FRM, SVP, Head of Model Risk Management, Ameris Bank

BIAS

9:35 Reviewing model input and understanding output to better monitor and capture bias

Session details 

  • Analyzing bias in data sets
  • Quantifying bias
  • Reducing bias in data sets
  • Aligning decisioning with ethics of organization
  • Qualitative approaches to analyzing data bias
  • Regulatory expectations and update on bias
    • CFPB and Fed exams
  • Standardization to detect bias

Jaya Sil, Senior Director, Validation Team, Model Risk Management, Santander

10:10 Morning refreshment break and networking

10:40 Adapting to Change: the evolution of EUCs from spreadsheets to analytic assets, and its implications for model risk management

Session details 

  • The role of emerging technologies such as low-code/no-code platforms, open-source software, and trends in the evolution of EUCs
  • The impact of analytic assets, including low-code/no-code and open-source applications, on model risk management
  • Approaches for integrating EUC/analytic asset control functions into existing model risk management frameworks

Diane Robinette, President and CEO, Incisive Software

MACRO ECONOMY – PANEL DISCUSSION

11:25 Modeling the macro economy: Model risk challenges in a volatile environment

Session details 

  • Provision techniques to estimate impairments
  • Regulatory expectations of management and judgement overlays
  • Adjusting models in a fast-changing economic environment
  • Mitigations or validations to make business users comfortable with models
  • Frequency to update models in a fast-changing environment
  • Impact of pandemic data and overlays on outlook
  • Recalibrating or rebuilding models post pandemic
  • Reactions to significant interest rate changes
    • Challenging assumptions of a different environment
  • Managing unanticipated and unprecedented outcomes
  • Identifying models predicated on low inflation
    • Modifying assumptions or deploying alternatives

Jing Zou, MD, Model Risk Management, Royal Bank of Canada

Alex Shenkar, SVP, Senior Model Validation Officer, Truist

Xiangyin (Jane) Zheng, Audit Director, BNY Mellon

12:20 Lunch break and networking

RECESSION

1:20 Managing models in a recession environment and proactive identification of changes

Session details 

  • Recalibrating or refitting models
  • Reduction in model development timelines in an evolving environment
  • Rules or triggers requiring a model risk deep dive vs. streamlined review
  • Spotting performance deterioration or degradation
    • Identifying early indicators or triggers
  • Sensitivity of machine learning models to change
  • Identification before model is decommissioned
  • Credit modeling in a downturn
  • Scenario and sensitivity analysis
    • Planning ahead and identifying models requiring action

George Soulellis, Chief Enterprise Model Risk Officer, Freddie Mac

 

1:55 Inflation Risk Management

Session details 

  • Best practices to get ahead of inflation as a potential long-term risk to business forecasting, analytics and models
  • In the absence of sufficient data, what risk assessments we ought to be doing?
  •  What models and analytics may be more vulnerable to a lengthy high inflation period?

Piero Monteverde, Assistant Chief Model Risk Officer, Capital One

2:30 The section of asset price bubbles in the cryptocurrency markets with an application to risk measurement of model risk

Session details 

  • We analyze of the impact of asset price bubbles on the markets for cryptocurrencies through applying the theory of local martingales.
  • We find consistently across several widely traded cryptocurrencies that in isolation there is no evidence of a  bubble, but modeled jointly with an equity market index we do detect a bubble.
  • We also apply the principle of relative entropy to measure the model risk from failing to detect a bubble through ignoring the the relationship to equity prices.
  • We find that in most cases the model risk “multipliers”, used for establishing a model risk reserve

Michael Jacobs Jr, SVP – Lead Quantitative Analytics and Modeling Expert, PNC Financial Services Group

3:05 Afternoon refreshment break and networking

LINES OF DEFENSE – PANEL DISCUSSION

3:35 Collaboration of three lines of defense for effective oversight and validation of model risk

Session details 

  • Developing transparency across the organization
  • Aligning with data, audit and compliance
  • Developing a control framework for advancing technology
  • Criticality of integration across teams
  • Auditing AI, machine learning and automation
    • Inclusion of technology audit team

Xiangyin (Jane) Zheng, Audit Director, BNY Mellon

Kerri Anderson, Assistant Director of Model Risk Management Northwestern Mutual

Shawn Tumanov, Director, Dara and Analytics (AI/ML/RPA) Governance, BMO Financial

RESILIENCE  

4:20 Enhancing resilience and robustness of models to manage changes

Session details 

  • Ensuring robustness of models exposed to different environments
  • Model monitoring practices
  • Ensuring robustness and resilience during validation
  • Developing a proactive oversight approach
    • Integrating at development and validation

Agus Sudjianto, EVP, Head of Corporate Model Risk, Wells Fargo

4:55 Chair’s closing remarks

5:05 End of Congress