8:00 Registration and breakfast

8:50 Chair’s opening remarks

REGULATION

9:00 Reviewing the global regulatory agenda and future updates to include additional scope

Session details 

  • Reviewing regulatory expectations for model risk teams
  • Expectations across different sized organizations
  • Uses of automation, AI and machine learning
  • Global regulatory requirements and lessons learnt across jurisdictions
  • Revisions to model risk guidelines to include AI and machine learning
  • Application of SR11-7 to evolving model risk programs

Reserved for, Federal Reserve Board

INVENTORY – PANEL DISCUSSION

9:35 Managing evolution of inventory and expanded definition of model beyond traditional concepts

Session details 

  • Explosion of model inventories and new methodologies
  • Managing data limitations
  • Best practice to manage evolution of inventory
  • Methodologies and controls variations across inventory
  • Inclusion of non-quantitative models
  • Model components across entire risk appetite
    • Moving beyond financial models
  • Updates and modifications to policies and procedures to accommodate change

Olga Collins, ED, Global Head of Model Risk Infrastructure and Reporting, Morgan Stanley

Alexandre Maurel, Head of US CIB Model Validation Team, BNP Paribas

Irfan Kazi, Managing Director, US Credit Risk Management, CIBC

10:20 Morning refreshment break and networking

OPTIMIZATION

10:50 Enhancing and optimizing model risk management programs in line with internal business objectives

Session details 

      • Maturity of model risk function
      • Automating model risk practices
      • Automating manual tasks and processes
      • Ensuring quality control and consistency
      • Maintaining standards to do more with less
      • Revising model risk frameworks to include AI and machine learning
      • Scaling up model risk in a fast-paced deployment environment
      • Speed and agility of model risk management

Mazin Joumaa, VP, Head of Model Risk Management,  Navy Federal Credit Union

NFR

11:25 Inclusion of operational risks under model risk including cyber risk and fraud models

Session details 

    • Cybersecurity validation requirements
      • Threat identification and mitigation
      • Categorization criteria for models
    • OCC booklet inclusion of cyber security model component
    • Expansion of areas leveraging AI and machine learning
    • Technical expertise and talent
    • Understanding nuances and requirements across domains

Chris Smigielski, Model Risk Director,  Arvest Bank

AI/ML

12:00 Implementing a dynamic model risk management framework for oversight of AI and machine learning

Session details 

    • Success stories implementing an AI or machine learning approach
    • Managing model input and interpreting output
    • Transparency of uses of AI and machine learning
    • Regulatory appetite and expectations
    • Aligning with fair lending requirements
    • Understanding model risk requirements using AI and machine learning
    • Updating on a continuous and ongoing manner
    • Enhancing data governance programs for use of AI
      • Increased volume and complexity of data
    • Risk mitigants on AI and machine learning
    • Managing black box models with no access to data

Rafic Fahs, Chief Model Risk Officer, Fifth Third Bank  

12:35 Lunch break and networking


BLACK BOX MODELS – PANEL DISCUSSION

1:35 Reviewing expectations and management of vendor/black box models with limited control and visibility

Session details 

  • Vendor compliance with model risk regulations
  • Governance and oversight of vendor models
    • Transparency of AI models
    • Documentation for effective validation
  • Visibility and transparency challenges
  • Explainability of black box models
  • Determining assurance and control processes
  • Identifying use cases for black box models

Steve Zhou, MD, Model Risk Management, Webster Bank

Seyhun Hepdogan, Director, Model Risk Management, Discover Financial

Rafic Fahs, Chief Model Risk Officer, Fifth Third Bank

EXPLAINABILITY

2:20 Enhancing methods for explainability of AI and machine learning models

Session details 

  • Explaining decisions from input to output
  • Privacy concerns with excess data collection
  • Data requirements with fragmented programs
  • Understanding which inputs drive certain outputs
  • Development of AI best practice document – PPI/ Policy institute??
  • Ensuring compliance with consumer laws and regulations
    • Integrating compliance teams
  • Customer requirements for explainable AI
  • Countering misconceptions on explainability
  • Defining and understanding the difference between interpretable and explainable from an AI and machine learning perspective
  • Separating explainability and fairness

Kiran Yalavarthy, EVP, Head of Risk Modeling Group, Wells Fargo

EXPLAINABILITY & FAIRNESS

Session details 

  • Integrating fairness into credit models
  • Model development perspectives to consider fairness from outset
  • Managing Headline and reputation risk
  • Determining appropriate metrics
  • Collecting data for fairness testing
  • Managing algorithmic fairness in uncertainty
  • Understanding ethics and fairness of AI results

Raghu Kulkarni, Head of Model Development, Discover Financial

Kiran Yalavarthy, Head of Risk and Finance Model Development, Wells Fargo

Harsh Singhal, C3.AI

Moderated by Agus Sudjianto, EVP, Head of Corporate Model Risk Wells Fargo

3:30 Afternoon refreshment break and networking

BIAS

4:00 Reviewing model input and understanding output to better monitor and capture bias

Session details 

      • Analyzing bias in data sets
      • Quantifying bias
      • Reducing bias in data sets
      • Aligning decisioning with ethics of organization
      • Qualitative approaches to analyzing data bias
      • Regulatory expectations and update on bias
        • CFPB and Fed exams
      • Standardization to detect bias

GOVERNANCE

4:35 Updating governance practices in line with evolution of model risk management scope

Session details 

        • Classification of models
        • Defining a model vs a tool for decision making
        • Validation to ensure models are fit for purpose
        • Model risk for trade surveillance and market abuse
          • Managing black box models and risk to compliance
        • Accountability of AI decision making
        • Automating and streamlining governance processes

5:10 Chair’s closing remarks
5:20 End of day 1 and networking drinks reception

8:00 Registration and breakfast

8:50 Chair’s opening remarks

LARGE LANGUAGE MODELS

9:00 A deep dive into large language models

Session details 

  • Overview of Large Language Models
  • Pros and cons of Large Language Models
  • Use cases for LLMs in Model Risk Management

Roderick A. Powell, FRM, SVP, Head of Model Risk Management, Ameris Bank

DATA

9:35 Understanding new methodologies and trends in the fundamentals of data

Session details 

  • Data as the starting point to automation and technology
  • Techniques and methodologies for data validation
  • Leveraging public source, external and contextual metadata
  • Identifying correlations for proactive risk management
  • Developing forward looking metrics
  • Developing data standards consistent across the firm
  • Leveraging analytics for better risk identification
  • Oversight and controls to govern data end to end
  • Developing frameworks to adhere to privacy regulations

10:10 Morning refreshment break and networking

CLIMATE RISK MODELING

10:40 Developing frameworks to for climate risk data and model requirements

Session details 

  • Harmonizing climate risk rules
  • Testing requirements
  • Model inventory on climate risk
  • Availability of talent and skills
  • Developing and validating climate risk models
  • Updating models and ensuring explainability
  • Data limitations for effective modeling
  • Incorporating economic capital against climate risks

MACRO ECONOMY – PANEL DISCUSSION

11:25 Modeling the macro economy: Model risk challenges in a volatile environment

Session details 

  • Provision techniques to estimate impairments
  • Regulatory expectations of management and judgement overlays
  • Adjusting models in a fast-changing economic environment
  • Mitigations or validations to make business users comfortable with models
  • Frequency to update models in a fast-changing environment
  • Impact of pandemic data and overlays on outlook
  • Recalibrating or rebuilding models post pandemic
  • Reactions to significant interest rate changes
    • Challenging assumptions of a different environment
  • Managing unanticipated and unprecedented outcomes
  • Identifying models predicated on low inflation
    • Modifying assumptions or deploying alternatives

Jing Zou, MD, Model Risk Management, Royal Bank of Canada

Alex Shenkar, SVP, Senior Model Validation Officer, Truist

12:20 Lunch break and networking

RECESSION

1:20 Managing models in a recession environment and proactive identification of changes

Session details 

  • Recalibrating or refitting models
  • Reduction in model development timelines in an evolving environment
  • Rules or triggers requiring a model risk deep dive vs. streamlined review
  • Spotting performance deterioration or degradation
    • Identifying early indicators or triggers
  • Sensitivity of machine learning models to change
  • Identification before model is decommissioned
  • Credit modeling in a downturn
  • Scenario and sensitivity analysis
    • Planning ahead and identifying models requiring action

QUALITATIVE MODELS

1:55 Development and effective validation techniques for qualitative models

Session details 

  • Managing lack of data for qualitative models
  • Challenges with rules and assumptions
  • Reliance of qualitative estimates and forecasts
  • Appropriate validation techniques
  • Developing standards for qualitative models
  • Differentiating types of models

COVID-19

2:30 Reviewing lessons learned from Covid-19 and impact to model outputs with extreme data

Session details 

    • Key lessons learnt as a result of changing environment
    • Benefits of connectivity testing ahead of time
    • Uses of overlays to challenge extreme data
    • Changes to models based on Covid-19 experiences
    • Reviewing uses for data collected during pandemic
      • Extreme macroeconomic movements with no significant losses
    • Preparation as a result for future severe events
      • Backup model in event of model failure
    • Proactive approaches and overlays to stay ahead

3:05 Afternoon refreshment break and networking

LINES OF DEFENSE – PANEL DISCUSSION

3:35 Collaboration of three lines of defense for effective oversight and validation of model risk

Session details 

  • Developing transparency across the organization
  • Aligning with data, audit and compliance
  • Developing a control framework for advancing technology
  • Criticality of integration across teams
  • Auditing AI, machine learning and automation
    • Inclusion of technology audit team

Olga Collins, ED, Global Head of Model Risk Infrastructure and Reporting, Morgan Stanley

Kerri Anderson, Assistant Director of Model Risk Management Northwestern Mutual tbc

Shawn Tumanov, Director, Dara and Analytics (AI/ML/RPA) Governance, BMO Financial

RESILIENCE  

4:20 Enhancing resilience and robustness of models to manage changes

Session details 

  • Ensuring robustness of models exposed to different environments
  • Model monitoring practices
  • Ensuring robustness and resilience during validation
  • Developing a proactive oversight approach
    • Integrating at development and validation

Agus Sudjianto, EVP, Head of Corporate Model Risk, Wells Fargo

4:55 Chair’s closing remarks

5:05 End of Congress