Advanced Model Risk Agenda

Day One | March 12


Registration and breakfast


Chairs opening remarks

Day 1 Moderator: Mehdi Esmail, Chief Product Officer, ValidMind


Enhancing model risk programs to manage divergence in regulatory expectations across jurisdictions

  • Restructuring and leveraging existing resources to comply with existing regulatory expectations.
  • Managing current market environment changes due to bank collapses.
    • Navigating tighten regulatory expectations.
  • Enhancing model risk programs to deal with AI regulations.
  • Different standards of regulations between the US and Europe.
    • Aligning standards of international banks with US regulations.
  • Adjusting liquidity balance management models due to new regulation requirements.
  • Technological expectations to comply with FRTB and Basel 4.
    • Building capacity, governance, and infrastructure to meet requirements.
  • Leveraging technology to streamline Model Governance and bridge gaps arising from diverse regulatory frameworks
  • Adopting emerging considerations (AI/ML models, ESG, etc) for holistic model governance

Alberto Ramirez, FCA, MAAAPartner – Risk Analytics, Solytics Partners

Oscar Zheng, Executive Director, Head of Model Validation, Natixis CIB Americas

Charlie Lu, Former Managing Director, Head of AI/ML Model Risk Management, Barclays 

Nikolai Kukharkin, Head of Model Risk Management, MUFG


Adapting traditional model risk frameworks to align with large language models

  • Testing and validating large language models
  • Developing a forward-look approach to large language models
  • Understanding the complexity of generative models for large language model validation
  • Creating a platform to implement and develop large language models to mitigate risks
  • Managing large language models beyond regulatory reporting
  • Building use cases of large language models using NLP
  • Controlling large language models whilst leveraging their functions
  • Transparency and best practices for risk frameworks

Agus Sudjianto, EVP, Head of Corporate Model Risk, Wells Fargo

Tarun Joshi, Executive Director, NLP/ GEN AI Model Validation, Wells Fargo


Morning refreshment break and networking


A model builder’s perspective on AI/ML model risk in risk management

  • Can hallucinations in risk management be tolerated?
  • Can AI be useful in risk management?
  • One size does not fit all – Considerations for capital models, trading models, monitoring models, risk management tools.
  • Sharing some use cases.
  • A vision for the future.

Hany Farag, Head of Modeling Methodology, CIBC 


Developing an effective AI and machine learning model risk management program

  • Understanding machine learning lifecycle in banking
  • Understanding key elements of machine learning model validation
  • Testing modules for conceptual soundness
  • Testing modules for outcome analysis
  • Developing inherently interpretable benchmark models
  • Developing the automated pipeline for streamlined validation
  • Enabling automated validation and monitoring for dynamically updating models

Aijun Zhang, SVP, Head of Validation Engineering, Wells Fargo


From theory to practice: embedding AI risk management in the organizational fabric

  • Understanding Risks from AI models and how they differ from other model risks
  • An overview of the AI regulatory landscape
  • How to quantify risks from models and Generative AI
  • Establishing periodic model testing and evaluation protocols
  • Risk mitigation strategies across the full model inventory
  • Instituting robust governance structures to mitigate Organizational Risk
  • Integrating end-to-end technological solutions and industry best practices to operationalize governance and build trustworthy AI systems

Anant Agrawal, Head of AI Risk Management, CIMCON Software


Lunch break and networking


Assessing and monitoring AI, machine learning, and large language models to avoid bias and toxic results

  • Safeguarding models to avoid reputational risk
  • Balancing model performance, robustness, and fairness
  • Going beyond traditional MRM functions to build models for explainability and bias testing
  • Managing uncertainties of acquiring data to ensure models are not biased
  • Transparency and explainability analysis within regulatory expectations
  • Having a consistent approach to bias and explainability
  • Managing stronger requirements for explainability of models in medium-to-long-term investing.

Roderick Powell, SVP, Head of Model Risk Management, Ameris Bank; NFR Leaders Advisory Board member, CeFPro


Understanding the methodology of black box models and increasing transparency

  • Managing black box inputs and outputs
  • Machine learning tools outside of vendor black box models
    • Developing machine learning models and tools to validate internally
  • Validating machine learning models outside of black box models
  • Transparency with vendor black box models
  • Approaching black box models from a different perspective to further understand
    • Having a business approach
  • Obtaining documentation and tools from vendors to understand how the model performs well

Seyhun Hepdogan, Director of Analytics, Fifth Third Bank; Fintech Leaders Advisory Board member, CeFPro

Moez Hababou, Head of Compliance, CCAR and Credit Models, BNP Paribas

Aleksey Leksanov, Managing Director, Head of Model Risk Management, Mizuho Bank


Afternoon refreshment break and networking


AI-powered SMB risk modeling

  • Addressing challenges in SMB lending
  • Role of AI in analyzing actionable insights from massive data sets
  • Robust relationship analysis: enriching data with non-traditional niche sources
  • Beyond conventional approaches: replacing traditional linear models, statistical distributions, and simple decision trees with modern machine learning methods
  • End-to-end case management: Building a seamless SMB underwriting system

Sal Rehmetullah, Co-CEO and Co-Founder, Worth AI

Tom AliffSVP-Risk and Analytics, Equifax


Operationalizing governance best practices of AI and machine learning models

  • Integrating skill sets and disciplines
  • Governing AI beyond a traditional model space
    • Dealing with ethics, intellectual properties, reputational risks, and cyber security Psychology and linguistic experts to manage limitations of AI governance
  • Governing AI beyond use case dependency
  • Quantitative tools to measure and manage AI Modeling
  • Meaningfully managing and governing AI/ml with model expansion

Rodanthy Tzani, Head of Model Risk Management, New York Life Insurance Company


Managing complexities with the ongoing expansion of model risk scope and inventory

  • Putting enhancements in place to accommodate AI
  • Using 8 categories of Enterprise Risk Management for managing model governance and inventory
  • Considering generative AI in model risk inventory
  • Incorporating machine learning models into inventory
    • ChatGPT and Chatbots
  • Governance of new technologies and advanced models
  • Developing replacements for model risk management in inventory
  • Addressing the lack of inventory in risk management

Chris Smigielski, Director of Model Risk Management, Arvest Bank; NFR Leaders Advisory Board member, CeFPro


Chair’s closing remarks


End of day one 

Day Two | March 13


Registration and breakfast


Chairs opening remarks

Day 2 Moderator: Chris Smigielski, Director of Model Risk Management, Arvest Bank; NFR Leaders Advisory Board member, CeFPro


Managing models with continued volatility and geopolitical challenges and the impact of change

  • Using current data to cross reference climate impact on modeling portfolio
    • Hurricanes, wildfire risks, flooding risks
  • Climate change impacts on ability to provide loans
  • Modelling physical and transition risk
  • Defining climate risk and constituting where it falls within the model portfolio
  • Managing the lack of data to foresee climate stress testing
  • Understanding the impact of specific climate stress on the path of macroeconomic variable
  • Standards for modeling climate risk

Alisa Rusanoff, Head of Credit, Crescendo Asset Management

Julia Litvinova, Managing Director, Head of Model Validation and Analytic, State Street

Deeptha Anand, Former, Head of Model Validation, Societe Generale


Reviewing the impact of climate risk and incorporating within model risk management

  • Using current data to cross reference climate impact on modeling portfolio
    • Hurricanes, wildfire risks, flooding risks
  • Climate change impacts on ability to provide loans
  • Modelling physical and transition risk
  • Defining climate risk and constituting where it falls within the model portfolio
  • Managing the lack of data to foresee climate stress testing
  • Understanding the impact of specific climate stress on the path of macroeconomic variable
  • Standards for modeling climate risk

C.Robin Castelli, Author of Quantitative Methods for ESG Finance


Morning refreshment break and networking


Risks in AI

  • an overview of high-profile recent model risk incidents
  • focus areas for implementing AI risk management
  • extending these methodologies to generative AI
  • how technology can help

Jos Gheerardyn, Co-founder & CEO,


Mitigating overfitting risk in AI/ML models

  • Model risk and overfitting
  • Why AI/ML models have a tendency to overfit
  • How to detect the overfit condition
  • Mitigating techniques and methods

George Soulellis, Enterprise Model Risk Officer, Freddie Mac


Understanding how models are incorporating macroeconomic effects and associated considerations

  •  Identifying key factors / macro drivers that models need to capture
  • Adapting models for emerging risks using historical data and macro effects 
  • Adapting models for future macro scenarios outside of historical experience 
  • How do you manage the model limitations in the interim before you are able to recalibrate the models to capture the emerging risks?
  • Challenges and enhancements of deposits modelling
  • How do you make sure current models can calibrate successfully given current rates/economic environments, especially inverted yield curve?
  • How can you make sure models will continue to calibrate successfully if and when Fed starts to reduce rates? What on-going monitoring mechanism would you put in place systematically?

Katherine Zhang, Managing Director, State Street

George Soulellis, Enterprise Model Risk Officer, Freddie Mac 

Charles Shen, Managing Director, Head of Model Risk Management, Societe Generale


Lunch break and networking


Quantification of model risk and the aggregated model portfolio for end-to-end model risk management

  • Assessing the overall risk of an interconnected model network
  • Developing a quantitative measurement beyond risk ratings
  • Practical and effective ways to quantify model risk
  • Understanding the importance of an aggregated model portfolio to mitigate emerging risks
  • Intersection and connection point between models
  • Verifying and monitoring the data between model

Manoj Singh, Managing Director, Model Risk Officer, Bank of America


Managing the evolution of data requirements as model requirements expand

  • Incorporating Covid data into modeling
  • Controlling covid data
  • Updating models using credit loss forecasting models
    • Should Covid data be included in these?
  • Addressing unpredictable Covid data in the models
  • Ensuring models predict credit loss appropriately
  • Addressing credit loss due to unprecedented scenarios
    • Pandemics

Imir Arifi, Head of AI Model Development, UBS


Managing the increase in fraud and financial crime tools in model risk management inventory

  • Gathering data to govern and review financial crime tools
  • Fitting in tools into the traditional definition of models
  • Clearly defining financial crime models with regulatory expectations
  • Getting appropriate results from quantitative modeling tools
  • Evaluating conceptual soundness to ensure the model is fit for use
  • Including new dimensions and updating models to capture fraud
  • Capturing potential fraudulent events using fraud detection modeling
    • Building systems to flag potential threats

Ankur Goel, SVP, Head of Consumer and Fraud Modeling, PNC


Afternoon break and networking


Strengthening model risk management through ongoing performance monitoring

  • Regulatory expectations of ongoing performance monitoring of model risk
    • Tracking and validating model performance
  • Addressing the thresholds of ongoing model performance monitoring
    • Setting tolerance levels on performance
  • Incorporating automation in ongoing performance monitoring
  • Identifying risks between performance reviews

Arthur Robb, Managing Director, Head of Model Risk Management, TIAA


Incorporating emerging credit risks into model risk management frameworks and measuring exposure

  • Measuring model exposure to a high inflation environment
  • Limitations of models and anticipation of risks
  • Adapting concurrent views of model risk to mitigate counterparty credit risk
  • Incorporating modeling metrics into daily risk management
  • Incorporating overlays into models for credit risk management
  • Balance sheet and trading book positioning for decision-making
  • Alignment for business changes in the horizon on credit risk
  • Developing credit risk modeling strategies

Daniel Saunders, Head of Model Risk, USAA 


Chair’s closing remarks


End of Congress