Model risk and model validation of fraud and KYC models having AI and ML as emerging trends

Has there been a change in model use across the risk appetite?

The risk appetite of the bank is constantly evolving. Regulatory models in IRB, IFRS, Economic Capital space are still at the epicentre of the Risk Appetite but there are evolving trends and increasing interest on Know Your Customer, Loan Pricing,  Fraud and Analytics models. Cyber and Climate risk are also important topics to look forward to in the near future.

What do you imagine the future of model risk management programs to look like?

MoRM is all about dealing with unexpected circumstances, Covid being a prime example and how prepared are we as an organization for these situations. Some key areas where MoRM should focus on in future are as under:

• Make note of response time risk (risk that end-to-end Model Life Cycle process is slow) and include it in the definition of Model Risk Appetite
• Integrated Model Management approach – Joint effort between the first and the second LOD to manage model risk
• Use better tools for model inventory, model development, validation and model use
• For model inventory, involve the first LOD as well and register all the models within the bank in the inventory along with model dependencies
• Model governance to include the role of expert judgement, third party/ vendor models and new Auto Machine Learning tools
• Independent review – Encourage complete separation between model development and validation teams
• Invest in automation
• As scope of models increase, invest in hiring Model Risk Managers having both technical expertise and business acumen

What are some of challenges in AI models within KYC and fraud domains?

Some of the challenges are as under:

• Rule based models, either developed in house, or vendor solutions built on KYC policy that might not have followed Model life cycle strictly
• The performance measurement in these rule-based KYC models, typically in Customer Due Diligence, Screening and Monitoring space, cannot be easily assessed
• Defining what constitutes material model change can be difficult to define
• For fraud models too, performance measurement can be difficult since model output feeds into rules which eventually generates alerts. So model performance cannot be directly related to false negative or false positives
• There might be delay in reporting fraud which needs to be taken into account when making design decisions
• Monitoring of the model performance is highly dependent on the model use
• There could be model ethics concerns related to bias, obscurity and disclosure

What do you see ahead for the future of automating model risk management activities? Do you believe one day the human element will be obsolete?

There is definitely scope for automation in future. Some processes within model development and validation can and should be automated for effective and efficient model lifecycle. Also, model inventory should be maintained in more sophisticated tools which can then generate automated dashboards for reporting on model risk to the senior management.  However, overall model governance will still require human in the loop and that is something which will and should never be obsolete.