Model risk and model validation of fraud and KYC models having AI and ML as emerging trends
The views and opinions expressed in this article are those of the thought leader as an individual, and are not attributed to CeFPro or any particular organization.
By Ayan Mukherjee, Senior Product Owner, VP, Model Validation, ING Bank
How do you manage model risk at ING?
Models are imperfect reflections of reality; they generate model risk which cannot be eliminated, but can be made transparent to the organisation and managed. Model risk is the risk that the financial or reputational position of a financial institution is negatively impacted because of the use of models. Process of identification, assessment, control (acceptance or mitigation) and monitoring of the risks caused by the use of models is done through Model Risk Management (MoRM). The main aim is to identify, manage and mitigate model risk by ensuring the right level of MoRM control mechanisms per model class. This can be achieved through:
- Effective model governance and controls
- Sound Model life cycle
- Development, to implementation, to use
- Rigorous model validation
Has there been a change in model use across the risk appetite?
The risk appetite of the bank is constantly evolving. Regulatory models in IRB, IFRS, Economic Capital space are still at the epicentre of the Risk Appetite but there are evolving trends and increasing interest on Know Your Customer, Loan Pricing, Fraud and Analytics models. Cyber and Climate risk are also important topics to look forward to in the near future.
What do you imagine the future of model risk management programs to look like?
MoRM is all about dealing with unexpected circumstances, Covid being a prime example and how prepared are we as an organization for these situations. Some key areas where MoRM should focus on in future are as under:
- Make note of response time risk (risk that end-to-end Model Life Cycle process is slow) and include it in the definition of Model Risk Appetite
- Integrated Model Management approach – Joint effort between the first and the second LOD to manage model risk
- Use better tools for model inventory, model development, validation and model use
- For model inventory, involve the first LOD as well and register all the models within the bank in the inventory along with model dependencies
- Model governance to include the role of expert judgement, third party/ vendor models and new Auto Machine Learning tools
- Independent review – Encourage complete separation between model development and validation teams
- Invest in automation
- As scope of models increase, invest in hiring Model Risk Managers having both technical expertise and business acumen
What are some of challenges in AI models within KYC and fraud domains?
Some of the challenges are as under:
- Rule based models, either developed in house, or vendor solutions built on KYC policy that might not have followed Model life cycle strictly
- The performance measurement in these rule-based KYC models, typically in Customer Due Diligence, Screening and Monitoring space, cannot be easily assessed
- Defining what constitutes material model change can be difficult to define
- For fraud models too, performance measurement can be difficult since model output feeds into rules which eventually generates alerts. So model performance cannot be directly related to false negative or false positives
- There might be delay in reporting fraud which needs to be taken into account when making design decisions
- Monitoring of the model performance is highly dependent on the model use
- There could be model ethics concerns related to bias, obscurity and disclosure
What do you see ahead for the future of automating model risk management activities? Do you believe one day the human element will be obsolete?
There is definitely scope for automation in future. Some processes within model development and validation can and should be automated for effective and efficient model lifecycle. Also, model inventory should be maintained in more sophisticated tools which can then generate automated dashboards for reporting on model risk to the senior management. However, overall model governance will still require human in the loop and that is something which will and should never be obsolete.
You may also be interested in…