By Roderick A Powell, SVP, Head of Model Risk Management, Ameris Bank
What, for you, are the benefits of attending the ‘Risk Americas Convention’ and what have attendees learnt from your session?
I am a big proponent of machine learning. I am dedicated to helping my colleagues in the industry implement this technology in a responsible way. Attendees to my session walked away with a basic understanding of some of the most widely-used machine learning algorithms and how to validate models based on them.
What are some of the challenges of defining an AI model?
A.I. is a broad term. It encompasses machine learning, natural language processing, robotics and so on. In the realm of machine learning, you have further classifications, such as “supervised,” “unsupervised,” and “self-reinforcement.” There are numerous algorithms that fall under each category. You can also classify machine learning algorithms as being either “shallow” or “deep.” Deep algorithms encompass neural networks and deep learning. Technically, any model that leverages these algorithms could be referred to as an A.I. model.
Robotic process automation (“RPA”) tools may sometimes be defined as AI models. In many cases, bots are automating manual tasks that involve extracting information from documents and classifying documents. Natural language processing (“NLP”) is often a necessary component of bots. NLP is also used for chatbots in the customer service space. There is no consensus in the industry on whether to refer to bots or chatbots as models or not. I am hopeful that new regulatory guidance may address the issue.
What are the benefits and drawbacks of using machine learning models?
The benefits of using machine learning models is that they allow you to improve on existing models. However, the degree of benefits depends on the amount of relevant data at your disposal. In most cases, the more data you have, the more accurate the machine learning model predictions will be. Many of the machine learning algorithms in use have been around for many years. However, it is much less expensive to store data now and companies are doing a better job of collecting data. Big data is driving the adoption of these models. In the area of fraud detection and cybersecurity, the potential of some of these machine learning models to reduce the number of false positives is a definite benefit. There is a lot of time and expense wasted on researching legitimate transactions that traditional fraud detection models flag as fraudulent.
The drawbacks of using machine learning models include the lack of transparency of some of these models and their complexity. In addition, from a validation perspective, some of these models are difficult to replicate. A final issue is the potential bias that may result from using these models. In the banking arena, this is a particular worry when using these models to make loan decisions. Intentional or unintentional bias in these models could expose a firm to legal and/or regulatory risk.
In your opinion, what skillsets are the most desired for development and validation?
In the arena of machine learning, it is desirable for developers and validators to have solid skills in big data analysis, statistics, and coding. People in the machine learning space need to be continuous learners because advances in the field are coming rapidly. The historical tools by developers for machine learning, such as SAS and SPSS, are increasing being supplemented or replaced by free, open source programming languages, such as R and Python. I have no doubt that other languages and tools will arise in the future. Developers and validators need to constantly upgrade their skills. In addition to having great math and coding skills, it is valuable to have domain knowledge. In many cases, domain knowledge must be gained on the job.
It is also a plus to keep up with technology that is tangential to AI. Blockchain technology and digital currencies come to mind. While not currently a hot topic, I can see a time when knowledge in this area will prove very useful. Blockchain technology requires an understanding of encryption, digital currency mining, and smart contracts.
What do you see ahead for the future of AI and machine learning within model risk management?
I believe that we will see more regulatory guidance specific to AI and machine learning in the future. I also believe that more banks will adopt models and solutions that incorporate this technology. Therefore, model risk managers need to make sure that their skill-sets and those of their teams keep up with the latest AI and machine learning applications. I also believe that some of these techniques will be leveraged by bots to automate model validation tasks in the future, especially for relatively simple models.
