A featured contribution from Leadership Perspectives: a curated forum reserved for leaders nominated by our subscribers and vetted by the Construction Tech Review Advisory Board.
DZ BANK
Machine Learning and Model Risk Management
Machine learning has permeated almost all areas in which inferences are drawn from data. The range of applications in the financial industry spans from credit rating, loan approval processes in credit risk to automated trading, portfolio optimization, and scenario generation for market risk. Machine learning techniques can also be found in fraud prevention, anti-money laundering, efficiency/cost control, and marketing models. Machine learning has demonstrated significant uplift in these business areas, and the use of machine learning will continue to be explored in the financial industry.
The banking industry is becoming increasingly aware of model risks related to the use of machine learning techniques for risk management purposes. Even though quite comprehensive, regulatory guidance such as the Fed’s SR 11-7 will not answer all financial practitioners’ questions related to the implementation and use of machine learning algorithms in their daily business.
What are the main challenges when it comes to the application of machine learning in a regulatory context?
Explainability/ Interpretability.
One should be in a position to explain how the algorithm makes a prediction or decision for one specific case at a time.
Overfitting. One should recognize that there is some amount of randomness in the training data. If not taken care of, algorithms show good performance on training data – but fail on data not seen before.
Robustness and transient environments: One should account for the fact that markets or environments can change, which calls for a good balance of adaptability and robustness.
Bias and adversarial attacks: Compared to classical statistics there is a much more prominent role for (training) data in machine learning applications.
Of course, some of these issues have been addressed within the machine learning community. What is needed now is the transfer to the banking industry without “reinventing the wheel”. For that reason, the Model Risk Managers’ International Association (mrmia.org) issued a white paper to discuss some (banking) industry best practices. That would only be a starting point since the applications are rapidly evolving.
How should the Model Risk Governance react to the challenges mentioned above?
Model review: If machine learning algorithms frequently change their “inner workings”, how should model validation react? What should be the contents of the validation activity? How should aspects of conceptual soundness (Fed’s SR 11-7) be treated?
Model development, implementation, and use: How to account for the more prominent role of data? What level of complexity can users handle? What kind of explanations would be accepted by users or by senior management?
-
The banking industry is becoming increasingly aware of model risks related to the use of machine learning techniques for risk management purposes
Model identification and registration: How to account for model complexity, the role of data, model recalibration within the model inventory?
Excellent quality standards: Existing frameworks need to be enhanced by additional checks for overfitting and sensitivity analysis to test for robustness. Tests for possible bias and discrimination may be reviewed with respect to reputational risk.
Some banks have already developed frameworks to deal with model risks of machine learning applications, while other banks are still in the midst of soul searching for viable starting points. There definitely is a need to share emerging industry best practices and to develop a comprehensive framework to assess model risks in machine learning applications.
