Jing Zou, Managing Director at Enterprise Model Risk Management, Royal Bank of Canada
With many of us looking for the new normal in risk management, we must assess what we need to do to change. Model risk management is, of course, no exception. In fact, models themselves need to be improved in order to adapt to the new market conditions. Furthermore, model risk reporting, in particular compliance reporting and model risk quantification, has become a key issue this year. In this article, Jing Zou, Managing Director at Enterprise Model Risk Management, Royal Bank of Canada, explores what we know about these challenges and what it takes to meet them.
Model risk is “the potential for adverse consequences from decisions based on incorrect or misused model outputs and reports.”[1] Uncontrolled model risk can potentially cause poor business decisions, trigger financial losses, or even damage a firm’s reputation. For a financial institution with hundreds or even thousands of models, model risk needs to be aggregated, summarised, and reported periodically to the board of directors and senior executives. In particular, there are three main questions to be answered:
In the next few sections, we will discuss the details.
To address the first question of whether a firm complies with model risk policy, we conduct periodic compliance reporting at the level of the top of the firm and the level of a few of the largest modelling groups.
There are several compliance metrics to report, such as:
Moreover, the trend of these metrics over time can also be analysed to determine if model compliance has been improved. Ideally, thresholds should be imposed on the above metrics to establish the model risk appetite.
Compliance reporting is also beneficial because it is a significant venue in which to enforce the modelling groups’ improvement of model risk compliance statistics, as the reports are communicated to different levels of senior executives.
One challenging component of model risk reporting is to quantify and aggregate model risk. One potential approach is the bottom-up model risk buffer methodology. The distribution of model uncertainty (not just one single number) can be determined by backtesting, benchmarking, sensitivity analysis, and/or validating findings. Once the percentage distribution of model uncertainty is calculated, we can multiply it by the corresponding model revenue/loss to convert this to dollar value model uncertainty. Then we sum the dollar value model uncertainty across all the models and obtain the model risk buffer at the top of the firm level.
Moreover, model performance monitoring is critical to ensure that a model performs reasonably well as the market environment changes. Typically, there are performance metrics and thresholds associated with a model. A breach in the model performance threshold and its impact should be reviewed by model stakeholders, and if material, be escalated and reported to senior executives. The impact of the breach can be estimated by sensitivity analysis, backtesting analysis, benchmarking, or expert judgment. For example, for a prepayment model, we can evaluate how much the firm’s Risk Weighted Asset (RWA) would be affected. Then the total impact of various model performance breaches in RWA models can be aggregated in dollar values and reported.
To provide a more comprehensive view of model risk and the linkage to the market and industry, we also need to communicate with the board and senior executives on the following items:
The above three perspectives would provide an overview picture of model risk in a firm to the board and senior executives.
[1] SR11-7 Supervisory Guidance on Model Risk Management, Board of Governors of the Federal Reserve System, Office of the Comptroller of the Currency, 2011.
Jing Zou is a Managing Director at Enterprise Model Risk Management at Royal Bank of Canada. The views expressed in the blog are those of the author and do not represent the views of Royal Bank of Canada. Jing Zou would like to thank David Sykes and Richard Koss for their feedback.
