Jing Zou, Managing Director, Enterprise Model Risk Management, Royal Bank of Canada
To make real, tangible changes in model risk management, one needs to improve the model risk management culture of the organisation. Jing Zou, Managing Director, Enterprise Model Risk Management, Royal Bank of Canada, shares her experience in this field and explores how a more resilient culture could improve performance and efficiency.
Interview with Jing Zou
The Covid-19 pandemic imposed significant new challenges to models when unprecedented shocks happened to many market variables, such as unemployment rate, stock market, commodities prices, mortgage forbearance requests. Some models failed to generate good results for the following reasons:
The relationship between dependent and independent variables is built on the pre-Covid historical period, which might change under extreme shocks.
Manual government intervention, such as stimulus plan and forbearance program, cannot be captured in the existing models.
Model assumptions might no longer hold during the Covid-19 period.
All these motivate us to re-think model risk management. As a result, we raise the three following questions:
The most fundamental answer to these three questions is to build a strong model risk culture. Building the model risk culture means educating the first line of model risk defense (model developer and model business owners) to deepen their understanding of model risk policy, model life cycle, roles, responsibilities, model documentation, and testing expectation. A well-established culture would lead to a due-diligence model performance monitoring, a robust model overlay process, and help efficiency with a well-documented model overlay submission memo, which we will discuss as follows.
As part of the model risk culture, the most efficient way to closely identify model problems is to closely monitor model results and performance during the crisis period. As the first line of model risk defense, the model business owners and users use models for their daily business and quickly identify counterintuitive results. Moreover, an ongoing model performance monitoring process is also helpful in identifying potential model concerns during the crisis. The process includes metrics, thresholds, stakeholders, escalation plan, and frequency. Typically, the frequency might not be sufficient enough to capture the crisis period. Thus, more frequent model performance tracking exercise needs to be conducted during the crisis period. Once the thresholds are breached, stakeholders, including the model risk team, will be notified, and the remediation plan is discussed. The plan could include model overlays, model adjustments, model updates, or even build a new model.
Given the quick turn-around time required in the crisis, the model overlay is common as a tentative solution, allowing more time for model owners to work on strategic solutions, such as model update or redevelopment, in the long run.
A good model risk culture requires a transparent and adequately governed overlay process. In particular, the model risk policy provides clear guidance on the entire model overlay process, including submitting, reviewing, and approving a model overlay.
As the first step, the model owners must provide the overlay document, including the method, the rationale, and impact analysis. The overlay rationale includes, but is not limited to, business judgment, benchmarking, historical data analysis, and sensitivity analysis. The overlay document will be submitted to the model risk team for approval before being used in production for official purposes. Then model risk team reviews the rationale and impact analysis and determines if the overlay can be approved. After it is approved, the overlay can be used for official purposes.
Given a large number of overlays and model updates submitted around the same short period, model risk managers need to enhance efficiency. This effort is also essential in reducing the cost and handling the increasing model inventory.
It has been widely recognised that educating the first line about model risk culture can enhance efficiency significantly. This education of model risk culture would lead to better quality model submission, a better understanding of testing expectations, and a reduction of back and forth communication between modelers and model risk teams [5].
Besides an excellent model risk culture, efficiency can be achieved by agility, automation, and a shared code library.
Agile is one of the most popular approaches in project management because it provides a lot of flexibility to handle urgent projects in a short amount of time. When dealing with multiple urgent model overlays, we prioritie our review based on the materiality of the overlays and corresponding model risk rating. Furthermore, for a large and complex review model update project, we can break it down into a few smaller projects for multiple analysts and speed up the process. Finally, we built an exceptional approval process for urgent tasks: if there is no critical concern on the methodology and important testing results, we allow it to be released into production while the model risk team continues in-depth reviews and identifies potential model issues.
Automation can lead to a high productivity rate and more efficient use of resources. Potential areas include but are not limited to: the model performance monitoring process, the generation of preliminary validation reports by sourcing the model-related information and testing results, the communication emails to the model owners, and project management. For example, we built a project management tool that could visually manage a hundred projects by automating tracking project progress, managing resources, and sending warnings of milestone overdues.
The shared code library can save the cost of writing your code from scratch, ensure consistency and accuracy, and be maintained and expanded quickly. The library can be used for replication and benchmarking purposes.
That said, we need to emphasize that enhancing efficiency saves the model risk personnel from repeating the routine work; it should help improve the effective challenge to models and overlays.
Re-thinking model risk management in light of the Covid-19 pandemic provides more direction to future model risk development areas. Of course, in the new era of risk, artificial intelligence and machine learning models will also play a very important role, which we will not discuss here. If you have insights on how model risk management would evolve, we would be happy to hear from you.
The views expressed in the blog are those of the author and do not represent the views of the Royal Bank of Canada.
