Governing models in the AI era

What the updated guidance says

• The direction focuses on a robust, board-approved MRMF rather than MRM policy.
• The responsibility of the board is broadened to approve model risk related thresholds for an entity’s risk appetite, tolerance limits, and risk tiering of models.
• The Risk Management Committee of the Board (RMCB) shall be responsible for review of validation reports of high-risk models, annual review of model tiering, and overseeing model monitoring of critical models—such as those approved with exceptions, third-party models, and models involving AI.
• The guideline has also focused on aspects such as Business continuity management (BCM) and decommissioning. Details of decommissioned models for a minimum period of ten years from the date of decommissioning.

Our view

The requirement for board approved model risk appetite threshold elevates model risk from a technical validation exercise to a strategic risk management issue which is required to be governed through a formal risk appetite and board oversight.

The RBI expects entities to have a process to sunset model and have end-to-end traceability of the model’s lifecycle—such as a record/change log which will be useful for impact assessments, regulatory purposes, audit, and retrospective assessment of model-driven decisions.

What the updated guidance says

RE should implement the three lines of defence: model owners being the first line of defence, an independent MRM and validation function being the second line of defence, and a robust and independent internal audit function being the third line of defence.

Our view

Validation teams reporting into model development teams may no longer be considered sufficient for independent oversight as a clear three-line defence is mandated.

What the updated guidance says

Model inventory shall include active, inactive (including under development), and decommissioned models. Further, a model can be used, relied upon, or deployed only once it has been included in the inventory. Along with the key model details, its dependencies with upstream and downstream models and key observations from validation, monitoring, and audit are to be included as well.

Our view

While this may appear to be a small component in the framework, it proves to be a key workstream within the overall implementation roadmap. The larger challenge for institutes would be to identify what constitutes as ‘model’ and further gather the relevant information with respect to their dependencies. However, once the relevant information is collated, it would be a critical foundation for effective model governance.

What the updated guidance says

• Risk-based model tiering is expected to be carried out for all the models and the same shall be approved and reviewed at least annually by the board and RMCB respectively.
• The model tiering shall be based on model’s complexity and materiality.
• The model risk tiering should be further used to define the intensity and frequency of model validation and monitoring; approval process; risk controls and mitigation; level of detailed inventory and documentation.

Our view

The model tiering requirements discussed represent the RBI’s move to set model tiering as the foundation of MRMF rather than a mere classification exercise. This has further advocated a risk-based allocation of time and resources rather than a one-size-fits-all framework for the different activities required during the model lifecycle.

What the updated guidance says

• All models including third-party models are subject to independent validation by the RE.
• Model validation shall also be conducted periodically and in case where internal or external thresholds get triggered.
• Validation reports should be submitted to the RMCB or relevant authority within three months from completion of validation.
• REs should perform due diligence prior to the use of third-party models and obtain minimum technical model documentation. 

Our view

REs will further need to review the specialist expertise and decide on enhanced methodologies for validation of AI and ML models. This may in turn establish potential reliance on external subject matter validation experts.

Selection and onboarding of vendors becomes a critical process by determining clear acceptance criteria based on vendor experience, opacity of model, detailed documentation, and suitability of data and methodology.

What the updated guidance says

REs are expected to test behavioural characteristics of AI models under stressed scenarios by considering the following:

• Define thresholds for explainability and transparency of AI models based on their reliance in taking material decisions.
• In cases where explainability thresholds are not met, RE should define enhanced requirements for model monitoring and validation, back testing, human interference in case of critical decisions, and impact assessment with existing traditional models.

Our view

The RBI has directed entities to decide on the minimum level of explainability expected from AI models based on their importance in business decisions while also acknowledging that certain AI technologies may not be able to achieve the highest expected levels of transparency. REs can maintain a list of models where explainability is limited or not fully achievable. These may include deep neural networks, large language models (LLMs), vendor black-box AI solutions. Thus, defining thresholds is critical for institutions to ascertain further controls and mitigants.

What the updated guidance says

• REs should identify biased AI models that unfairly favour or disadvantage certain customer groups. Fairness testing should be conducted and necessary changes should be made to the model—such as recalibration, redesign, or reducing model complexity.
• Models should be tested on new and different datasets (out of time, out of sample) to confirm that they continue to perform reliably in the real world and changing business conditions.
• RE should conduct structured testing, including red-team exercises, to challenge AI models and identify weaknesses, vulnerabilities, unexpected behaviour, or harmful outputs before and during deployment, especially for customer-facing and generative AI models.

Our view

REs will have to define fairness metrics and establish thresholds for acceptable disparities in model outcomes.

Red teaming is a structured testing process where a team deliberately tries to make an AI model fail, behave incorrectly, generate harmful outputs, or bypass controls to identify weaknesses before real users or attackers do. The RBI recognises that AI models may behave differently under abnormal situations than during normal testing. Therefore, entities should proactively test AI models against these scenarios.

What the updated guidance says

The RBI has emphasised that AI models should not be the sole decision maker but also have a robust human oversight with arrangement for human intervention on need basis. Effective controls should be in place in case of requirement for override or deactivation. Model outputs and decisions should be back tested and reviewed periodically through human intervention.

Our view

The RBI has emphasised that AI should support rather than replace humans in decision making. As the scale and complexity of AI models increases, the RE will have to create robust governance on monitoring these models through human oversight, thereby leveraging efficiency gains of the technology through automation while preserving the accuracy and effectiveness of decisions.

What the updated guidance says

• Deployment, usage, and modification of models should be secured by relevant access controls.
• Cyber risk from third-party components, APIs, external interfaces should be safeguarded with relevant controls.
• External users and customers should be informed with appropriate disclosures and warnings that they are interacting with AL/ML systems and allow them to access human assistance as required.
• Dynamic or automatic updates should be regulated with appropriate controls, thereby defining the scope of automatic updates, data checks, and monitoring requirements.

Our view

The RBI has emphasised that the role of RE is not just limited to deployment of AI models and their validation. Models also require explicit monitoring and controls to define who can access them and what changes can be made. This puts additional responsibilities on the first line of defence—model owners for efficient use of models. At the same time, the third line of defence—auditors—is required to periodically review breaches of access controls and unauthorised changes to the model.