What Boards Need to Know to Validate CECL Models

The
transition to the new approach for estimating loan losses has necessitated that
banks develop new financial models to calculate the allowance for credit losses
for financial statements. As institutions continue implementing the standard,
called the current expected credit loss method or CECL, the question of how to
appropriately validate those models has become a timely concern.

Model validation is inherently complex, technical and data-intensive, and boards of directors have rightfully delegated much of the related decision-making to senior management and specialists. Nevertheless, directors, officers and senior executives do need to understand some important general principles about model validation and the CECL model validation processes used for their organizations in particular.

Why model validation matters
Validation is an essential step designed to verify that the financial models a bank uses are performing as expected. The major regulatory agencies are explicit about their expectations, stating in a joint 2011 publication: “All model components, including input, processing, and reporting, should be subject to validation; this applies equally to models developed in-house and to those purchased from or developed by vendors or consultants.”

The
guidance spells out three core elements of effective validation:

• Evaluating conceptual
  soundness:
  This element involves assessing the quality of the model’s design and
  construction, including a review of the documentation and empirical evidence
  supporting the development methods.

• Ongoing monitoring: This tenant is about
  verifying that the model is implemented correctly, is performing as intended, and
  that internal and external data inputs are accurate, complete, and consistent
  with the model’s design.

• Outcomes analysis: This facet compares
  the model’s outputs to actual real-world outcomes using a variety of statistical
  tests, including back-testing.

Changes
in markets, products, customer base, the economy or other factors can affect a
model’s performance. Banks should conduct a periodic review of each model used
in estimating expected credit losses to verify that they are applicable and working
as intended. It generally is good practice to perform such reviews prior to
initial implementation, and then at least annually. Banks may need to validate
more frequently while the new CECL modeling methodologies are evolving and
stabilizing, or when other material change in conditions occurs.

Who is responsible?
While board members are not involved in all the ins and outs of model development or validation, they do need to understand what management is doing to address validation. Board responsibility for this oversight is typically assigned to the risk committee or audit committee. However, all directors should ultimately be aware of the mechanisms used to evaluate their bank’s CECL models, as CECL affects both the balance sheet and income statement.

Management’s
responsibility for model validation will depend on factors such as the bank’s size,
complexity and the maturity of its risk management function. For some institutions,
this responsibility falls to a dedicated model risk management team; in many
others, the general risk management function is responsible for the adequacy of
model risk validation. Internal audit also takes an active role in some organizations.

Regardless of whether the CECL models are developed in-house or acquired externally, the validation process should be performed by people who were not involved in the models’ development or use and who have no stake in the outcome. Those conducting the validation also should have the requisite technical skills and knowledge to effectively challenge potentially highly complex models.

In
addition, they must have a thorough understanding of the new CECL standard and
should be familiar with the relevant business lines and loan products whose
performance is being modeled. Assembling a team with such diverse, specialized
capabilities can be challenging, which is why many banks use third-party
specialists to conduct or manage model validation.

A validation report provided by the model vendor or developer might contain useful information, but lack the required independence. The report’s scope might be limited to the design, coding and function of the model itself. Similarly, obtaining a vendor’s system and organizational control report likely will not address unique user considerations, configuration, portfolio risk characteristics or segmentation, data inputs (including from third-parties), governance and oversight and other related features that also should be validated, including qualitative adjustment frameworks. Directors also need to understand that the external financial statement audit does not address this responsibility to model validate.

As banks implement the new CECL methodology – and as executives, model developers, auditors and regulators become more familiar with its impact – the financial models used to calculate the allowance for credit losses undoubtedly will undergo revision and calibration. In this environment, directors should take extra care to understand and ensure that those responsible for validating these models have the independence, authority, understanding and technical capabilities they need.