Connect ► twitter| youtube|  Log In ► Members Only  |  Corporate One Safekeeping  |  Search

Model validation and its importance in a BSA/AML and OFAC program

Jennifer Morrison, VP, Senior Risk Manager

Many credit unions use some sort of transaction-monitoring software or a manual transaction-monitoring process or procedure to review transactions for filing the required Currency Transaction Reports (CTRs) and/or Suspicious Activity Reports (SARs). Our banking brethren, who are regulated by the Office of the Comptroller of the Currency (OCC), were notified in OCC BC 2011-12 that the OCC considers transaction-monitoring rules to be models. According to the OCC, models must undergo validation testing to mitigate “model” risk or the risk that your transaction monitoring is an inadequate mitigation of your BSA/AML and OFAC regulatory and compliance risk.

Now, the OCC is not the NCUA or a state examiner, but regulators tend to coalesce around certain guidance. Even if not required, validation testing is an important mitigation of your regulatory and compliance risk.

Transactions that are key risk factors for your BSA/AML and OFAC programs should be monitored in some manner. Many core systems have add-on modules that perform transaction monitoring in order to identify transactions that require CTR filings and that might require investigation for a possible SAR filing. Many of us also rely on a hybrid system that might involve transaction monitoring with some Excel-based tools and a good dose of human intelligence. Wire-transaction monitoring is a good example of transactions that require a hybrid approach in many cases.

  • What are some examples of activity types that should be monitored for your credit union?
  • What are some examples of risk factors that should be considered?
  • Should your monitoring program only review transactions that have risk factors, or should your credit union review all transactions?

Why validation testing? Many of us confer considerable reliance upon our transaction monitoring systems or reporting to tell us when to file the CTR and for alerts of suspicious activity. What if your credit union recently added an ATM at the local university student union, and those transactions were inadvertently omitted from your CTR reporting? Or what if your ATM network’s daily cut-off time is incorrectly set in your CTR reporting? What if your ATM cash transactions are not aggregating with your branch cash? Your credit union could find itself in technical violation of CTR regulations!

Financial institutions, including credit unions, should have a strong governance framework around model development, implementation, and use that provides explicit support and structure to risk-management functions through the allocation of resources, policies that define relevant risk management activities, procedures that implement these policies, and compliance mechanisms. Do not forget about change management.

Internal audit plays a key role in verifying that acceptable policies are in place and are followed. Your internal auditor should take an active role in ensuring that the models are independently validated. Internal audit can perform the validation testing of the models if that fits within the auditors’ skill set, contract (if you contract with a third party), and calendar of audits due.

All model components, including input, processing, and reporting, should be subject to validation. This applies equally to models developed in-house and to those purchased from or developed by outside vendors or consultants. The rigor and sophistication of validation should be commensurate with the credit union’s risk profile, the overall use of and reliance on models, the complexity and materiality of the models, and the size and complexity of the credit union’s operations.

The validation methodology of transaction monitoring should be aligned with the credit union’s risk assessment process, considering changes in products and services, members and member type, business-entity members, and geographies. When adding products and services, such as a new ATM location (as mentioned in the previous example), your BSA/AML Officer must be intimately involved in the entire project, and that includes developing relevant transaction testing, integration into existing transaction monitoring, and testing prior to the product launch.

In the validation testing, the review of scenarios and/or thresholds should be documented. The validation should include testing the interface between the core system that houses the transactions and the monitoring system or process. The validation testing should also test any “drill-down” capabilities between the alert, the monitoring system, and the source transaction or transactions. If there is a separate case-management system, the interface between it and the alert generation and/or core system should also be validated.

Comprehensive documentation helps make model-risk assessment and management effective and promotes continuity of operations, compliance with policy, and tracking of recommendations, responses, and exceptions.

If you use any sort of transaction monitoring, including human and hybrid approaches, a critical step in validation testing needs to be a review of the defined monitoring rules or models that isolate the risk factors you or your core system vendor are using. These monitoring rules or models are what determine the alerts or reports of potential unusual activity. The monitoring rules or models should be defined around the key risk factors or patterns of transactional activity you are looking for in creating your alerts and/or reports.

Once your validation testing is complete, the resulting report should be provided to your BSA/AML Officer and probably to your supervisory committee and board of directors, consistent with your credit union’s governance structure. Any remediation should be addressed by the BSA/AML Officer with a specific schedule for completion, the interim steps that will be completed to address any previous lapses, and to cover interim periods while the automated system is addressed. This follow-up should also be reported to the aforementioned committees consistent with your credit union’s governance. In addition, do not forget to validate the remediation, too.

Validation testing is important for your BSA/AML and OFAC program to consider in order to keep your credit union compliant. An outside, third-party reviewer of your BSA/AML and OFAC program may suggest validation testing now as “best practice” even without the NCUA issuing specific guidance. At some point in the future, it is highly likely that an NCUA examiner or your state examiner will want to review your validation documentation, too.