May 26, 2020

Thresholds are Not Rocket Science


Share this post: 

Imagine trying to catch deer using a mouse trap; the chances of that working out in one’s favor are slim. Whether it’s Anti-Money Laundering (AML) Compliance or Marketing Acquisition models, the same holds true for model tuning methodology.

Each model has a distinct objective, risks, and implications. An AML Compliance model helps in identification of suspicious money movement where a single false negative could lead to a regulatory breach and reputational damage for the business. On the other hand, ineffective execution of a Marketing Acquisition model would lead to potential market share losses.

What is model tuning?

In summary, it is a combination of qualitative and quantitative analyses to calibrate the parameter thresholds of rule-based models.

There is a common belief that model tuning is a highly quantitative process involving complex analytics. While this is mostly true, qualitative analysis also plays a critical role in the successful design and development of models.

Validation or calibration of model thresholds is a multi-step process, but before initiating this, it is essential to understand the business justification for the model. During the validation of an AML Compliance Model, it is imperative to first review the purpose and scope of the model, the business’s product offering and its customer footprint. Taking consideration of such qualitative factors form the backbone of the model tuning process.

The next step involves the application of statistical techniques to determine initial model thresholds. Later in this blog series, we will discuss how data quality and data analytics can be leveraged to develop robust AML Compliance models.

One of the most crucial steps of the tuning process is to review the model output by someone with knowledge of the business, such as product mix and customer footprint. This step is essential in calculating the number of false positives or negatives, thereby verifying the thresholds.

Calibration of thresholds and implementation of the model are not the end of the tuning lifecycle. In this series, we will discuss how the continued performance monitoring, and subsequent adjustments to model thresholds, are part and parcel of model development, as well as how the integration of robotic process automation could be leveraged to save resources on repetitive tasks.

Lastly, it is important to emphasize the role of thresholds validation and potential re-calibration of models as we emerge from periods of volatility. As has been observed during the COVID-19 crisis, the mix of customer profiles has changed and so have the products being offered by organizations. As these parameters evolve, it is essential for model validation teams to reconsider the role each model plays and their effectiveness in achieving its objectives.

 

Our latest guidebook provides leaders with a roadmap to enhance resiliency plans, simplify operations, address new financial requirements, and more. To download, please click the link below.