Start with simpler, more transparent, and explainable and bias-free models and graduate to complicated models over time.
MODELS – Experiment and evolve ML is a powerful tool for fraud prevention, but if not done right, it is remarkably easy to build models that are counterproductive to goals.
It is vital first to develop the organization’s ML muscle.
Principle 7: ML models need a consistent goal, a north star metric that aligns with the overarching strategy Choose a metric that pairs both a measure and a counteracting measure to protect against overreacting in one direction.
For example, a team could decide to reduce the portion of fraud the model correctly catches (minimize “recall”), while deciding on an upper bound for the portion of legitimate customers that the model incorrectly tags as fraud (cap the “false positive rate”).
Finally, to make the numbers tangible, estimate the resulting cost to the business based on the cost of rejecting a good customer and the cost of unidentified fraud.
Principle 8: Develop multiple models and retrain often to align with the real world of fraud ML models try to mimic the real world.
The real world of fraud has a couple of realities that your models should handle.
First, fraud characteristics could vary a lot across geographies and types of fraud.
Build geo- and use-case-specific models if they perform better.
Second, the real world is dynamic, and fraudsters keep evolving their tactics.
Keep a constant flow of new data to retrain models to ensure that the quality of model output does not degrade over time.
Principle 9: Learn from other ML use cases with characteristics similar to fraud Nearly all of the ML modeling issues teams face in fraud have analogs in other fields with recommended solutions.
Experiment with ideas from these analogs.
Take the example of imbalanced class distributions in fraud, where nearly all the records in the data belong to the non-fraudulent category.
This problem is similar to cases such as product defect detection.
Or consider the issue of the fraud model in production biasing the output, impacting the ability to get additional data for continuous learning.
This counterfactual evaluation problem is one that the online advertising industry also faces, and teams will be able to find several ideas for experimentation.
To derive real value out of ML for fraud detection, your team must treat ML as an organizational capability.
It calls for product, engineering, data science, and privacy teams working together.
A company’s success will hinge on implementing working models that solve real business problems.
Start small, experiment, and iteratively grow your capability.
Over time, your business will not just survive, but also start thriving.
[1] The Nilson Report 2018, “Card Fraud Worldwide” Sign up for the free insideBIGDATA newsletter.
.