Blog / Fraud analytics, Link analysis, Machine learning

How to measure the success of your fraud prevention tools

Fraud prevention is a delicate balance between stopping fraud and maintaining good customer experiences. But what is the most effective way to measure this outcome?

12 September 2023

How to measure the success of your fraud prevention tools

Our research shows that all major fraud types are on the rise. And, to put it plainly, you can’t afford to get complacent. So it’s promising to see that 75% of fraud teams expect their budgets to increase. This investment includes fraud prevention technology, but how can you ensure your money is being well spent?

There’s little consensus between merchants on the most effective tool for fighting fraud. But, in truth, different solutions are effective at fighting different types of fraud. What’s important is that you have a robust tool stack and strategy that takes into account the dynamic nature of fraud – all without frustrating customers.

Fundamentally, a solid fraud prevention strategy stops fraud without disrupting customer experiences. Here, we'll help you identify ways to measure the impact of your fraud solutions on both sides of this coin.

Q1: Is your fraud prevention tech preventing fraud?

Fraud Prevention Tools

It’s important to understand whether your fraud prevention strategy is actually helping you identify and prevent fraud. Largely, this means looking at metrics related to the impact of fraud on your business as a whole.

Critical metrics to consider include:

Review rate

This is the percentage of transactions that are manually reviewed by your fraud team. Manual review is costly, and a high manual review rate would suggest that you're not allocating resources effectively.

Manual review
is most effective as a last defense against fraud to avoid false positives. You should only have to manually review orders that are difficult to decide on. Otherwise, transactions that incur a medium fraud score could be sent through 3DS.

Prevent/block rate

Also called a block rate, this is the percentage of transactions that are prevented due to suspected fraudulent activity. This is a slightly more complex metric and would very much be determined by your organization's risk appetite.

A high prevent rate would mean you're stopping a lot of fraud. But some genuine customers are likely to get caught up in the net and have their payments prevented, i.e. false positives. On the other hand, a low prevent rate would suggest better precision, but you’ll probably end up letting some fraud through.

An effective fraud prevention tool should try to balance the above and take into consideration your business goals. Your prevent rate should scale with fraud attacks and minimize false positives.

Chargeback rate

This is the percentage of transactions that resulted in a chargeback.

A high chargeback rate is often a sign of high fraud levels or machine learning models with poor recall. In other words, your risk threshold is so high you’re not effectively stopping fraud.

A good fraud prevention strategy should balance your prevent/block rate against your chargeback rate to maximize revenue but minimize risk.

Fraud losses

This is the total costs incurred by your business due to fraud. This can include lost merchandise, shipping costs, and chargeback/dispute fees. It can also include indirect losses like reputational damage.

Tracking fraud losses can provide insight into the effectiveness of your strategy and the return on investment of your tools.

Q2: How is your fraud prevention impacting customer experience?

Fraud Prevention Technology

Fraud prevention and providing great customer experience aren’t separate aims. If anything, a good fraud prevention strategy has the customer journey in mind and enhances their experience.

This involves being strategic about where you add friction. For example, adding verification steps to account creation has a far lower impact on conversion than verification steps at check-out.

When it comes to reviewing the impact on customer experiences there are two critical metrics you can look at.

False positive rate

These are legitimate transactions mistakenly flagged as fraud. Different industries and businesses will have different acceptable false positive rates. However, in all cases, false positives are terrible for customer experiences. You’ve lost a genuine sale and potentially a loyal customer.

Abandonment rate

Cart abandonment is a good indicator of too much friction at check-out. An effective fraud prevention strategy is about finding that sweet spot. Communication is also invaluable here – help your customer understand why they need to take these extra steps.

The true value of fraud prevention

The role of your fraud team and tools is to help the business see where its risks are, what its mitigation options are, and anticipate problems before they arise. Your fraud prevention strategy should be looking to answer the question – how can we combat fraud without harming good customers?

Your business, your customers and your fraud risks are unique. This is why bridging the gap between strong fraud prevention and good customer experience can be a challenge. The hallmark of a good fraud prevention strategy is using all of the tools at your disposal in ways that work for your business.

When trained on your customer data and your business data, machine learning is a highly effective and highly scalable tool to spot patterns that could otherwise be missed. Rules can then be used as a supplementary measure to quickly stop attacks with patterns that the model may have not seen before.

Manual reviews remain necessary to limit false positives. And graph networks can play an important role here by visualizing large volumes of data to quickly investigate these complex cases.

Check out stats on the tools your peers see as most effective.


Related content