Machine learning
for fraud detection

There is some truth in that AI is akin to computers acting with human intelligence.

Machine learning is a subset of AI, and the key difference is the learning. With machine learning, we are able to give a computer a large amount of information and it can learn how to make decisions about the data, similar to a way that a human does.

Machine learning has many uses in our everyday lives – for example, email spam detection, image recognition and product recommendations. You may have seen this in your social feeds or streaming network recommendations.

Deep learning is a subset of machine learning. The key advantage deep learning gives is the ability to create flexible models for specific tasks (including fraud detection). Deep learning allows us to create bespoke models more easily and more frequently.

Traditionally, businesses relied on fraud rules alone to block fraudulent payments. Using them on their own, however, causes some issues.

False positives

Using lots of rules tends to result in a high number of false positives – meaning you’re likely to block a lot of genuine customers here to buy from you.

For example, high-value orders and orders from high-risk locations are more likely to be fraudulent, but if you enable a rule which blocks all transactions over $500 or every payment from a risky region, you’ll lose out on lots of genuine customers’ business too.

Fixed outcomes

The thresholds for fraudulent behaviour can change over time, which is something rules cannot help with unless constantly reviewed and adapted.

If your prices change, the average order value can go up, meaning that orders over $500 become the norm, and so rules can become invalid. Rules are also based on absolute yes/no answers, so don’t allow you to adjust the outcome or judge where a payment sits on the risk scale.

Inefficient and hard to scale

Using a rules-only approach means that your library must keep expanding as fraud evolves. This makes the system slower and puts a heavy maintenance burden on your fraud analyst team, demanding increasing numbers of manual reviews.

Fraudsters are always working on smarter, faster and more stealthy ways to commit fraud online. Today, criminals use sophisticated methods to steal enhanced customer data and impersonate genuine customers, making it even more difficult for rules based on typical fraud accounts to detect this kind of behavior.

Machine learning has delivered a huge upgrade to fraud detection systems, allowing for flexibility, better results and better scaling.

However, this doesn’t mean you should give up using rules entirely. Your anti-fraud strategy should still include some rules where it makes sense, and also incorporate the benefits of machine learning technology.

We use a few different forms of machine learning learning at Ravelin.

Here’s a simple explanation of how a supervised machine learning system works. You can listen to our podcast episode on machine learning to hear more detail about the process.

1. Input data

When it comes to fraud detection, the more data the better.

For supervised machine learning, the data must be labeled as "good" (genuine customers who have never committed fraud) or "bad" (customers with a fraudulent chargeback associated with them or who have been manually labeled as fraudsters).

2. Extract features

Features are calculations using data points which help describe customer behavior. And fraudulent behaviors are known as fraud signals.

At Ravelin, we group features into several feature megafamilies, each of which has hundreds or thousands of individual features:

4. Train algorithm

An algorithm is a set of rules to be followed when solving complex problems, like a mathematical equation or even a recipe. The algorithm uses customer data described by our features to learn how to make predictions – e.g. fraud/not fraud.

In the beginning, we’ll train the algorithm on an online seller’s own historical data.

We call this a training set. The more fraud in this training set the better, so that the machine has lots of examples to learn from.

5. Create a model

When training is complete you have a model specific to your business, which can detect fraud in milliseconds.

We constantly keep an eye on the model to make sure it is behaving as it should, and we’re always looking for ways to improve it.

We regularly improve, update and upload a new model for every client so that the system will always detect the latest fraud techniques and respond even better to evolving fraud trends.

At the point of the transaction, the model gives each customer a risk score on the scale of 1 to 100. The higher the score, the higher the probability of fraud.

Fraud appetite is subjective, as is fraud strategy.

With Ravelin, merchants can choose what level of risk is right for their business and set thresholds for what proportion of transactions you want to allow, block and manually review or challenge using 3D Secure.

The next step is to ask yourself, where on the scale is the right risk threshold for my business?

Threshold analysis: Precision and recall

Determining the right risk threshold involves doing data analysis based on the principles of precision and recall.

It’s a complicated balancing act between:

• True positives (how many fraudsters we block)
• False positives (how many legitimate customers we block)
• False negatives (how many fraudsters we allow)

Scale is very important here.

For context, the typical acceptance rate for a Ravelin client is usually higher than 98 or 99%, so almost all transactions are approved.

It’s within the small band of rejected transactions that the optimization occurs.

The right level of risk is individual to each business. A business with a high volume of low-value transactions (eg. food delivery) may set the risk threshold very high, so that they can ensure they are blocking the least possible amount of genuine transactions.

Our investigation analysts are experts in these calculations and can help you find the right thresholds to suit your risk appetite.

In short, yes. Any large merchant would benefit from a model based on their own data.

Ravelin knows that it’s best to use your own customer data to predict fraud for your business as it will be the most accurate at detecting fraud within your future customers.

It could be normal for someone to order from a food delivery business every day, whereas this would be very unusual for online clothes sales.

But different business models can have very different customer order cycles and amounts, even within the same sector.

There are also huge variations in other aspects. For example, it might take customers only a few minutes to order from a ticketing site, but a taxi app order can take as long as the ride lasts.

Unlike other fraud providers, Ravelin builds 100% custom models for each of our merchants, so predictions will be based on fraud signals in their customer base alone – while also taking into account consortium data as one megafamily of features, among dozens.

This stops the model being swayed by patterns in unrelated industries and unrelated companies, creating more specific predictions and better performance.

There’s always a chance that a merchant might not have enough data to train their own model right away.

A business might have a very low sales volume, mainly sell through affiliates, or sometimes the logging simply hasn’t been set up to collect the data in the right format.

It’s no problem if you don’t have enough data to train your own model right away.

To get your business up and running quickly, we’ll use a generic model based on historical fraud patterns we’ve seen before. We don’t share any sensitive customer data between businesses, but we can reuse the algorithms we’ve already trained.

This makes it easy to pick the right components off the shelf and it means you can start using a model to detect fraud sooner.

Because we use a multi-model approach, we can pick and choose the models which are most suitable for your individual business to make up a semi-customized larger model.

As soon as the model starts working on your data, it will begin to adapt and tailor to your customer base, and therefore become more effective.

The model improves as we give it more data, chargebacks and manual reviews.

Why it’s important to use historical data and not just recent data

Chargebacks and other reports of fraud lag behind real-time results.

We’ve found that within a month, we have chargebacks for around 30% of fraud, which means up to 70% of fraud hasn't been recorded yet.

If we used only the most recent data, the model wouldn't be able to distinguish the hidden fraudsters (whose actions haven't resulted in a dispute yet) from the rest of recent genuine customers.

Let’s imagine we’re building a machine learning model to detect fraud for a food delivery business. Our fictional business is called DeliverDinner.

When DeliverDinner joins Ravelin as a new client, they start to send live transaction traffic to our API.

Every time a customer registers, adds an item to their basket, or does anything on the DeliverDinner website, it sends a JSON request to the API. This means we store lots of data about DeliverDinner customers and everything they’ve ever done in their account. We bundle these into customer profiles.

To use this data for machine learning we need to do three things:

1. Label the customers as fraud/not fraud
2. Describe the customers in computer language
3. Train the model

Step 1: Assign labels

We look at any customer which has had a chargeback or which has been manually reviewed as fraudulent by the merchant, and label them as fraud.

Step 2: Create features

Creating features is basically describing each customer in a way that the computer can understand. We want to describe the characteristics of a customer which indicate if they would be fraudulent or genuine. This is based on the same aspects that a fraud analyst would look at to make the decision.

Some very simple examples of features which could be good indicators of fraud are:

• Order rate: Fraudsters order at a much more rapid pace, we quantify this as number of orders per week.
• Email: Fraudsters might have a dodgy-looking email – for instance, we may quantify this as the percentage of digits in the email address.
• Delivery location: It could be somewhere typically genuine/unlikely to be fraud like a penthouse apartment, or it could be somewhere fraudulent that implies a "drop location", such as a park. We quantify this as the location fraud rate %.
• Card velocity: The number of different cards used or attempted to be used by a customer within a reasonable amount of time can also be a fraud signal.

There are, of course, several much more elaborate features – yet the idea remains the same: Each supports the overall calculation of how likely a customer is to be a fraudster or abuser.

All features are created as a number, as the model can’t absorb raw text. We build up our features and categorize them into groups. We call these groups megafamilies – and we surface them on the Dashboard as well, to help our merchants know which aspects of a customer's presence are unusual and might indicate fraud.

Step 3: Train the model

We need to feed the algorithm the data so that it can learn how to solve the problem. At this stage, we feed in the training data.

The training data is a bunch of DeliverDinner data about customers, described in terms of their features and labels to let the algorithm know if they are a fraudster or a genuine customer. This helps the model learn how to tell the difference between genuine/fraudulent.

Within DeliverDinner’s dataset, this might show that genuine customers tend to order around once a week, they tend to use the same card each time and the billing + delivery address are often the same. Fraudsters might show that they order several times a week, use lots of different cards, that their cards have failed registration and that the billing and delivery address don’t often match.

The algorithm will take this at face value, and learn the perfect way to check if a customer features look more like the genuine customer pile or the fraudulent customer pile.

When we show the model a new customer it hasn’t seen before, it compares it with the genuine/fraudy customers it has seen before and produces a fraud score. This score represents how likely the new customer is to be fraudulent.

For the majority of customers, the fraud score will be quite low, as there are many more genuine customers than fraudsters. When it’s a low score, we recommend allowing the customer and the transaction to go through. If it’s a medium score, we recommend a Review of the transaction, eg. sending the customer a 3D secure challenge to authenticate. If the score is very high we’d recommend blocking the customer from making the transaction.

Allow, review or prevent – and how do you decide?

For the majority of DeliverDinner's customers, the fraud score will be quite low, as there are many more genuine customers than fraudsters.

• When the score is low, Ravelin recommends allowing the customer and the transaction to go through.
• If it’s a medium score, we recommend a Review of the transaction. For example, sending the customer a 3D Secure challenge to authenticate.
• If the score is very high we’d recommend blocking the customer from making the transaction.

Setting the right limits for allow, review and prevent thresholds depends on precision and recall. These are key concepts in machine learning, as our Head of Machine Learning explained in a podcast.

1. Precision asks: Of all the prevented customers, what proportion were fraudsters?
2. Recall asks: Of all the fraudsters, what proportion did we prevent?

If your prevent threshold is at 95, you’re blocking a very small percentage of customers. You’d have very high precision – you’re only blocking a few customers that you’re fairly sure are fraudsters.

This means you'll have a very low false-positive rate. However, recall is likely to be low as there are likely to be fraudsters with scores under 95 which you’re not blocking.

Let's look at the opposite situation. If you have a block threshold of 5, you’re preventing a huge amount of your traffic and so you’re likely to have very poor precision – and probably end up with lots of false positives. You will have high recall, because as you’re going to block most if not all of the fraudsters.

Setting the right risk threshold

Of course, the above are exaggerated numbers – in reality, most fraud managers would not block everyone with a score of over 5, nor would they allow through everyone less than 95.

But there's a balancing act between the two. Where you set your thresholds depends on your individual business priorities. It’s easy to tweak these depending on your risk appetite, current goals, or if you are more concerned about chargebacks or false positives.

Sometimes, fraud managers think about fraud detection in terms of "accuracy". Yet, because AI-native fraud protection such as Ravelin's is based on sophisticated machine learning algorithms, understanding precision, recall and setting risk thresholds is key for to assessing the efficiency and success of ML models, and make sure they are always is improving.

Ravelin builds custom fraud prevention for each of our merchants – which involves several models for each merchant, always improving and ensuring we continue to provide the best possible results.

Every business has a lot of data, but not all of it is relevant for fraud. Here's how we select specific data features to analyze and get an indication of fraud.

What is a feature in machine learning?

Features are the input to a machine learning (ML) model. In other words, it's the data a ML model uses to generate a prediction – and it can range from simple to very complex.

ML features at Ravelin are grouped into feature megafamilies, which share some characteristics and help provide transparency for fraud recommendations and for each client's fraudscape.

How are ML features engineered at Ravelin?

Ravelin uses a continuous integration and deployment philosophy to build machine learning models at scale to support all our ML-enabled solutions, including payment fraud and refund abuse prevention. These ML models are built on features and feature megafamilies.

For Ravelin, every company is different, even within the same sector. So we believe in building a dedicated, custom model for each merchant rather than one for everyone – or grouping them together, for example, all retailers, or all travel websites. Our ten-year experience in fraud detection and prevention has shown that the deployment of individual ML models brings the best possible results.

In practice, it is a multi-model approach: There are different models for each Ravelin client, aligned with their industry, fraud appetite, KPIs and strategy. When a new model is prepared for a Ravelin merchant, it competes against the existing model, with the best performer of the two deployed.

Types of ML features at Ravelin

The blackbox analogy has been commonly used in machine learning to describe the fact that in many cases, a human does not see the reasoning or calculations behind the output – behind the fraud recommendation, in Ravelin's case.

Simply put, users of ML technology appreciate knowing why a decision has been reached. We speak of blackbox machine learning as well as whitebox or clearbox machine learning to refer to explainable machine learning output.

We can get an understanding of how it works through testing cause and effect. We make subtle, controlled changes to the data we feed into the model and measure the output so we can tell what data the model used in the prediction.

Ravelin surfaces the reasoning behind a fraud recommendation or fraud score by providing a detailed breakdown of the types of features that contributed to a fraud score.

This conveniently explains why a customer has been scored the way they have. For example, someone may receive a "Block" recommendation and a risk score of 93 out of 100 for reasons related to the payment methods they are using and/or their history of previous orders.

Every single customer’s prediction is instantly explained in full on the dashboard. Here's an example. Each item on the left is further expandable, with sub-items for even more clarity. This way, human analysts can have more confidence in the scoring and conduct their manual reviews and spot-checks easily.

Machines are exceptionally good at doing the heavy lifting in data analysis, number crunching and output. They work tirelessly through the night and never complain about working weekends. Machine learning removes heavy burden of data analysis from your fraud detection team. The results help the team with investigations, insights and reporting.

Machine learning doesn’t replace the fraud analyst team but gives them the ability to reduce the time spent on manual reviews and data analysis. This means analysts can focus on the most urgent cases and assess alerts faster with more accuracy, and also reduce the number of genuine customers that are blocked from transacting.

Machine learning makes the role of a fraud analyst more efficient, as their time is freed up to do more strategic work. Analysts improve and optimize machine learning fraud detection systems through reviewing and labeling customers and tuning the rules.

Machines are less good at dealing with uncertainty. There are cases that are new, difficult, or somehow different. Edge cases are those that require more attention and may be difficult to determine. This is where the human insight comes in and provides massive value.

The expert human intervention here is not just at the point approving a transaction. It’s more a case of analysis after the event and labeling the data in a way that gives rapid, meaningful feedback to a machine. Remember, labelled data is the ultimate training set for a machine. So, the more confirmed behavior labels it can receive, the more accurate a result will be.

During live fraud events, fraud analysts can use the manual reviews to let us know when an attack is happening. Human insight is key to stop fraud attacks and limit the negative impact.

When used correctly, machine learning can transform the way fraud teams operate, doing the heavy lifting and examining enormous data points and features while leaving strategic decision and guidance to humans – and unlocks time for them to upskill and improve their fraud detection.

1. Super-fast, real-time decision-making

For enterprise merchants, making instant fraud decisions is crucial to ensure that the buyer's checkout journey is never delayed. Machine learning systems act like massive teams of analysts, running hundreds of thousands of complex queries in mere milliseconds.

These models constantly analyze individual shopper behavior in real-time, instantly blocking or flagging any anomalies as they happen. This speed guarantees that genuine customers enjoy a frictionless experience while bad actors are stopped immediately.

2. Unmatched scalability for growing businesses

As an enterprise scales and expands, and transaction volumes increase, traditional rule-based systems often struggle to keep up with the expanding data.

Machine learning actually benefits from this growth, as larger datasets provide the algorithms with more examples of good and bad behavior. By consuming more data, the models continuously improve their ability to predict fraud and rapidly adapt to new transactional patterns. This scalability ensures that a merchant's fraud prevention capabilities grow seamlessly right alongside their expanding global operations.

3. Operational efficiency and cost-effectiveness

Replicating the analytical power of machine learning would require hiring hundreds of human fraud analysts, which is financially unfeasible. The financial cost of running machine learning models is incredibly low, essentially just the operational cost of the active servers.

These systems efficiently handle repetitive, tedious data analysis 24/7 without fatigue, doing all the heavy lifting automatically. Human teams are freed from manual reviews, only needing to step in when highly specific insight or edge-case evaluation is required.

4. Pattern recognition and accuracy

Machine learning is far more effective than humans at spotting non-intuitive patterns and subtle trends hidden deep within large datasets. The algorithms establish a baseline of normal customer behavior and quickly detect deviations long before a chargeback is ever filed.

Models can assess complex signals, such as how long a customer browses or if they are copying and pasting details, to flag risky users. This enhanced accuracy allows merchants to significantly lower their false positive rates, ensuring legitimate revenue is not turned away and good customers don't churn.

5. Customizable to each merchant's unique priorities

Ravelin does not take a one-size-fits-all approach. Instead, models are trained on the merchant's specific historical data and tailored to their unique KPIs. Merchants can seamlessly align the machine learning thresholds with their own risk appetite, balancing chargebacks against block rates.

This flexibility transforms fraud prevention from a mere defensive measure into a strategic business enabler that protects, and even promotes, profitability. By optimizing exactly what the business cares about most, the models actively help to maximize overall conversion and secure revenue growth.