Protecting the entire customer journey using maths and data

When people ask me what Ravelin does, I always say that we use maths to fight crime. It may be a simplistic way to describe fraud detection, but it’s true.

Here’s how we use math and data to fight crime and protect the customer journey at Ravelin, with the help of machine learning features and models.

Data and checkpoints

There are many checkpoints along the customer journey where your customers or your business might be targeted by fraudsters – such as log in, check out pre-authorization, and payment method registration.

A flexible approach with different models at different checkpoints gives you the greatest chance to prevent crime by both making it harder to commit fraud and abuse and by increasing the chances of catching fraudsters and abusers.

This offers the advantage of agility and flexibility. In terms of fighting fraud, it means you can apply the relevant layers according to where your data shows you are vulnerable.

Our first consideration is whether we have enough data and enough information, from which we’ll build our features and train our machine learning models.

We want to find out:

• Who are our fraudsters and potential fraudsters?

• What are their tactics?

• What points of the customer journey are vulnerable?

If we don’t have enough data to start answering these questions, then the first step is to try to get more data.

Once we have that data, let’s discuss what we should be doing with it and what we should be asking it to illuminate the checkpoints where we are being targeted.

From raw data points to combined features

In machine learning, a feature is defined as a measurable property or characteristic extracted from a dataset.

Features can be simple, raw data points, or they can involve complex transformations using feature engineering techniques.

But what does this really mean?

You’ll be familiar with order value, or the payment method type, or the payment gateway – these are features by themselves. We can also do transformations on these, to gain deeper insights. And these are what we call features.

Here’s a simple feature that comes from raw data: the order value. This can be valuable on its own when it comes to feeding data into our ML models, or to building rules.

But we can do more using math, to transform this into a deeper insight.

For example, compare the order value for any given order to the average order value over the entire customer base, giving us insight into the relative size of a given order.

Here, order number three is nearly 2.5 times the average, while order number five is less than half the average order. We call this feature Normalized order value.

This can be taken further by combining different types of data, like combining order data with location data to calculate the average and normalized order values for customers in different markets.

This is useful to know when looking for unusual, divergent behavior that might indicate a pattern of fraud or abuse.

What should I ask my data? Robust features

Another consideration when we build features from our data is whether or not these features are robust to changes in data and trends.

In the context of fraud, this means trying to ensure that our machine learning features can keep up with changes in fraudsters’ tactics.

Let’s illustrate this with an example:

You might find that a lot of fraudsters are using the domain @thisishighrisk.com. So, you block all customers using this email domain with a rule, or create a feature like “email = thisIsHighRisk.com” and feed this into an ML model.

This would ban every customer whose email is @thisishighrisk.com, and might help in the short term. But what if the fraudsters change tactics, and start using another email domain, such as @thisisabsolutelyfine.org?

Maybe you can then tweak your rule or rewrite your feature and retrain your model.

But what if this was happening on a scale of several email domains? Perhaps it might still be possible to track this manually and adjust your rules and features.

But what if the scale is hundreds of fraudsters using dozens and dozens of different email domains? We can see that this is quickly becoming difficult to track manually.

And it’s when it’s difficult to track something manually that machine learning shines.

A better feature here would be, for example, to calculate fraud rate by email domain.

In this case we can see @dodgydomain.com has a 99% fraud rate and angelicemails.com has a 1% fraud rate.

This of course says that a new user whose email is at the former is quite likely to be fraudulent, and one that has the @angelicemails domain is likely to not be fraudulent. Of course, this is just one fraud indicator – one feature of many taken into account by the model.

By using a feature which tracks the fraud rate by email domain, and other such features, we are able to keep up with the fraudsters’ tactics: emailDomainFraudScore.

In general, if your fraud detection is just looking at a specific characteristic rather than something that is dynamic and can be tracked over time, then you may struggle to scale your fraud prevention strategy and remain reactive rather than proactive.

What should I ask my data? Variety

We also want to make sure our data is varied. We don’t just want to collect the same data in different ways.

We want to look for new sources of data to shine light on our very diverse and inventive fraudsters and their methods.

For example, I’m trying to protect a dark alleyway and have already installed CCTV. I can upgrade it from 720p to 1080p and improve our ability to catch criminals.

However, I’m probably better off installing:

• better lighting

• alley gates

• night vision cameras

rather than upgrading the existing CCTV.

Even outside of the fraud use case, when building a model, ML practitioners advocate for having as diverse a dataset as possible as this makes your models more robust to real-world use cases.

What does this mean in practice, for fighting fraud at different checkpoints of the customer journey?

If you are already collecting order and payment data, you might want to consider collecting other data types – such as location data and device data – before upgrading the quality of data you already collect.

Here’s how we take advantage of diversity in our data at Ravelin, to combat fraud. We group features by megafamilies, and these include:

We have our “bread and butter” features – like order and payment method features. Examples of these are order value, time of order, gateway, payment method type…

These are raw features. As we saw earlier, we can do some maths and transformations on these to get more complex and meaningful features.

For example, here is the formula for normalized order value, starting with our raw order data:

Another example. From payment data, we can calculate the payment registration velocity.

It shows how many payment methods the customer has added, considered within a certain time frame. A fraudster looking to test stolen card credentials will add several payment methods, so this feature could be a strong indicator of fraud.

We’ve got a whole host of other features, such as devices, consortium, network. And the list goes on…

But when we think about stopping crime, not everything we do to protect the potential victims is equally important. With our fraud prevention models, too, not all types of features are equally important.

In this chart, we can see a list of feature importance overall at Ravelin, across all of our clients and all our verticals.

Order and payment features are the most important but many different types of features play a part.

To demonstrate this, we can now look at this same data – this time aggregated by sector or vertical.

Each bar here represents the feature importance within a sector.

We can see here that different types of features are important in different contexts. This is because the fraud landscape varies a lot from sector to sector.

In this example, network features are very important in retail, and NLP features – which use the item names and how they relate to one another – are more important in the digital goods sector.

We can drill into this on an even more granular level. These two bars represent the feature importances for two different client models within the same sector.

We can see that even within the same sector, we have very varied feature type importance:

On the left we see that customer and network data are more important for that model whereas, on the second model on the right hand side, orders and IP data are more important.

This chart here shows feature importances across a selection of some of our individual client models – with each vertical bar representing the importance of features within an individual model.

And here are some more examples, for different clients:

This shows us once again that different types of features are important in different contexts.

And this is why we believe in custom models for each of our clients at Ravelin: Because the fraud patterns and tactics can vary even within the same sector.

Now that we’ve covered the importance of having diversity in our data, I’ll spend a bit of time talking about some of the more complex features we use at Ravelin: graph network features and consortium data features.

Using math for network features

Network features, also known as graph network features, are built using link analysis to identify connections between different elements of our data.

At Ravelin, we use an in-house graph database called Connect, which can visualize shared connections between various types of data to provide ML features, as well as being used in fraud rules and supporting manual investigations and actioning.

Each type of data on a graph database is called a node, and the connections between them are called edges. In our case, the nodes are emails, phones, devices and cards and tracking how they connect to each other helps us to identify fraud rings.

We use these in rules, but we don’t stop there. We use the mathematical properties of the graph to derive features and input them into our ML models.

This means that by training on historic network data, we can identify fraud rings and put a stop to them before they have the chance to do any damage.

These types of features are useful not only for card-not-present (CNP) fraud but also for account takeover prevention, where a single device can be used to try to take over hundreds of customers' accounts.

The example below looks like a fraudulent network with hundreds of customers and email addresses is all connected to a single device.

In this screenshot from Connect, there are hundreds of cards, email accounts and customer personas all sharing one device. This can alert us to a case of multi-accounting – commonly used to conduct CNP fraud as well as policy abuse.

And this example is more likely to be a genuine network:

An example of family members using the same device, phone number and payment card (as well as others) to shop online.

Examples of network features include the number of customers connected to a device or any other node, and the network edge growth rate, which tells us how quickly a network is adding edges:

Using math for consortium features in fraud prevention

Our consortium data comes from leveraging fraud signals across all of our merchants’ data, not just the one merchant of a given model.

This is done securely by anonymizing any identity characteristics before we calculate the features. It works by cross-checking customers’ identities over our entire database of 9+ billion identity elements to see if we’ve seen this characteristic before and asking questions like, “Was it genuine or fraudulent last time we saw it?”

Based on this data, we can build fraud rates on a variety of these identity elements, using different label sources, including manual reviews and chargebacks.

An example of a feature here is ConsortiumEmailProviderFraudScore, which is similar to the emailDomainFraudScore we saw earlier – but this time applied over our entire database rather than just a single merchant’s data.

Protecting the entire customer journey

Now that we’ve talked about having diverse data, building features from it and making sure they’re adaptive, how do we apply these insights to protecting the customer journey?

Ravelin’s multi-layered approach involves several checkpoints along this journey. By deploying different ML models that make use of maths at different points of the customer journey, we increase the chances of stopping fraud and abuse.

To illustrate this layered approach, let’s take the checkout checkpoint as an example. CardNotPresent and AccountTakeOver are 2 distinct fraud vectors at this part of the customer journey, and we have custom models operating on both.

We use different combinations of labels:

For CNP, we use chargebacks and manual reviews. And, for the account takeover checkpoint, we use failed login rates, manual reviews and suspicious device characteristics.

Whereas in CNP and ATO we are focused on stopping fraudsters, in transaction optimization we focus on making the journey as smooth as possible for genuine customers. Here, we use historic transaction data and issuer intelligence to give genuine customers the best chance of making a successful purchase.

Summing up

Maths and data can shed light on important questions such as:

• Who are the fraudsters and abusers?

• How are they committing fraud or abuse?

• Where in the customer journey do they target this business?

Make sure you get as much context for these as possible. In our case, we can do this by ensuring that our sources of data are diverse.

Be curious; ask the data lots of questions, to maximize insight. Once we have some insights we’re confident in, we can think about how we can apply our learnings to increase the effort and risk for the fraudsters and make the journey smooth for the genuine customers – thus hitting our business goals.

My parting advice? Be curious about your data, ask it questions, and use math to uncover the answers.

Book a call with the Ravelin team to discover how our multi-layered, multi-model maths-first approach to fraud detection can support your business.

Related resources

• Webinar on-demand: Demystifying machine learning

• How Ravelin selects and engineers ML features to detect fraud

• Case study video: How Spotify uses machine learning to prevent fraud

• AI vs AI report: Fraudsters vs fraud fighters and the future of fraud

• Consortium data at Ravelin: A nuanced approach