Harness the power of your data
Support and investigations
Support services for Ravelin
Online payment fraud
Deep dives on fraud & payments topics
API & developer docs
APIs, glossary, guides, libraries and SDKs
Global Payment Regulation Map
Track PSD2 & more with a full report
The latest fraud & payments updates
In-depth guides to fraud, payments &
Discover the story about Ravelin
Join our dynamic team
Read more about our happy customers
Get the latest Ravelin news
Support & investigations
Accept more payments securely
Protect your customer accounts
Stop policy abuse to protect your bottom line
Ravelin for marketplace fraud
Ravelin 3DS & SDKs
Global Payment regulation map
In-depth guides to fraud, payments & security
Read more about our happy custmomers
How we’ve balanced fraud risk and friction: Deliveroo's journey with rule experimentation to reduce 3DS use by 40%...
Share this article:
Hello there! I’m Jack, a Data Scientist working in Deliveroo’s Trust team. The Trust team works on safeguarding Deliveroo and its customers against fraudulent activities and abuse. We tackle various challenges, including payment fraud, compensation abuse, and promotions abuse, among others. Lately, my focus has been on optimising our payment fraud rules. In this blog post, I will share my insights and offer some actionable advice for teams trying to do the same.
Any merchant receiving online transactions is at risk of payment fraud - this is where fraudsters use stolen credit cards to transact on their platform. The main way merchants protect themselves and their customers from payment fraud is through the use of 3D Secure (3DS). 3DS is an authentication protocol designed to provide an additional layer of security for online transactions, for example by requiring the cardholder to provide a one-time passcode to help verify their identity.
Payment fraud can impose significant financial burdens to a company’s profitability through various means, such as chargeback expenses and severe financial penalties imposed by card networks like VISA and Mastercard in cases of excessively high fraud rates. While preventing fraud is of paramount concern for companies, employing excessive and inefficient fraud prevention measures can have equally adverse consequences. These measures can detrimentally impact the experience of legitimate customers, leading to a decrease in order volume, profitability, and customer retention.
Deliveroo decides which transactions to send to 3DS through two main methods: Machine learning Models (such as Ravelin’s Machine learning model) Custom 3DS rules. These are rules manually created by our operations team, using Ravelin, to help quash emerging fraud trends they’ve identified, or that we think we need extra protection against.
Ravelin is a fraud detection and prevention company that offers machine learning-based solutions to help businesses combat online fraud. In the payments fraud space, we use their models to prevent fraud, and also use their platform to create 3DS rules.
It is worth noting that other parties involved in the payment processing flow (PSPs and Card Issuers) can also trigger 3DS based on their own fraud engines or fraud rules, so the overall 3DS customer experience is not only dependent on Deliveroo.
So, why not just 3DS everyone?
Whilst 3DS is highly effective in preventing fraud, it comes with its trade offs. Whenever you authenticate a customer, you introduce friction into their purchase journey, and inevitably cause a percentage of transactions to be lost. So, whilst we could send all orders to 3DS and eliminate almost all fraud, we’d see sizable drops in both order volume, profit, customer retention/experience, and negatively impact what is mostly genuine customers. Indeed, extremely high 3DS rates sometimes result in lower authorisation rates as Issuers rate the merchants as riskier.
The below outlines how we at Deliveroo are thinking about 3DS:
The question we first and foremost needed to answer was: how good are our 3DS rules? To do this, we conducted A/B tests (or experiments). We conducted large scale A/B tests over Q4 2022. For each rule that we experimented on, we divided our users into two groups – the control group and the experimental group. The control group was subject to the existing 3DS rule as usual, while the experimental group was exempted from 3DS for that particular rule. We then monitored the performance of both groups. We chose to experiment on the highest volume rules first for speed and practicality. There are a lot of interesting quirks around 3DS experimentation such as the randomization unit, and challenges with overlapping rules which are out of scope for this blog post.
In practice, we ran all of the experiments through Ravelin. Ravelin’s rule platform and UI makes it extremely easy to run A/B tests, without the need for engineering work traditionally required for experiments such as adding feature flags and relevant data logging. We used the Ravelin tags feature to split our population into groups, and made use of Ravelin’s rule hierarchy to exempt the correct orders from 3DS.
To evaluate the rules, we devised an equation that accounted for various financial factors beyond fraud prevention alone that the rule could have. Instead of just thinking about the fraud it prevented, we added in information about the operational profit, 3DS fees, and any additional compensation costs. Below shows the equation we used.
Cost benefit = Additional Operational profit + 3DS fees saved - Additional Chargeback costs (inc. fees) - Additional Compensation costs
Another thing we consider when removing rules is fraud rates - Card schemes (such as VISA and Mastercard) impose strict penalties if you breach certain fraud levels. We always make sure to keep well within card scheme rules by keeping our fraud rates very low.
Based on our analysis of the data, we identified a large number of rules that were overly restrictive and were on balance net negative to the business. By optimising those rules we have been able to and reduce our 3DS rate by nearly 40% without exposing Deliveroo, our customers or partners to additional fraud risks. As a result, we’ve been able to reduce friction for genuine customers, improving conversion for Deliveroo and increasing orders for our partners and riders. A win-win-win-win.
Jack Dai, Data Scientist at Deliveroo
Blog / News
E-commerce CFOs need to understand the scale of the fraud risk that their businesses face. Our survey dives deep on where smart CFOs are directing investment to keep their companies secure...
Gerry Carr, CMO
Blog / Account Takeover
Two-factor authentication (2FA) is a widely used security measure designed to prevent account takeover (ATO). But there are very real gaps and limitations in its effectiveness that fraudsters can exploit...
Clayton Black, Product Manager
How have global approaches to fraud & payments changed over the past year? Get the latest data and find out what’s top of the agenda for over 1900 ecommerce merchants in the 2023 Global Fraud Trends Survey…
Lola Omo-Ikerodah, Content Writer
Subscribe to our newsletter to get the latest fraud & payments updates
sent direct to your inbox.