Machine learning and fraud detection: PhDs give their views

Over the Summer we were fortunate enough to work with the S2DS 2015 programme. The goal of the S2DS is to bridge the gap between academia and the workplace for students emerging from PhD studies. There is a really polished video you can watch below which gives a terrific overview of what what goes on there. We were lucky enough to have 4 students work with our data scientists to use various techniques to create machine learning models to detect fraud – evolutions of which we are using today on the data of our beta customers - read more about this here.

Following up their time with us we asked the students to share with us their thoughts on whether they thought machine learning was a useful way to detect fraud and and some more detailed information on their experiences in Ravelin. We thought they were interesting enough to share and we will over the course of the week.

You can watch a recap here.

The first interview is with Fabian Peters, a PHD Student (Grand Challenges Scholarship) at University College London.

Q. What is machine learning good at in detecting fraud and what weaknesses do you think there are?

Many, if not most of the features going into the machine learning model are actually quite straight forward and are used in very similar ways in ‘classic’ fraud detection. However, it is finding the optimal combination of these many indicators of fraud – which taken by themselves are often rather weak – and assigning them the right weight that machine learning excels at. Also, good machine learning models turn what classically is often treated as a binary answer into confidence measurements and thus allow customers to pick their own trade-off of precision and recall. Finally, good machine learning is flexible and may use multiple combinations of features, making it able to cope with missing data.

The big challenge in machine learning, I think, is to remember that there are real moral implications to how the numbers produced by the algorithm are used and that the fact that a machine make the decision does not free one from responsibility.

Q: Which machine learning techniques did you deploy and which do you think were the most effective?

LogReg, Random Forests, and Gradient Boosted Trees. GBT was the strongest candidate but RF has the best balance of accuracy, speed, and interpretability and seems like the perfect basis for model development. In the end, I think model stacks (combinations) are the way to go.

Q: Were there techniques you decided against and why?

I tried to stay away from everything too black-box-ish. In particular neural nets, but for starters also GBT. I think that in the feature building stage, interpretability tops small increases in performance.

Q: Would you recommend a merchant employs ML technology to detect fraud?

Absolutely – I think it is the best way to detect fraud whilst also staying in control of how to deal with it (probability scores)

Q: Are there any things during the programme that you think are particularly interesting and would like to share?

I think I learned that understanding your data and being rigorous about cleaning it and filtering out ambiguous data points often more effective than using more powerful algorithms.

To learn more about machine learning visit our insights page.