Success at this year’s Smart Hackathon

Transforming an idea into a business solution in just 24 hours

By

Olga Evstafyeva

28/7/2022

I joined Smart as a Senior Data Scientist just over a year ago, right as the 2021 Engineering Hackathon was taking place. I remember thinking that it sounded like an awesome event for a number of reasons:

  • 24 uninterrupted hours to be innovative and concentrate on delivering a project
  • You are provided with plenty of drinks and snacks (key!)
  • Your colleagues are there to cheer you on throughout the event, meaning that the atmosphere is fun and you can meet and interact with a lot of new people

I was very keen to get involved in 2022 to showcase my skills and innovate new processes at Smart. When one of the back end engineers got in touch and described a Hackathon idea with great potential, we quickly formulated both a plan and a team to execute it. 

Our diverse team consisted of two QA engineers, a back end engineer and two data scientists. 

The idea was to use anomaly detection methods to monitor financial transactions processed through the Smart platform, harnessing technology to simplify complex processes. The solution would need to automatically identify any transaction that does not follow the normal path and might need manual help to be resolved.

The approach

What we wanted to address was clearly a data science problem. In fact, there is a lot of value to using machine learning algorithms rather than pre-programmed rules to support this functionality of the platform infrastructure. Using a machine learning algorithm would enable us to develop a solution that is agnostic to changes in business rules. 

In this case, we chose to combine two complementary techniques: unsupervised algorithms to learn new anomalous patterns, and supervised algorithms to continuously learn from past anomalies and enhance performance. 

Importantly, the data was not labelled, which meant that transactions were not identified as “anomaly” or “not anomaly”. Therefore, we needed to use an unsupervised learning algorithm to identify the most common patterns in the data. All records would then be assigned anomaly scores, based on how different they were to the most common patterns. In the event that a score exceeded a set limit, the record would be considered an anomaly. 

Using this process, we could label the data and then pass the new dataset to train a supervised learning algorithm. The resulting anomalies would be presented to the Operations Teams, who would check the anomaly suggestions and label them ‘true’ or ‘false’. This newly-labelled data would be used to improve the supervised learning algorithm, as shown below. 

The implementation

We wanted to make sure that we explored any new patterns in the transaction data, but also used already gathered information on known outliers to make our model continuously smarter. We combined a number of models, and by 9pm, we were happy with how the algorithms were performing and the outliers we found. 

After confirming several outliers with the QA engineers, we had a well-deserved coffee break and started building an interactive report showing the outliers we found. 

The results

We worked through most of the night to stitch it all together, finishing our last team meeting just in time for the 10am Hackathon presentations. When the presentations started, we were all amazed at the standard of the different projects presented. In just 24 hours, teams from across the company had created some incredibly creative and innovative things. 

This made it even more incredible when our team was announced the Smart Hackathon 2022 winner! The judge’s feedback stated that they could see genuine commercial value to the business from our idea and how it had been executed.

It was such a fun experience – we had an amazing team and a fun project to tackle. The adrenalin and excitement meant that the lack of sleep had been worth it!

We’re really lucky to be given the opportunity to contribute to the wider business with our idea and we cannot wait to see where it goes. I also hope this experience will lead to more ideas coming from the business to the Data Team, so we can find other new ways to transform pensions, savings and financial well-being, across all generations, around the world. 

Find out more about the Smart Hackathon 2022 here.

THIS IS JUST USED FOR RICH TEXT STYLING

The rich text element allows you to create and format headings, paragraphs, blockquotes, images, and video all in one place instead of having to add and format them individually. Just double-click and easily create content.

test for a pullquote here needs to be separate to test

Static and dynamic content editing

Static and dynamic content editing

Static and dynamic content editing

Static and dynamic content editing

A rich text element can be used with static or dynamic content. For static content, just drop it into any page and begin editing. For dynamic content, add a rich text field to any collection and then connect a rich text element to that field in the settings panel. Voila!

How to customize formatting for each rich text

Headings, paragraphs, blockquotes, figures, images, and figure captions can all be styled after a class is added to the rich text element using the "When inside of" nested selector system.

alkdhjda

  • Bullet 1
  • Bullet 2 aodj afhiahf iafhpioafh paofh oapfh aopfhaofh aofhaofh aofh aofhafo ahof haf
  1. Bullet 1 apofdhjaof haopfhoap fhaopfh aofhaofh aofhaofhaofh aofhaofh
  2. Bullet 2

FOOTNOTES
CONTRIBUTORS

Written by Olga Evstafyeva

Success at this year’s Smart Hackathon

Transforming an idea into a business solution in just 24 hours

I joined Smart as a Senior Data Scientist just over a year ago, right as the 2021 Engineering Hackathon was taking place. I remember thinking that it sounded like an awesome event for a number of reasons:

  • 24 uninterrupted hours to be innovative and concentrate on delivering a project
  • You are provided with plenty of drinks and snacks (key!)
  • Your colleagues are there to cheer you on throughout the event, meaning that the atmosphere is fun and you can meet and interact with a lot of new people

I was very keen to get involved in 2022 to showcase my skills and innovate new processes at Smart. When one of the back end engineers got in touch and described a Hackathon idea with great potential, we quickly formulated both a plan and a team to execute it. 

Our diverse team consisted of two QA engineers, a back end engineer and two data scientists. 

The idea was to use anomaly detection methods to monitor financial transactions processed through the Smart platform, harnessing technology to simplify complex processes. The solution would need to automatically identify any transaction that does not follow the normal path and might need manual help to be resolved.

The approach

What we wanted to address was clearly a data science problem. In fact, there is a lot of value to using machine learning algorithms rather than pre-programmed rules to support this functionality of the platform infrastructure. Using a machine learning algorithm would enable us to develop a solution that is agnostic to changes in business rules. 

In this case, we chose to combine two complementary techniques: unsupervised algorithms to learn new anomalous patterns, and supervised algorithms to continuously learn from past anomalies and enhance performance. 

Importantly, the data was not labelled, which meant that transactions were not identified as “anomaly” or “not anomaly”. Therefore, we needed to use an unsupervised learning algorithm to identify the most common patterns in the data. All records would then be assigned anomaly scores, based on how different they were to the most common patterns. In the event that a score exceeded a set limit, the record would be considered an anomaly. 

Using this process, we could label the data and then pass the new dataset to train a supervised learning algorithm. The resulting anomalies would be presented to the Operations Teams, who would check the anomaly suggestions and label them ‘true’ or ‘false’. This newly-labelled data would be used to improve the supervised learning algorithm, as shown below. 

The implementation

We wanted to make sure that we explored any new patterns in the transaction data, but also used already gathered information on known outliers to make our model continuously smarter. We combined a number of models, and by 9pm, we were happy with how the algorithms were performing and the outliers we found. 

After confirming several outliers with the QA engineers, we had a well-deserved coffee break and started building an interactive report showing the outliers we found. 

The results

We worked through most of the night to stitch it all together, finishing our last team meeting just in time for the 10am Hackathon presentations. When the presentations started, we were all amazed at the standard of the different projects presented. In just 24 hours, teams from across the company had created some incredibly creative and innovative things. 

This made it even more incredible when our team was announced the Smart Hackathon 2022 winner! The judge’s feedback stated that they could see genuine commercial value to the business from our idea and how it had been executed.

It was such a fun experience – we had an amazing team and a fun project to tackle. The adrenalin and excitement meant that the lack of sleep had been worth it!

We’re really lucky to be given the opportunity to contribute to the wider business with our idea and we cannot wait to see where it goes. I also hope this experience will lead to more ideas coming from the business to the Data Team, so we can find other new ways to transform pensions, savings and financial well-being, across all generations, around the world. 

Find out more about the Smart Hackathon 2022 here.

About Smart

Smart is a global savings and investments technology platform provider. Its mission is to transform retirement, savings and financial well-being around the world.

Smart partners with governments and financial institutions (including insurers, asset managers, banks, financial advisers) to deliver retirement savings and income solutions that are digital, bespoke and cost efficient. In addition to the UK, Smart is operating in the USA, Europe, Australia and the Middle East with more than a million savers entrusting over £4 billion in assets on the platform. 

Smart supports its clients with a 750 strong global team.

Legal & General, Fidelity International Strategic Ventures, J.P. Morgan, the Link Group, Barclays, Natixis Investment Managers, DWS Group and Chrysalis Investments are all investors in Smart.

We tweet as @SmartPensionUK.

For media enquiries

Email: [email protected]