JAVA

Why Bet365 swapped Java for Erlang:

Dan Macklin, head of R&D at Bet365, explains why the gambling company is making the switch that he believes other organizations are likely to make

Madhees Techno Consulting Pvt Ltd

These days, very few businesses have the luxury of standing still – especially one like bet365. We operate in an incredibly competitive and fast moving industry. Add to that the zero cost of entry for the consumer and any interruption in the experience and you risk losing your audience.

It’s not surprising then, that there’s zero tolerance for glitches at bet365. But maintaining the peak performance of our systems comes at a cost. It’s what I call the ‘Scale/Innovation Dilemma’.

As well as ensuring our systems can scale to meet the demand, we are under constant pressure to develop leading edge products that will excite customers and continue to promote growth. Problem is when you’re seeing the kind of explosive growth that we do, you end up spending most of your time re-engineering existing systems to cope with the ever-increasing loads.

And when your developers are having to run harder and faster, just to keep up with the problems of scale, it becomes increasingly difficult for them to find time to innovate as well.

The challenge of scaling our systems was the reason the R&D team was set up three years ago. Back in 2012, we could see that our existing systems would not scale to meet user demand moving forward. So, Martin Davies, CEO of bet365’s technology business, brought together a small group of developers who would work on finding solutions to the difficult problems of scale and reliability.

The team would exist outside the commercial business so that it could devote all of its resource to the problems at hand.

The main remit of the R&D team is to challenge the conventional way of doing things and to look at different ways of solving problems. To do that, we look at both new technologies and re-purposing old technology.

In this case it’s an older technology rather than a new one that has provided a solution to our massive parallel processing problems.

Although it is only now earning column inches, Erlang is actually a very mature technology. It was developed in the 1990s by Ericsson for telephone switches. From an IT perspective, it’s interesting that the problems telephone switch providers had to deal with the in 1990s are very similar to the kind of problems we have today – problems of reliability, scalability and simplicity.

With Erlang we’ve found that we can massively reduce the complexity of our code base and build relatively simple systems at pace. Hence the switch from Java.

Our first big success with Erlang was on our system that pushes data to customers using our In-Play product. We have 17 million customers worldwide and currently, during peak times, we are making up to 100,000 changes to the system every second, whilst serving over 2.5 million concurrent connections.

In the past we used a third-party product that worked well but was becoming more complex to tailor to our needs. Written in Java, there was a lot of code to rewrite. On top of its technical complexity it also required a lot of hardware to run.

My team was tasked with making it more efficient, reliable and flexible, simplifying the code environment and reducing the hardware needed to run it.

Martin had followed Erlang’s use in large-scale middleware projects and suspected it could work for us because it was built for distribution, reliability and concurrency.

We were sceptical at first. Coding in Erlang requires a different approach to Java and for people that have built a career as Java developers, it meant changing an ingrained mindset. That scepticism, however, didn’t last long. The team picked up and adapted to Erlang very quickly and within a few weeks we were convinced it was the right tool for the job.

It took us six weeks to produce a working prototype of the new Push product despite the need to learn a new language and write the new system from scratch. It took us a few weeks more to refine the system and get it into production.

The results were amazing. We’ve achieved a profound reduction in the complexity of the code and maintenance of the system. We went from tens of thousands to hundreds of thousands of users supported on a single machine.

Significantly, we also saw a significant increase in the speed of product development and delivery.

This has seen it play an essential role in the creation of our new Cash-Out product, launched in 2014. A feature that allows users to close a bet early, before an event has finished. Something that requires massive computation of odds in real time.

More recently, Erlang has taken an important role in our migration to NoSQL. Although attractive, migrating to NoSQL is not straightforward. When legacy systems are all based on SQL systems, you can’t just switch to NoSQL overnight because re-engineering the existing system and its multi-million lines of code would take lifetimes to achieve in terms of coding hours.

By integrating our Erlang systems with Basho’s NoSQL system, Riak (also written in Erlang) we are able to build a NoSQL system in tandem with our core technology platform. Moving forward, this is likely to be the real answer to our scale/innovation dilemma.

While I would not suggest that Erlang will replace Java throughout all of our systems, it is true to say that the adoption of this technology has been the key to meeting the challenge of scale and ushering a new era of innovation into bet365.

The small amount of pain getting used to the new language has been worth it and we now have a new tool that will play a major role in the development of our next generation betting platform.

Having been through it and come out the other end with the successes we have, my advice to other software development teams would be to look outside your Java comfort zone and instead find the right tool for the right job.

Author: Ben Rossi

Advertisements

Recent Posts

Senior Java Engineer [5-7 Years]

Job Description

Are you an aspiring Senior Software Engineer looking for an opportunity to work in a fast paced and fun environment? We’re looking for strong candidate who will be responsible for creating designs and implementation for the  parts of enterprise Java application running on Tomcat. We believe that exceptional employees are the key to our success and are looking for an engineer  who can  work in a highly collaborative, distributed and cross functional Agile team and will take a lead role in providing design , solution and code for our learner management system  and also be responsible for code reviews and ensuring unit tests and overall integration tests to  help with the  quality of the product.

We are expanding our development team and adding additional positions to start developing and maintaining our enterprise learner management system which lets our 6000+ customers consume content from a variety of sources and with different features that fit the needs of learner. We believe that our customers deserve the best experience and we are delivering on those with state of the art product that  you will be working on. You could be the one to help us achieve our goal!

Position Responsibilities

  • Ability to understand  critical customer needs and provide solution, design and code for our enterprise JAVA application.
  • Work hand-in-hand with rest of the team for functional implementation and also help identify coding patterns and anti-patterns and also support of the implementation of the patterns through code reviews.
  • Helps break down, estimate, and provide just-in-time design for small increments of work.
  • Collaborates with different people in variety of situations.
  • Deliver quality micro services and promotes improvements in version control, continuous integration, project build, and project automation.
  • Test-drives clean, quality code, working closely with other Agile team members.
  • Consistently follows software development methodology

Knowledge and Skills

  • 5- 7 years of industry experience
  • Must have the ability to understand a requirements / problem, provide solution , create design documents, help with  implementation and create unit and integration tests
  • Must have experience creating enterprise web application using server side JAVA running under Tomcat web service
  • Must have experience with  web development frameworks such as Struts and Spring
  • Must have experience  with JDBC driver and SQL server technologies  such as MS SQL Server
  • Experience with AJAX UI  technologies such as EXTJS, Angular JS is a big plus
  • Experience with Elastic or SOLR a big plus
  • Experience developing micro services a big plus
  • Ability to provide support and mentor to junior developers
  • Great team player

Education and Certifications

BS/MS in Computer Science Preferred


Share Profiles on vamsi.sagiraju@madhees.com

  1. UI Developer – [3+ Years] Leave a reply
  2. QA Manager [10 -16 Years] Leave a reply
  3. Vice President Operations@Hyderabad Leave a reply
  4. Vice President(10+ Years) for IT Hub in Hyderabad Leave a reply
  5. AVP Delivery@Mumbai/Noida Leave a reply
  6. Perfect Pick – The Recruitment Process Leave a reply
  7. Perfect RPO@Madhees…!!! Leave a reply
  8. Java Developer(2+ Yrs)@Madhees Leave a reply
  9. Manager-Human Resources(8+ Yrs) Leave a reply