Why bet365 bought Basho’s Riak NoSQL key data store and set it free

Riak exploded onto the NoSQL scene in February 2012, to much fanfare. And for good reason. Rather than develop a shiny new technology and then find a market, the founders at Basho Technologies had a better idea. They set their sights on tackling a significant issue experienced by internet companies operating at massive scale. How to reliably store and distribute huge volumes of concurrent data in real-time.

Taking its inspiration from computer science, the team took academic theory from Amazon’s dynamo white paper, CAP Theroem and CRDTs and transformed it into reality. Written in Erlang, a language developed by Ericsson in the 80s to manage telecoms data flows, the resulting NoSQL database offered unmatched data throughput and resilience. 

Riak secured immediate interest and found quick success for two reasons.  First, it solved a very specific set of real-world problems that were widely understood and to which no one had found a workable solution.

Second, it appealed to the developer mindset. The Ricon.io Conferences had a heavy computer science bent. Far from hyperbole, it was something you could go to Github, download and use. Open sourcing the code gave it velocity. It filled a vacuum and a community built around it very quickly.

We brought Riak on board in 2013 as part of our first wave of Erlang development. bet365 is a SQL house but there were certain data sets we couldn’t put through our legacy systems because the databases couldn’t scale to accommodate the load.

Our betting platform supports millions of users, playing across millions of markets and processes gigabytes of data a second. It’s a mammoth task and one that our SQL systems were incapable of supporting.

Not only could Riak handle the huge data sets we threw at it, it enabled us to radically reduce the complexity of our betting system.

Crucially, Riak was also reliable. During our exhaustive testing, while some competitor databases experienced bleed under failure scenarios, Riak didn’t. In a tightly regulated market like gambling, if we lost even a minor amount, we’d be turning off the lights and closing the doors.

We also bought into the Enterprise license because it offered Multi-Datacentre Replication, which was essential for disaster recovery and business continuity.

Then something unexpected happened. Rather than evolve the core technology further, the company appeared to move away from Riak’s primary proposition.

Having enjoyed successful rounds of funding, Basho began to bolt on new features in what felt like an attempt to broaden its appeal.  Most notable was the Time Series that tapped into the then embryonic IoT market.

The strategy created buzz but in the final analysis, that was possibly all. In July this year, Basho went into receivership, putting our project and 1,000s of others in jeopardy.

That was until the possibility of acquiring the IP arose and an idea began to form. One where we would ensure the survival of the Riak code and continue to help drive its development. As we thought through the implications, it started to make more and more sense.

Buying lock, stock and barrel

We saw it as an opportunity not a strategy. This was not a commercial decision. We have no interest in making money from Riak. We don’t want the support contracts nor is there a plan to spin out new IP for sale. We see it as a chance to save a core technology and ensure it survives and thrives.

By buying the Riak IP, lock, stock and barrel, we can ensure that anyone working on it can do so unhindered. While projects could still have continued had no one bought the code, it would have meant turmoil for us and others as the Github repositories would have closed and the links into them ceased to function.

In the long-term, the community that had built up around it would have disbanded, creating potential for a splintered evolution of the product. At worst development would have ceased altogether. The best thing for all concerned is business as usual.

For us, the best way to achieve both our long and short-term goals is to make the entire codebase available to all and that is exactly what we are going to do. We see this as a community project. One where we all have a voice and the opportunity to steer the development roadmap.

We have a rough idea of where we want to take Riak. We will begin by bringing some of the essential enterprise functionality back into the core open source product. Up first is multi-datacentre replication.

Basho had made replication work but it wasn’t as efficient as it could be. There are also a couple of other areas we are looking at. Big sets, which is the Riak equivalent of Big Data is one and indexing is the other. We can currently index data manually but want to automate it.

On 12th & 13th October we will start to bring our vision to fruition with a 2-day workshop. We will be inviting the Riak community to come together to discuss the future direction of the technology. One of the key decisions that need to be made is which open source licence we are going to use.

It’s a simple but key question. To be successful we have to ensure that the license benefits the community as a whole as well as individuals who want to take the technology in a new direction.

It’s hard to say whether you will see more deals of this nature but it is an example of the growing influence the Sports Betting industry is having on the direction of Enterprise technology.

Andrew Deane, Systems Manager, Middleware, bet365
Alan Reed, Head of Systems Development for Sports, bet365

Image source: Shutterstock/Kritchanut