As long as there has been data, there has been a need to put that data somewhere. This led to the creation of the database, and these data management systems have been evolving ever since. Now, databases are shifting more and more into the cloud – Gartner has predicted that 80 per cent of new databases will be implemented in the cloud, rather than on on-premise infrastructure, as more companies use databases for analytics and follow a SaaS model to achieve this.
This change in approach has been driven by the sheer volume of data that companies want to store today on one side, and by the need to make use of data more quickly on the other. From broad use cases like analytics through to more specific machine learning or artificial intelligence, using the cloud to host this data is perceived to be simpler and initially more cost-effective than building out more IT infrastructure in the data centre.
Cloud services should provide faster scale and more ease of use, but it’s not as simple as just implementing on ‘someone else’s computer,’ as the famous joke goes. Instead, there are many more options available to make use of cloud to host, run and manage those new database instances. So how can you choose the right one?
Decision #1 – Should you pick a database, a managed service, or a DBaaS?
Firstly, it’s important to understand that not all options around running databases in the cloud are created equal. It’s worth spending time to understand how you intend to use your databases over time. This is essential to get an ‘apples to apples’ comparison between cloud database offerings, and how well they suit your needs.
As part of this, you should start by asking yourself how much control you want to retain over your new database instance. This should be simply about how happy you feel at running in the cloud, although that is an important question to ask. It also involves understanding how much this workload may need tweaking or changing over time in order to deliver value back to the business.
For some workloads, a fully managed service might be the best option. This “database as a service” option involves letting the cloud company provide the full database instance and manage it over time. It’s great for low value databases where the initial implementation is not expected to change much over time, but the data itself needs to be saved. These instances will sit in the cloud, accumulate data, and be there when you need them to be. The drawback is that these services tend to be very limited in how much customisation they can support – if you need more flexibility then these options won’t be right for you.
For other workloads, you may need more control over the parameters involved. The database itself may power an application that will grow or evolve over time, or where the data it collects will power a business critical service. To run this kind of database, you have to be able to manage the database instance and make changes that suit you, rather than what your cloud provider thinks is best.
Over time, this can be more expensive than looking at the capital expenditure associated with running your database on-premise. Judging the financial impact of upfront spending against long term costs is an important financial consideration, but it is also important to look at what economists call opportunity cost too – this is what you would have done with your time and money if you don’t go ahead with a project, or what you can do instead with the time you free up. Using cloud, you can get ahead on other projects when you don’t have to run all your activities based on internal IT and resources.
However, it’s not as simple as just handing over to a cloud provider. It’s also essential to confirm what responsibilities exist around the database service and who will handle them. Behind this point is the real world experience that not every “fully managed” database service is the same. For all cloud services, there will be some responsibility on you as the customer for how you manage and maintain your data, while the cloud provider is responsible for their infrastructure; in between these extremes, there may be some confusion over who is responsible for which requirements.
Decision #2 – Should you pick a specific cloud, hybrid or multi-cloud?
So, you have decided on the kind of cloud database service you want to run. Now it’s time to pick the cloud provider that you want to work with.
There’s a great word – fungible – for when you can easily replace any individual item with another and there is no difference to the experience. A good example here is the choice around a pen, pencil or crayon. Each one of these can make marks on paper – if that is all you want, then therefore they are fungible and you can pick any of them. What they are not, however, is the same.
Cloud providers are the same at the base level – whether you prefer Amazon Web Services, Microsoft Azure or Google Cloud Platform, they all provide cloud services that are similar but not identical. Other regional cloud providers in Europe or Asia can provide similar support for cloud services too, but again they are all different. Depending on what you want to achieve around moving your database to the cloud, you may have a greater choice in who to work with or not.
Similarly, it’s also worth looking at whether you will use a hybrid or multi-cloud approach over time. Hybrid cloud involves linking up internal IT with an external cloud service, while multi-cloud involves using two or more cloud services together. Alongside providing additional availability compared to using a single cloud service, hybrid and multi-cloud services provide you with a way to retain more control over your data rather than being locked into a specific service. This can also make it easier to port data between services or locations if you need to move.
Decision #3 – what database is actually the right one for you?
The third big decision around cloud databases is which database you are going to implement. From the small number of relational databases that were previously available, there are now a plethora of different databases that can be used. These new services and options have been developed as companies have created more data and more specific ways in which that data may be used.
To begin the process of choosing your next cloud database, it’s important to look at what your use case is and find the right option for that project. For example, if you require fast scale and deployment for your next database project, then a NoSQL database like MongoDB might be suitable. This scales rapidly and is very easy to deploy, making it a great fit for new applications. Conversely, it is not available as a managed cloud service except from MongoDB itself due to the database’s licensing, so you may have to implement in your own cloud instance if you want to go down this route.
For other applications, a relational database may be the best fit. Offerings like MySQL have been popular for years, and the support for SQL means that the number of potential database administrators that can help you run any new database instance in the cloud should be high. Relational databases remain a great fit for use cases where schema are well-defined at the start, so moving into the cloud should be a simple enough move. There are plenty of different deployment options too, from fully managed services and DBaaS products through to deploying in your own public cloud instance.
If you have data requirements to bear in mind, then other database options may be a better fit. For time series and geo-spatial data, PostgreSQL can be a better option – this database supports SQL, making it easy to support and manage data, but it also handles these data use cases with ease too. For companies looking at Internet of Things deployments where geo-spatial and location data will be common use cases, PostgreSQL can be a good choice.
Whatever your use case, it can be worth sourcing additional expertise around both cloud and database management so that you can check your assumptions around your choice. Getting this advice early can help avoid additional expenses or a further migration over time from one cloud – or cloudy database – to another.
Getting cloud and data right
Whichever approach is right for you, it will be based on understanding your specific requirements and goals. There is no single path or one size fits all approach that can take your databases into the cloud, and equally there is no single database that can fit all the myriad new use cases that are developing around data. Get used to “multi-cloud” and “multi-database”.
Instead, it’s worth the time to think critically about how you can efficiently use cloud and database technologies together both technically and commercially. The speed, cost and value opportunities around cloud are real, but they will take work and expertise to deliver. However, the advantages around cloud more than justify the additional planning and time investments.
Martin James, Vice President EMEA and APAC, Percona