Skip to main content

The top four critical capabilities of data management

(Image credit: Image Credit: StartupStockPhotos / Pixabay)

In my experience with storage and databases, data management is a very broad and loosely used term. For instance, if you want to build data for performance reasons, being able to move the data between spinning media and higher performance media (without the application knowing) could be data management, and it's the job of the infrastructure to do so. When looking at systems, anything that is implicitly ‘scale out’ and is designed to run across hundreds or thousands of nodes must have the capability to guarantee multiple copies in the event of hardware, disk, or network failures. That can also be considered data management. Data management can also be defined as horsepower or capacity for a largely distributed cluster or re-balancing the data to take advantage of this newer real estate.

With these examples in mind under the umbrella of data management, here are a few best practices for critical capabilities to consider for any storage system you implement into your enterprise. 

Critical capability #1: Remote and multi-site replication

Replication is now more sophisticated than previous, more traditional storage systems. If you take remote replication, driven by either a separate entity that sits on top of the traditional array and does the replication, it's cumbersome to set up its point to point, and is (for the most part) uni-directional. If there is any configurational issue, it becomes a nightmare to track down and to debug, and it also increases a lot of clutter in the data centre.

Modern systems have a different look at how replication can be done. We are in a global economy, but the data governance and data sovereignty laws are all local. We have entered an era where applications mandate the geographical boundaries within which the data ought to reside. From a replication standpoint, an application and data must have the ability to be replicated across regions A, B, and C, and it's up to the infrastructure to live up to that. That kind of paradigm never existed before, and it’s even more a necessity to deliver such capabilities.

Regarding remote replication and snapshots, protection against things like ransomware by having a sophisticated snapshot in capability becomes key. Snapshotting requirements have become a lot more sophisticated; and with various physical sites, you need multi-site replication capability. When you go to a cloud provider, you should think of each region as a different site. But what’s also needed is multi-site replication capabilities naturally provided through the infrastructure.

Critical capability #2: Standardised infrastructure

If you look at the larger Internet companies like Amazon, Google, Facebook, and Azure, how do you think they have solved the data deluge problem? They do so by standardising on an infrastructure, which is key to operational efficiency. If you go into these companies, 90 per cent of the applications that developers internally build runs right off an infrastructure-as-a-service (IaaS) that's been built out for them.

The key to achieving operational efficiency is to have one single platform on which one can get complete protocol consolidation. You must be able to create "virtual desks" or "volumes," which applications should be able to consume either as a block device, an NFS SMB share, or as an S3 bucket.

You don't want to go from one set of silos to a totally different set of newer silos. The way you can drive operational efficiency is to provide one system that can lend itself to all these different workloads either by changing a hardware skew or policies in the system.

If you go into any medium to large enterprises, you never tell them to rip out everything, throw it away and use this new one. That's the wrong approach. You start with a specific use case, which any vendor knows can be a perfect fit for them. It could be a big bang for the buck for them and for the customer.

Critical capability #3: Cloud adjacency

There is a new phrase coined in the industry; cloud adjacency is the new ‘on premise.’ When you have co-location facilities, you are a millisecond or two away from any of the cloud providers. Now, if you deploy a fabric that spans on prem then any of these cloud providers with a system becomes a reality.  When it comes to latency from going from on prem to any of the cloud providers, you’re using cloud adjacency like an on-prem environment without being hit by the laws of physics. Liberate applications and make them more mobile to be able to move the app alone from one region to another or from one cloud provider to another without being forced to pull the data along with the app.

Critical capability #4: Software-defined integration

The moment you classify “software-defined,” you must have complete programmable capabilities for both the data and control plane. It’s that programability on the control plane that helps systems to integrate into vSphere, OpenStack, Kubernetes, Mesosphere, and others. The lines between on prem and any cloud provider are blurred at this point, and even the lines between hybrid and multi cloud should be blurred.

If you look at the fundamental storage primitives provided, none provide any fault tolerance across availability zones or across regions, or across clouds that doesn't even exist. That’s because there is no incentive for them to provide that capability. You should be able to build your own orchestration around an entire product, because if you think about the cloud, you can click a few buttons and everything is configured and provisioned for you. It's the same kind of semantics that software-defined systems must bring to the table.

There are a few features of storage systems that really make a difference. When you look at critical capabilities, the key is not just CapEx, but also to drive and increase operational efficiency. That is what's going to give modern enterprises a maximum bang for their buck.

Avinash Lakshman, CEO and Founder, Hedvig
Image Credit: StartupStockPhotos / Pixabay

Avinash Lakshman is the CEO and Founder of Hedvig. He founded Hedvig in 2012 after co-inventing Dynamo while at Amazon (2004-2007) and Cassandra at Facebook (2007-2011).