Taking a new approach to data management in the cloud

The move to digital with its accompanying shift to the cloud, use of IoT sensors and advanced analytics is a profound change for both business and IT. It is defining next generation products and services, forming the basis for new business models based on the technology, and redefining companies and markets. As a consequence of the move to digital, data continues to expand relentlessly in volume and variety while accelerating in velocity. In the midst of this, perhaps the largest technological refresh we have yet seen, it is no surprise that many organisations are struggling to ensure that data is a strategic asset and not just another challenge to be addressed.

However, it is already clear that an organisation that has its data under control will have a significant advantage over those that do not in the new competitive landscape. Pivotal to accomplishing this is a data management strategy and it is clear that the innate capabilities of backup/recovery, which include indexing and moving data, are foundational to managing and operating data effectively. Increasingly, this includes tasks such as cloud provisioning and data management. 

Gaining visibility of data 

Today it is impossible to imagine the Internet without the search engines that crawl and index webpages; the ability to search is taken for granted. Yet once inside a business the experience is very different. Here both users and operations teams have to navigate multiple systems to access the information they need to perform their work. This is because, unlike the Internet, enterprise data lacks a usable index.  

There is an exception to this. The structured data in Enterprise Resource Planning systems (ERP) and other systems based on relational technology do benefit from an index because it is built into the relational data structure. Unfortunately, the majority of the data in an enterprise consists of unstructured data (files, videos, machine data etc.) and is not indexed. So, while ERP data can be searched and queried, and the information stored within it can flow across the organisation, the bulk of the data in the enterprise sits in infrastructure or application silos. This presents an unsolved industry challenge that has become significantly more urgent of late as organisations modernise IT as part of the drive to digital.

This problem is compounded by the increasingly distributed nature of enterprises today. Data has been moving out of central data repositories into share drives and endpoint devices for some years now. With the addition of the cloud as a major part of IT, the data landscape is even more complex, distributed and fragmented.   

Various attempts have been made to address this problem. However, solutions that index unstructured data typically require content to be moved into specific repositories or at least – in the case of search appliances – the addition of a separate process. Neither of these approaches is scalable across larger businesses, which has led to a lack of adoption.

The backup/recovery index

A potential solution to this problem is utilising the index created by backup applications. Uniquely, this captures all the data in an organisation together with a great deal of contextual information which can include:

  • What application created the data
  • The users who have access to it
  • The size of the data
  • When it was created
  • And so much more

If the index is dynamic, it can be used to create a virtual repository of data, providing visibility wherever it resides. However, there is an important caveat here, for the virtual repository to provide effective coverage in the hybrid environment, the backup application needs to be cloud native and support all of the commonly used hypervisors.

One copy, many uses

A virtual repository not only ensures visibility of data across the entire organisation, but can support many use cases that require access to a copy of the data rather than the original. There are many tasks that require an up-to-date copy, including, and very importantly, running dev/test routines. These typically consume a large number of copies to provide the views needed for each test. Currently, many organisations use multiple applications to create these copies, which results in duplicated effort and wasted resources. A virtual repository enables organisations to benefit from a single, well managed copy for this and many other use cases.

Promoting data agility

Having an indexed copy of data has other advantages. Sufficiently well described data can be converted from one format to another, for example, from a hypervisor in a hyperconverged system to a hypervisor in the cloud. This promotes agility, which is vital to IT leaders as they pursue the tough task of transforming IT to enable the business to compete and win in the digital marketplace.

Backup software aids in data volume reduction by utilising techniques such as de-duplication and compression. This enables data to be transported efficiently to the cloud in exactly the same way it would on-premises. This, in conjunction with a pre-migration archive, can have a dramatic effect on cloud migration projects, with huge time savings when migrating applications such as Office 365 to the cloud.  

Cost and service level benefits

The cost and service level benefits of simplifying and consolidating the operating environment across the entire data estate can be very significant, as IDC’s 2016 report Quantifying the Business Value of Commvault Software: Worldwide Customer Survey Analysis showed. 

The high level findings of the report included:

  • 54 per cent reduction in cloud provisioning and cloud data management
  • 52 per cent reduction in DR setup, test, and maintenance
  • 55 per cent reduction in annual unplanned downtime
  • 42 per cent reduction in storage and data management hardware
  • 52 per cent reduction in data protection and data management software

The savings identified in this report reinforce the need for better data management and can be repurposed into spending that supports the transformation to digital business. With all the rapid changes in IT – cloud adoption and digital transformation to name a few – showing no sign of slowing down, the business case for consolidating backup and recovery as a route to getting data management under control is both timely and compelling.  

Nigel Williams, Senior Marketing Director, EMEA, Commvault
Image Credit: TZIDO SUN / Shutterstock