Skip to main content

How M&E organisations can optimise storage archives during self-isolation and beyond

(Image credit: Image Credit: Nampix / Shutterstock)

The Covid-19 crisis has caused a major shift in the media and entertainment (M&E) industry. The sudden spike in the number of remote workers due to isolation directives from the government has directly resulted in an increase in requirements for specialised TV, DVR recordings, video-on-demand and streaming services. As US-based global measurement and data analytics company, Nielsen has predicted, “staying put in our homes can lead to almost a 60 per cent increase in the amount of content we watch.”

Shifting priorities

With the reduction in live programming however, broadcasters have had to change priorities, including looking to other options to source content. For example, maximising legacy content from within their data archives; increasing capabilities to enable remote access; and changing the focus of content subject matter such as airing more educational, fitness and cooking programmes. These adjustments have enabled many M&E organisations to keep up with the current viewing demands. It is imperative that broadcasters continue to adapt to these extreme fluctuations in viewership requirements in order to remain competitive. The key to achieving this lies in the development of a sound data storage management strategy, making key infrastructure updates that empower M&E organisations to respond to fluctuations in demand with greater agility.

Increasing agility in M&E organisations

So, what are the barriers to greater flexibility within M&E organisations today? It is not a well-known fact that up to 80% of the world's data is stored on the wrong tier of storage, costing businesses millions of pounds every year. The majority of broadcasters have collected massive volumes of ‘inactive’ or archive content that could be optimised at this time. But these data assets are traditionally hidden from view, stored amongst active content on a costly Primary Tier of storage. Without suitable asset management systems, in-production and post-production environments will simply not be capable of rapidly identifying and accessing the right archived assets on-demand. 

Even in today’s modern times, many other vital capabilities, such as workspace protection, periodic project recurrence, project-based file tiering and archive management are still dealt with manually in broadcast organisations.

Two types of assets in media and entertainment

There are mainly two types of assets in the broadcast industry: managed assets and unmanaged assets.

Managed Assets

Production Asset Management (PAM), Media Asset Management (MAM), or at times Digital Asset Management (DAM) systems often manage groups of assets, such as news, sports, seasons, events, etc. These groups are considered managed assets and are placed on the proper tier of storage by policies made by the asset management applications to balance cost and perceived value.

Unmanaged Assets

Unmanaged assets are not usually managed by the asset management applications due to a lack of support by the applications for such files. Unmanaged assets include raw or original content, graphics, still images, etc. In some cases, organisations might operate separate, dedicated management applications, but the cost of such software often overrides the benefits delivered for most businesses.

Storage capacities for unmanaged assets can grow rapidly, taking up costly primary storage space, and inevitably hitting a threshold that is both financially and operationally difficult to sustain. For organisations without asset management applications, if automation is used at all, it is semi-automated with the use of scripts. In these environments, if economical and smart storage management solutions are not deployed, organisations typically continue to buy expensive primary storage as their capacities increase. This also applies to the IT departments that have no way of sharing existing resources. What is the answer? A cost-effective, intelligent, tiered storage solution that can help remedy the challenges of expanding repositories of unmanaged assets.

Importance of storage management software in media and entertainment

Ideally, decisions as to where assets reside and how easy it is to access them should be driven by the needs of the business. Implementing dynamic storage management software (SMS) through a streamlined two-tier approach is a cost-effective way to ‘smartly’ deliver insights, and rapid access to data assets. At the same time, it ensures that assets reside on the right level of storage to be protected for as long as needed, providing the highest level of financial and operational value.

The traditional ‘four-tiered’ storage pyramid has already been eclipsed by the two-tier approach in high performance computing (HPC), before emerging in the oil and gas market; two industries well-accustomed to the challenges of dealing with massive data volumes. Recent SMS innovation has now made the two-tier approach readily available to all other market segments, including the M&E industry, which is beginning to realise its suitability and advantages, especially in post-production.

The two-tier storage approach enables organisations to move the majority of their inactive data off the costly Primary Tier of storage (consisting of flash, NVMe and other solid-state technologies and high-performance disk), to the Perpetual Tier, which is a more economical tier of storage, consisting of one or more type of cloud, spinning disk, object storage and tape. The SMS scans the Primary Tier for inactive assets and then migrates them to the Perpetual Tier. This process frees up costly primary storage capacity for active assets, reducing backups windows, increasing performance of the primary tier and of course saving money. It also ensures that inactive assets are moved to the best storage tier for future access and safekeeping. The Perpetual Tier also can be used for secondary storage, distribution, backup, archive and disaster recovery.

The Primary Tier (for active data) is file-based, while the Perpetual Tier (where the legacy assets reside), is based on object storage. The use of object storage is pivotal to enabling flexibility in using each of the storage types listed above to the maximum benefit. This way the content is always available for future work, including editing, colour grading, music scoring, visual effects, etc. And even as more up-to-date asset versions are created, prior versions are also kept on the Perpetual Tier for safekeeping and easy access or rehydrated when needed.

If the SMS solution has a Project Archive feature, it will enable the migration of whole projects, such as finished films or videos to be easily stored in the Perpetual Tier, then moved back again to the Primary Tier as broadcast and viewer needs dictate. If content within the Perpetual Tier is required at short notice, such as historical footage about a deceased celebrity who is back in the news, for example, it can be easily restored to the Primary Tier of storage, either on-demand or scheduled for a future date. Proficient SMS will assign file metadata to each managed asset, and the software also will allow users to add additional metadata to enable granular searches and comprehensive asset identification. Ultimately the Project Archive feature ensures that all content will be available and searchable as a single project – allowing single files in a project or the entire project, such as a whole commercial, to be searched for and restored.

Remote access: leveraging the two-tier approach with the hybrid cloud model

Traditionally, broadcast organisations require infrastructures capable of supporting content which can be geographically distributed but also easily accessible. This capability would typically come with high price tag. Increasingly, broadcasters are turning to the cost-effective hybrid cloud model - storing some data in the cloud and some on premise.

The hybrid cloud model is having an impact on the way M&E companies operate, enabling access to new capabilities in the workflow design, post-production and distribution of digital content. And while SMS supports the movement of files to popular cloud platforms (often used for data distribution and disaster recovery), utilising public cloud may not be suitable for all data as it still raises security, compliance, control, recovery objectives and budget concerns. So, by utilising SMS, broadcasting organisations can retain control of their data and the choice of where data is appropriately stored. For example, organisations can create a more secure multi-tier private cloud, with a copy of frequently accessed assets stored on solid-state disk, for example, and infrequently or unmanaged assets stored on cloud, NAS, object storage and tape as part of the Perpetual Tier.

For an M&E organisation that currently houses the majority of their data on a high-performance, high-cost Primary Tier of storage, it makes good sense to explore the utilisation of storage management software to migrate inactive data to a Perpetual Tier, which will result in substantial cost savings, as content and inactive projects are seamlessly moved off of this expensive high-performance production storage, and preserved for easy access or future use indefinitely.

With these innovative approaches and cost-effective pivotal upgrades to their environment, broadcast organisations can achieve the agility and competitive advantage required to thrive in these trying times and well into the future.

Matt Starr, CTO, Spectra Logic