Flashback: HDD vs. Flash and the future of storage

Most technologies enjoy a shining moment in the media spotlight. Currently, the cloud is enjoying such limelight, being perceived as a disruptor and has been dominating the content of almost every tech journal since the concept was invented not too long ago.

Through the early 2000s however, the tech sector was a great deal more productised and a great deal less homogenised, as a quick glance at the headlines of the day would confirm. Regular splashes back then for instance, were articles proclaiming the ‘Death of Tape Backup’. Tape-based backup technologies were apparently under serious and terminal threat from the ever increasing in size Hard Disc Drives (HDD) with an ever decreasing price tag.

As a result, it would surely only be a matter of time before tape disappeared altogether making way for Virtual Tape Libraries (VTL) backed by HDDs. Soon after, Disk to Disk backup (D2D) was a hot topic and almost every storage vendor had a D2D backup appliance offering of some sort, in fear of the inevitable death of tape backups.

Sure, all this fanfare did increase the utilisation of disk based backup solutions and started somewhat of a disk based backup trend in the market, steadily usurping in to tape’s territory. However, as things turned out, complete death of the tape backup has proved not to be the case yet.

While there was a clear use and often an important place in the enterprise data centre for the use of disk based backups, the poor old tapes still carry through as the final destination of a typical backup cycle for most organisations, where long term retention at a minimum price point is key. One can argue that this is because old habits die hard, but it would be more logical to deduce that’s because they both have their rightful place in today’s data centre - given their individual characteristics for different use cases.

Similar to that debate, HDD is now finding itself under threat from solid state and flash storage technologies at present. Flash storage is far from fledgling and with its latest incarnation of Solid State Device (SSD), has been around in varying flavours since the 1980s.

While flash has enjoyed huge popularity in smaller, consumer devices (tablets, smartphones, cameras etc.), SSD is gaining serious traction in larger high-end applications - up to and including enterprise-class storage arrays. Almost all legacy storage array vendors have an all flash (SSD) SAN offering available now, while a raft of Silicon Valley based startups have also sprung up specialising in just offering All Flash Array (AFA) storage solutions to meet every enterprise storage requirement.

The undoubted technical advantages of flash are obvious; it offers nanosecond seek times (aka. ultra-low latency) against the milliseconds of mechanical HDDs. It is also much quieter than HDD technology, more compact, uses around 50 per cent less power and - with no moving parts and a much greater resistance to high-G bumps and drops – is extremely reliable and robust too. But most importantly, flash storage is capable of performing an obscenely high number of Input Output (IO) operations. Often a single SSD can outperform the collective throughput of a substantially high number of HDDs together.

Despite the advantages of flash, the consensus agreement is that flash capacity costs more per raw unit of storage (GB) compared to HDD. This is true as flash uses more expensive NAND technology in comparison to cheaper mechanical disks used by HDD. As a result, many enterprise SAN vendors have had to come up with innovative techniques to maximise the use of flash capacity through the use of inline de-duplication and in-line compression of data, prior to being written to flash - effectively optimising the useable capacity. At the same time, it should also be noted that the cost of flash in general has been gradually coming down and capacity has been going up.

The evolution of the SSD storage technologies from expensive SLC (Single Level Cell) to MLC (Multi Level Cell) and the invention of new TLC (Triple Level Cell) technologies are all signs of flash storage continuing to become cheaper, while also increasing the capacity in time to come. This is likely to move in the same direction in the future and will therefore be the preferred choice for many workloads. It’s worth noting however, that at the same time, HDDs are also growing in capacity and further reducing in cost, maintaining their cost per GB advantage over flash somewhat. It is reasonable to assume that this this trend will continue in to the future and as such HDDs are very much likely to continue to maintain this price and capacity advantage over SSD for the reasonable future.

When considering HDD vs. flash, which will come out on top? The answer is unlikely to be straight forward and may depend on the context. In consumer electronics such as tablets and smartphones, traditional HDD based storage has almost ceased to exist as these have mostly been replaced by consumer grade flash storage of late.

These consumer devices do not require extreme capacity as current flash technologies provide more than sufficient capacity at a reasonable price point that make it an obvious choice. This, in addition to its other benefits, such as high throughput rates, directly translate into better product experiences for the consumers of those devices. So flash comfortably wins this round over HDD.

In the context of the enterprise storage requirements however, the decision becomes somewhat complicated. Enterprise storage requirements typically have two broad use cases:

  1. High performance (tier 1 or 2 workloads such as enterprise applications and private clouds)
  2. High capacity (backups, archiving, big data, public cloud storage and cloud native applications).

Flash storage has slowly been replacing HDDs for the first use case, where the high throughput capabilities of SSD technologies are naturally a better fit to deliver ever increasing demand for faster response times and larger throughputs. This is evident in the rise of All Flash Array vendors over the recent demise of legacy, HDD based storage in the data centre. This trend will continue and while future innovations in other storage technologies are likely to replace flash storage in some cases, for the foreseeable future, SSD based flash will likely saturate this segment - slowly pushing the HDDs out of consideration.

When it comes to the second use case of capacity however, HDDs indeed still remain the viable solution and therefore the de-facto choice for many. Similar to tapes in the backup debate, HDDs have far superior capacities at a much lower price point in comparison to flash and given the requirement of capacity over throughput; HDDs will continue to dominate this segment. This use case will only continue to grow, largely due to ever increasing appetite to generate more static data in today’s world.

Developments such as the Internet of Things (IoT) where most electronic devices in use in the future are likely going be generating large amounts of static data that would require being stored centrally over a long period of time, HDDs are best placed to meet such capacity requirements, given their lower price per GB, so HDD will be a clear winner here.

Therefore, for the short to medium term at least, SSD and HDD will both be winners and as a consequence, so too will hybrids of the two. Flash is undoubtedly set to capture more hearts, more minds, and more enterprise storage budget.

But, just as with tape, any reports of HDD demise will be an exaggeration for some time to come, given its invincible capacity characteristics that are clear necessities for many.

Chanaka Ekanayake, Lead Solutions Architect, Insight UK

Image source: Shutterstock/Ralwel