Skip to main content

Let’s stop data hoarding now

Research firm IDC just released an updated "Digital Universe" study (sponsored by EMC), which reveals some interesting information regarding the massive amounts of data we humans are accumulating. Basically, it tells us that every person in the world will soon have an average ownership of 5.3TB.

As reported in the press release: "IDC projects that the digital universe will reach 40ZB by 2020, an amount that exceeds previous forecasts by 14 per cent."

The release puts that number into perspective: "There are 700,500,000,000,000,000,000 grains of sand on all the beaches on earth (or seven quintillion five quadrillion). That means 40ZB is equal to 57 times the amount of all the grains of sand on all the beaches on earth."

It predicts that by 2020, there will be approximately 5,247GB of data for every man, woman, and child on earth.

Other highlights include the fact that only 15 per cent will be stored in the cloud, and the giant mass of data would be valuable if ever analysed.

4TB for sale

I'm reading this mostly thinking about my personal data storage needs. You can now pick up a Hitachi 4TB external hard drive, for example, for around £140. But do I need one? When you look at this from the perspective of, say, 1995, you have to wonder exactly what's going on here. Why are we hoarding so much data?

You can take all my writing, books, notes, and emails – much of them wisely stored as ASCII text – and easily fit them on a USB drive. The total is way under a gigabyte. The ASCII text of a complete novel is around a megabyte. I write about two novels' worth of columns and other articles per year, and I've done it for 30 years.

With ASCII, that would be 60 megabytes. With Microsoft Word, let's multiply the file size tenfold. That is still only 600MB.

So, text isn't the problem. The problem is the library of photos, videos, and PDFs, not to mention websites and huge music collections.

We are all hoarders

This need to keep all our photos, no matter how crappy, is part of the problem. I have almost a terabyte of photos – I'll shoot past the terabyte mark in late 2013 – but 90 per cent of them are crap. They are uninteresting snapshots that I wouldn't miss if I tossed them. Many are duplicates of the same subject. With a phone camera, the overall quality is so sketchy that many people now shoot the same scene multiple times – bang-bang-bang-bang – hoping to get one photo that is not blurry or screwed up. Then they keep them all!

And, yes, I know there are those of you who discard the eight crummy shots and keep the one winner, but most people do not. They just keep them all. After all, it is quite possible that some day in the future, some amazing anti-blurring software will arrive to fix the crap. But someone will have thrown out what would have been the best picture! So hoarding is in order, just in case.

There is no doubt in my mind that if everyone on the planet has an average of 5.3TB of data, then more than a few people are seriously hoarding junk. You need to stop!

Things were not so out of control during the era of slow growth of hard disk capacity. When the hard disk was a 20MB device, you'd have to do housekeeping a few times a year. This meant erasing duplicate files (who does that anymore?) and tossing programs you never used. Now we keep everything because storage is so cheap and plentiful.

Most of us do not even know what we have on these disks. It's horrible. We're beyond the point of no return. So go with it. Become a data hoarder and enjoy life as such. Get 20TB as fast as you can and just load up. Maybe it will encourage some smart guys to find a way to sort through this crap. I know I can't find anything.