For many organisations, 'big data' is a term that carries varying degrees of relevance. But for the US federal government, which deals with an incomprehensible amount of information on a daily basis, these two words have come to represent a very real and growing problem. Fortunately, according to a recent Forbes article by Hilary Kramer, cloud-based technology may be the answer to the fed's data dilemma.
Kramer, one of Wall Street's top equity analysts and investment managers, discussed today's very real need to be able to "sift through, maintain and catalogue just about every bit of information that comes to and from an enterprise – from e-mail to PDFs to streaming video." This is also true of the federal government. According to a directive signed by President Obama two years ago (Presidential Directive M-12-18), federal agencies will be required to "manage both permanent and temporary email records in an accessible electronic format" by 2016.
This is where the cloud could make a very real difference.
What is the cloud, and how can it help me?
The cloud, in basic terms, is an Internet service that utilises remote servers to store and process information. The big advantage of the cloud, in terms of satisfying the government's big data needs, is its ability to organise and archive huge amounts of information. This alone would save the government massive amounts of money by dramatically reducing the need for expensive hardware to store data onsite.
According to Kramer, the Department of the Interior (DOI), one of the federal government's largest agencies, is currently looking to shift their data management into the cloud. The DOI recently entered into a contract with IQ Business Group (IQBG), a company that offers cloud solutions for managing big data. Utilising IQBG's software as a service (SaaS) platform, the eight-year, $53 million (£32.68 million) contract is expected to provide the DOI annual savings of $100 million (£61.66 million).
The amazing eERDMS system
As far as the cloud computing capabilities of the platform known as the email, Enterprise Records, and Document Management System (eERDMS) are concerned, Kramer says that the platform "captures and auto-classifies (stores in folders according to subject) 100% of a staggering 75 million emails per month."
Equally impressive is the fact that IQBG was able to get eERDMS up and running for the DOI in only 45 days - a feat that further underscores the cost-saving benefits of software-based solutions.
Michael Beck, IQBG's CEO, noted in the Forbes article that IQBG's contract with the DOI makes his company "the first to provide a comprehensive solution for e-mail archiving and journaling for an entire cabinet level agency." In addressing the proprietary nature of the eERDMS platform, Beck pointed out that IQBG's competitors don't have "a fully-integrated product in production like ours that includes auto-classification, record keeping and eDiscovery support."
That kind of technology, along with the DOI contract, has helped IQBG expand its reach, attracting more and more state, local and higher education clients within the public sector. According to Beck, since the company started offering the SaaS platform a few years ago, revenues have increased by over 100 per cent.
In addition, Beck notes that the total information captured and processed by IQBG has grown from 10-15 terabytes in 2012, to an estimated 75 terabytes. That's pretty impressive, given that just one terabyte (1,000GB) is said to be the equivalent of 143 million pages of stored Microsoft Word documents.
Ever-increasing cloud demand
Based on a number of factors - particularly the success of those enterprises that have already adopted the technology - demand for cloud-based governance solutions is steadily on the rise, with no end in sight. An online community of Federal IT execs known as the MeriTalk Cloud Computing Exchange, which recently estimated that the shift from onsite to cloud-based systems could reduce government IT costs by $12 billion (£7.4 billion) annually, supports this trend.
In closing, the author noted that some IT managers are hesitating to commit to cloud database systems due to concerns that sensitive data could be compromised by hacking and poaching. In addressing the issue of security in the cloud, particularly for the DOI, CEO Michael Beck said that IQBG security was "like Fort Knox". The cloud is "private", he said, and remains positioned behind the DOI's own secure firewall.
As IT managers come to recognise that storing, managing and processing data on offsite servers is efficient, economical and safe, more and more enterprises will come to know the benefits of relocating big data to the cloud.
Gil Allouche is the vice president of marketing at Qubole. Gil began his marketing career as a product strategist at SAP while earning his MBA at Babson College and is a former software engineer.
Image: Flickr (Tattooed JJ; camknows)