How smart is Big Data?

These days it seems everybody is jumping on the big data bandwagon. It’s no surprise that companies want to take advantage of big data and the all the insight it might deliver if analysed.

But, what if many companies are missing the big (no pun intended) picture by focusing so heavily and so exclusively on Big Data analytics? By only focusing on Big Data analysis, key operational data is surely being omitted, missed or simply not integrated? When it comes to data analytics and making the most of business intelligence, shouldn’t businesses really be focusing on All the Data?

What is Big Data?

There are many different definitions of Big Data, as Forbes highlights in its article 12 Big Data definitions, what’s yours? However, it is largely accepted that Big Data covers the broad range of new and massive data types, which have appeared in the last decade.

Arguably the largest source being information received from sensor data – data from recording devices such as satellites, road cameras. Additionally, the Internet, and in particular the growth of social media, has given companies a wealth of unstructured data that they hope will give them insight into customer behaviour. Companies are turning to the analysis of Big Data to help them optimise the customer experience by tailor making their offerings so the right people are marketed with the right product at the right time. Big Data analytics is seen as the key to business intelligence.

Business Critical Data

But, what about the data that existed before sensor data and the Internet? Data, that might be in an existing data warehouse.

In their rush to create a Big Data strategy, many companies are overlooking their existing data and not laying down the foundations that will enable the integration of all their data. Yet, this existing data is business critical data. The best analysis comes from the best data. Being able cross reference business critical data (structured or unstructured) with Big Data will deliver far more comprehensive results than Big Data alone.

What companies need to consider is how to integrate Big Data with business critical data and to consider how they will integrate All the Data as early on as possible. The integration of data from all sources and all sets, is what companies need to do, to reveal the fuller picture. All the Data is this combination and integration.

Too Big?

Furthermore, Big Data can be too big for it own good. Big Data is an immense resource if used correctly, but it can easily become a burden to the IT department. As IT organisations struggle to maintain hard-won gains in business intelligence with fewer people and smaller budgets, the emergence of the Big Data revolution threatens to create an entirely new class of demand for data and application.


Vendors are marketing “Big Data in a platform”; products which implant themselves into the process, introducing latency and increasing complexity and fragility. So, how can companies integrate all their data without having analytics siloed and unleveragable across multiple business processes?


You need a strategy that has a data integration approach and a powerful approach such as an automated enterprise data warehouse. A holistic approach helps manage data from any source and so derive value from All the Data. Automated data integration accelerates the results while minimising costs and risks. Choose a provider that can automate the entire life cycle of your data warehouse and there will be no silo, no juggling of multiple products, processes and languages; each part will talk to the next.

Choose automation software which is agnostic to Big Data technology and you will also reduce the risk of lock-in to technology that might later prove a block to data integration.


For Big Data to be viable there needs to be specific infrastructure requirements (e.g. Hadoop). But, a holistic strategy requires the combination of Big Data with the idiosyncrasies of OLTP systems, warehouse platforms, analytic databases, NoSQL or big data repositories, BI tools, and all of the other “boxes” that collectively comprise an information ecosystem.

Companies make smarter decisions with data analytics as long as the data is accurate, accessible and most importantly complete. By automating the data warehouse design and build, companies can ensure they have an agile data warehouse that can pool data from all sources; existing data, big data and the increase in data expected from sensors and smart devices (IoT).

The way to make sure that you’re being clever with Big Data is to analyse it in context with All the Data.

Miriam Fois, GM for UK and Ireland, WhereScape

Photo Credit: T.L. Furrer / Shutterstock