While dazzling data visualisations and the work of smart data scientists mesmerise many in the business world, companies that want to squeeze all the value they can from their data might want to go backstage to consider the processes that are behind it all.
So many companies have built big data ecosystems for data management and analytics, often using different technologies supplied by multiple vendors. The problem for such organisations is that as they draw crucial data from these separate and disconnected systems, they find it very difficult to answer their own business questions in a timely manner and that in turn has an effect on their productivity.
The fact is that big data has already become so voluminous that IT departments struggle to keep pace with it. The massive volumes flowing into so many disjointed repositories make cohesive analysis and effective delivery of insights a tough challenge to pull off.
Obtaining the answers
The difficulty starts when companies have several data and analytics environments. They end up wasting too much time constructing IT processes and infrastructure instead of coming up with answers to substantial business questions.
Although the data lake is increasingly popular as a solution, companies will achieve little if the information stored there cannot be analysed in relation to other enterprise data. For example, an online retailer with a bug preventing shoppers completing certain types of purchases may want to analyse the error log file in Hadoop to find the customers who have experienced problems and inform them by email that the obstacle has been removed.
However, the customer contact information is likely to be in its own database and needs to be joined with the error log data stored that is in the web server on Hadoop. To achieve this with minimum loss of time, organisations will need data fabric technology, which makes big data manageable and boosts efficiency.
Weaving the data magic
This innovative technology helps weave together disparate data and analytics repositories across a variety of sources, producing a seamless fabric. It will disclose new insights into how customers interact with a company using different channels, providing a clear view of how to nudge up sales or intervene to forestall defections to rivals.
Data fabric software delivers value by orchestrating query execution with incredible efficiency and security, across multiple processing engines and repositories, including Hadoop and data warehouses.
By deploying a fabric that stretches across multiple systems, an organisation gives itself the freedom and confidence to choose its repositories, analytic engines, data treatment strategies and workload characteristics according to its specific business needs.
Yielding real results
In retail, for example, this means opening up new customer insights and incremental revenue increases.
Web events from an online store in Hadoop can be streamed to craft a dynamic view of each customer’s interactions in the enterprise data warehouse, including individual profitability, purchase and campaign history. Then analytics can be built that alert the company when high-value customers have difficulty concluding an online purchase, allowing the right assistance to be offered at the right time.
For a manufacturing company, on the other hand, data fabric technology acting on sensor data enables proactive maintenance of production machinery. The result is lower operational costs, increased customer satisfaction and even higher safety standards.
In car manufacturing, vehicle sensor information can be incorporated into a discovery engine connected to an enterprise data warehouse holding sales, dealer, plant and parts data. The orchestration of this layer enables the car-maker to build an analytics model leveraging smart time-series analysis to reveal the likely cause of part failures. It can then connect this insight to the sensor events that gave warning of the parts failures and trace that back to the root cause in the supply chain. This in turn can be acted upon by offering the right corrective procedure to the right vehicle owner at the right time.
The right weave
As big data technology makes inroads in many sectors, organisations right across the business spectrum can see substantial benefits of using data fabric technology in a cohesive and uninterrupted analytic ecosystem.
However, a warning note needs to be sounded because the simple acquisition of information will not on its own, bring tangible results. In order to use analytics to find the answers to their many complex business questions, companies must find a way to use all the data in their corporate ecosystem with minimal friction. All the capacity and data architecture in the world will get you little more than chaos without an orchestration layer to weave data together across multiple systems and processes in usable ways.
This is where the deployment of data fabric technology gives organisations seamless access to data and analytics across all their systems, including data warehouses and Hadoop data lakes, fusing the data without the need for IT professionals to use special tools or learn a completely new language.
From manufacturing to retail, indeed in any sector, companies can be assured that by stitching data fabric technology into their IT estate they really can take full advantage of all their data.
Nicolas Adamek, Big Data CoE, Teradata Labs
Image source: Shutterstock/solarseven