Data management is top of mind for many executives as they not only try to organize their data, but analyze it in a way that’s beneficial to the business. A hybrid approach to data is surfacing and helping enterprises solve a number of problems as it relates to data processing and analytics. Mike Hoskins, Chief Technology Officer at Actian, explains hybrid data management and best practices for enterprises in the following Q&A.
1. What’s your outlook on data management in today’s current environment?
Enterprises are continuing to struggle with increasing amounts of data coming from multiple sources. This data has never been more valuable to a business as it now informs the who, what, where, when and how of all decision making. Although, if businesses cannot get a proper handle on data management, integration and analytics, the important questions will be left unanswered. The first step to succeeding in today’s environment is implementing tools and techniques that effectively connect, manage and analyze this new hybrid data landscape.
A focus area is analytic processing which is growing leaps and bounds. Enterprises are starting to realize the importance of analytic processing, and want to pair it with the benefits of transactional processing. The ability to combine these related processing techniques will help enterprises improve integration and minimize the amount of databases handled, while at the same time reducing data errors. Forward thinking enterprises who take this hybrid data management approach of combining multiple processing engines will achieve improved efficiency.
2. What is your definition of “hybrid data”?
One size doesn’t cut it anymore. Hybrid data has multiple dimensions, including diverse data type and format, operational and analytic usage, self-service vs. data center, A2A and B2B connectivity patterns, and both on-premise and cloud deployment. This is the new hybrid data landscape. Our view is simple – in a hybrid world, data needs to be managed, integrated and analyzed across an enterprise’s entire data ecosystem by anyone at any point in time. Enterprises will move to exploit this new hybrid landscape with a new generation of hybrid products as the one-size-fits-all approach to data will no longer be sufficient.
Hybrid data offers companies new ways to achieve actionable, real-time insights they can use to inform business decisions without having to muddle through various disparate processing models. The concept of seamlessly connecting and managing operational and analytic data to drive performance, insights and outcomes is at the heart of hybrid data. It is only when an organization can adopt this progressive approach that it can address the inherent limitations of traditional monolithic data repositories or siloed point solutions.
3. How can enterprises better manage data during the process of collecting, organizing, aggregating and analyzing?
In order to derive the most value out of hybrid data, it needs to be free to go where companies need it to go and integrate across the enterprise in a very agile manner. Most enterprises today run their transactional workloads in a traditional OLTP database, then do batch ETL to transfer the data into an enterprise data warehouse for reporting and analytics. This old way of thinking puts an anchor on data, causing it to be stuck in silos in an organization or move slowly, dragging its anchor along the way. That creates challenges for data currency, security, provenance, governance and veracity, and increasingly cannot meet the service levels required to maximize the value of that data to the business.
Replacing that legacy thinking with a more dynamic approach that imagines multiple stages of the full data continuum will help enterprises better derive actionable insights from their enterprise data:
Early Stage: The data continuum starts off with raw data being born in multiple systems and flowing through enterprises on a continual basis. The data has to be collected and prepared for downstream operational processing. Edge data management and rapid integration products are essential at this stage.
Middle Stage: Data that has been prepped and aggregated is now organized into databases and is what drives the day-to-day operations of a business. This data is used to power products or systems that make transactions possible.
Thoughtful Stage: Now that the data is collected and organized, the important stage of analysis caps off the data continuum. Using “right-sized” analytic engines and methods, and even hybrid databases, can help dissect the data to provide valuable insights that can then be fed back into a company’s operations.
Understanding these stages of the data continuum will help enterprises more strategically manage data. This plan of attack can take the headache out of collecting, managing and analyzing data.
4. Why is it important for enterprises to manage data in a flexible, real-time environment?
Real-time analytics is vital for every organization, regardless of the industry. For instance, banks rely on real-time analytics for fraud detection and retail companies depend on real-time analytics to generate personalized, dynamically priced offers to the end customer. Managing data in a real-time environment is only going to increase in importance as customer expectations grow. If enterprises cannot keep up with the competition, they’re at risk of falling behind.
While speed is an important element, crunching data fast will only get you so far. Speed has to be paired with accuracy in order for enterprises to make the best business decisions. Accuracy comes from the ability to apply big data horsepower against data history for deeper information. How companies make decisions is moving away from a mostly human-driven process into a more automated, machine driven process. Analytic Engines that can crunch, collect and provide insights will help executives make more informed business decisions, leading to ever more timely and accurate insights.
5. In your opinion, what are enterprises struggling with the most in regards to data management?
Integration remains an unsolved problem for many IT professionals. Today’s variety of IT systems and constantly changing end-points guarantees a number of integration challenges. Luckily for businesses, there are hybrid integration platforms with dynamic and cloud-based solutions that will help tackle a variety of integration issues. Whether it’s varied end points, multiple patterns (A2A via APIs or B2B via data exchange), diverse skills (IT expert to LoB practitioner) or different delivery models (cloud or on-premise).
Another challenge enterprises struggle with is the proliferation of data. As mentioned earlier, data is being born every second in different styles and from various sources, making it difficult to manage. The explosion of data over recent years is forcing enterprises to move away from the one-size-fits-all strategy. Companies need more agile and dynamic ways to cope with this new data landscape, and there has been an uptick in demand for deploying different models and different kinds of databases (i.e. NoSQL, Key-Value store) to provide “best fit” solutions. Some vendors are even combining multiple database engines in a single hybrid offering for even greater efficiency.
The democratization of data is another concept enterprises need to wrap their heads around. Data needs to be in the hands of decision makers, whether that’s C-level executives or line of business directors and managers. Different data is important to different people, and it’s vital that everyone has access to the data that matters most to them. Data often needs to live close to the people who care about it in order to be successful. An example is the need for enterprises to understand trends like the consumerization of data which offers quick, easy and friendly access, and try to provide that same experience to its users.
6. What are a few best practices enterprises can implement to get a better handle on their data?
- Reduce the amount of data being handled: By intelligently reducing your data footprint, you can better manage data that’s most important to your business.
- Process data closer to where it’s born: Edge processing is a vital part of data management. Vast amounts of data are born at the edges of an enterprise (PoS devices, mobile devices, etc.). That’s why it’s critical to implement your data management, integration and analytics closer to the source, which requires a smaller footprint and best-fit data engines.
- Use the right data management technology: Depending on your company’s particular needs, you will have to decide which type of databases make the most sense for your business. It could be SQL (easier management), embedded (low administration), NoSQL (less structured data with complex hierarchy), or a hybrid combination of these and others.
- View data integration as a first class citizen: Many companies end up with siloed data that limits full value. Integrating data across the board will help companies paint a holistic picture.
- Understand that data is always moving: It’s important to implement modern data management integration and analytic infrastructure that understands data is constantly flowing. Having this understanding is the first step to appreciating and embracing today’s modern data landscape.
- Reimagine the whole Data Processing Pipeline: The old approach to data used to be about large, relatively static transactional systems as the center of everything. Enterprises are discovering that front-end data capture as well as back-end analytics are equally important and are now investing to build out the necessary data pipelines to support the modern digital enterprise.
7. How do you see hybrid data management evolving over the next five years?
We’re noticing the rise of low-administration “edge” databases and processing engines as the IoT gains more prominence. Many IoT stacks focus more on the pure end-points (sensor and cloud), and miss a key element of scalable architectures - a middle tier that can deliver right-sized and robust processing services throughout the IoT data pipeline. These richer architectures provide an elastic middle tier that offer a series of gateways emanating from the edge. This middle tier is intended for earlier capture, processing and local analysis of the sensor data before relevant and filtered information is sent to the cloud. An embedded IoT Edge/Gateway Database will offer security, crucial local filtering and streamlined data operations for these use cases.
Mike Hoskins, Chief Technology Officer, Actian
Image Credit: Alexskopje / Shutterstock