Why is a data-driven culture of business necessary for surviving the next major disruption, be it political, regulatory, new technology, or the next pandemic?
Large-scale change and transformation programs often fail when data management is an afterthought. Outdated IT strategies are preventing organizations from embracing industry disruption and taking advantage of the opportunities that technology presents, particularly in terms of understanding the value of the data they hold.
Although data management and analytics capabilities have now become a necessity in certain sectors due to the pandemic, for others the process has been much longer. It has taken time to look beyond shiny new platforms that often do not mix well with their legacy enterprise systems and to start building fundamental capabilities around data management, governance, quality, accessibility, and monetization.
- These are the best cloud storage solutions (opens in new tab) on the market right now
Where should organizations start when they're ready to better understand their data?
First analyze the organization's current situation. From that point, it becomes easier to understand how to reach their final destination. Not only does this help stakeholders clarify their current position but it's an opportunity to fully scope any weaknesses. The organization can then much more easily establish what it wants to know -- especially when a whole new world of data-driven business insights suddenly opens up as a result.
Data is data, no matter whether it exists in the cloud or more traditional data centres or even devices. Yet while cloud data platforms can provide horizontal as well as vertical scaling flexibility and speed, organizations should not confuse flexibility and scale with the quality of any insights they are able to get from their data. On its own, a cloud platform will not solve core data management issues.
We have always advised clients to start their journey by developing a robust data strategy and data operating model appropriate for the scale of what they do, then think about all the insights they wish to glean from their data.
It is not so much about the quantity of insights available but about accessing the few that will make most impact on the top and bottom lines and the experience of customers. Everything else is not much more than good-to-know management information.
Should you consider whether structuring data differently might be useful?
This is just one element of the data governance puzzle -- albeit an important one. Ensuring that data is correctly stored and structured can significantly reduce quality concerns, especially for organizations that derive insights by mixing their own internal data with external data streams.
If the internal data schema and model matches an external stream plumbed into data stores, it makes life much easier for stewards of that data and reduces the time it takes to achieve actionable insights.
Equally important is the task of cleaning the data. Pick up any recent survey of CDOs: data quality will likely figure very high in the list of barriers to analytics and data-driven decision-making. That said, cleaning can be a frustrating, budget-sucking task, and sometimes the outcomes are far from satisfactory.
We consistently advise clients to use external help to tackle this issue. organizations can also split multi-year projects into smaller outcomes and use-case-focused endeavors, enabling faster results and benefits.
Involve staff who are closest to the data in question -- 'data stewards', as it were -- because they have the best knowledge of what’s wrong with the data used to derive insights. This may sound obvious, but our sampling of data-quality projects across sectors and scales provides evidence to the contrary.
- Check out our take on the best cloud hosting services (opens in new tab) at the moment
An organization must ensure it collects the right data. How can it supplement older data -- now cleaned -- the right way?
As mentioned above, organizations must first decide what impact they want to drive, then look at the data they need and develop the strategy to keeping it clean, consistent, and current -- not the other way round.
Once the strategy, operating and governance and data-sharing principles and mechanisms are in place, it's time to think about tooling or specific platform needs. Most modern data platforms have built-in intelligence and capabilities to catalog data from varied workloads and engineer data pipelines with existing applications, either automatically or with relatively minimal intervention.
Data accessibility for parts of the organization where it can drive most benefit is often a key pinch point. Legacy enterprise systems that were never designed with data accessibility in mind persist in a majority of organizations, increasing the complexity of data silo resolution.
Thankfully, this is where a modern cloud data platform really helps in taking an iterative approach to centralizing useful data and driving good data-driven practices.
Organizations that start small and tie all data centralization efforts to key benefits and outcomes and iterate continuously over time get far better results than organizations that undertake it as a big-bang program of work across the enterprise.
AI and deep learning benefits can be achieved only by feeding in high-quality data and when organizations know what impact they want to drive from the data-driven insights they hope to create.
In summary, what are the keys to ensuring an organization maximizes meaning from its data, creating business value and empirical insights that power better decisions?
In all cases, start by thinking about the impact you want to drive across the organization. After that, work backwards -- allowing conclusions to follow through to ensure strategy, operating principles and commercial decisions are all objectively traceable to the outcome.
Foundational data capabilities should be developed and enhanced where possible, with rigorous data quality insisted upon. Only when the key building blocks of data capability are in place should an organization move towards an investment in pricey tools and technologies.
Remember that benefits from data can be realized incrementally and may not require huge investments in a dedicated data organization. When organizations understand that they can think big yet start small and look to scale fast, they can learn along the way.
- These are the best cloud storage solutions for photos and images (opens in new tab)
Shakti Mohapatra, Data and Analytics Lead, Coeus Consulting (opens in new tab)