Skip to main content

Dirty data: It’s time to clean up your act

Data is an organisation’s greatest asset. Nurtured and protected, it generates insight, drives decision-making and provides a foundation for growth. For some businesses, data has become an additional revenue-driver: logistics firm DHL, for example, offers intelligence on air quality which it captures as its vans drive around different city locations.

Similarly, mobile operators use aggregated anonymised data of customer behaviour to help other organisations improve their business strategy. But for a business to extract trustworthy, meaningful insight from data, it is relying on that information to be clean, current and relevant. If the data an organisation uses to make business decisions is ‘dirty’ then any decisions they make are by definition flawed.

Businesses face losing revenue, damaging reputation, ruining customer relationships, demoralising sales staff or risking fines for non-compliance. So what can they do to keep their data clean? And what happens if they don’t?

Consumers and businesses generate vast amounts of data. The rise in connected devices means that every movement, connection and digital interaction generates information that can be captured, processed and analysed. But even if businesses are stringent about data entry, it still has a ‘best before’ date: it decays at an average of 2 per cent per month (opens in new tab) – unsurprising when you consider that every 30 minutes, 120 business addresses and 75 phone numbers change, 20 CEOs leave their jobs and 30 new businesses are formed (opens in new tab).

Businesses find it hard to stay on top of this, and feel their data quality is deteriorating. Research has found (opens in new tab) that 94 per cent of businesses surveyed feel they have poor quality data in their organisation, and felt that inaccuracies have increased to 22 per cent, from 17 per cent in the previous year. Clearly, the more we generate, the harder it is for businesses to maintain. While some organisations start working towards the creation of ‘data lakes’ for analysis, there is a danger of ending up with a swamp of inaccurate, inaccessible data of little value to the business.

It also results in a cost to the business, as data quality has a significant impact on business performance, from inaccurately-addressed mailings to significant fines for non-compliancy. Research estimates that inaccurate data is responsible for a 12 per cent loss in revenue.

For organisations with customers in Europe, the EU’s General Data Protection Regulations (GDPR) will have major implications in 2016, creating far-reaching change in the security and management of customer and employee data, potentially resulting in hefty fines for non-compliance. Now is the time for businesses to clean up their act and put steps in place to banish ‘dirty data’ once and for all. Here’s our top ten to help you achieve this:

  1. Adopt data quality as an ethos: Audit your data, standardise entries, cleanse the data removing duplications, correcting errors, updating commercial information, de-dupe the data, fill in the blanks and enrich it where you can adding depth with additional information
  2. Integrate, integrate, integrate: Establish your most trustworthy data source and use that to inform other systems, be that CRM, ERP or another transactional solution
  3. Seek ways to exploit data-driven opportunities: A business doesn’t always need a Chief Data Officer to stay on top of data, but it does need the right leadership and owners, across all functions, to make the most of the opportunities presented by clean data
  4. Consider how you collect and store data: Implement technology to capture data reliably, making data collection secure and accurate at every touchpoint, from taking orders across the phone to collecting data at point of sale; incorporate within this a Data Entry Standards document to provide clear instruction and drive consistency; and build in document integrity to safeguard physical, as well as digital, data
  5. Encourage internal communication: 31 per cent of businesses questioned in recent research cited lack of communications between departments as a reason for data inaccuracy; staff need the correct communication channels in place so they can safely and securely share information between departments, perhaps through cloud platforms, encryption tools or file sharing
  6. Educate and train staff: With 59 per cent of businesses believing human error to be responsible for the majority of inaccuracies, a thorough training programme is likely to have a positive impact. From updating staff on regulatory changes and the result of breaches to these regulations – and the role staff play in this - to training on specific applications and tools, education is key to data accuracy
  7. Invest in software which drives compliance: In the financial industry, Anti Money Laundering regulators require robust compliance, with accurate and detailed information on customers, structured in a consistent, transparent way; Entity Resolution is provided by software platforms which take data from multiple sources across a business and determines whether they refer to the same individual, asset or location, driving compliance and ensuring clean, structured data
  8. Get ready for growth: Data is growing exponentially, and businesses and consumers are communicating across multiple channels all of which generate data, so organisations need to make sure their data management platforms align with their plans for growth
  9. Make hardware and software user-friendly: High-maintenance technologies which involve time-intensive activity and manual updates may not be most efficient for your business or its data; conversely, user-friendly software with built-in automatic updates lessen the risk associated with manual updates
  10. Implement best-practice governance: This should implement the people, processes and policies to maintain data:
    1. Clearly identify and inform owners and responsibilities across all departments
    2. Include information on legislation and regulation, and identify the differences by region
    3. Include information on safeguarding physical and digital data, from printed bills to customer emails
    4. Include plans and tools to analyse results
    5. Constantly evolve, changing to suit the demands of customers, staff and stakeholders

Keeping data clean is no longer a responsibility confined to the IT department. Organisations need to distribute the challenges of data cleansing and place ownership into the hands of the business as a whole. Operational users have expertise and understanding of the data pertinent to their function, and businesses must tap into this pool of knowledge.

This collaboration will help organisations reduce risk and generate precise, accurate insight for their business, helping them on their way towards achieving the utopia of box-fresh data.

Tim Barber, Vice President of Digital Commerce, EMEA for Pitney Bowes (opens in new tab)

Image source: Shutterstock/alexskopje