Your data is your most valuable asset: the way it’s stored, manipulated, analysed and managed is key to your business staying competitive. For many years, SQL databases were the only show in town. But a new generation of databases is bringing businesses a whole host of additional features and options.
As always, understanding the needs of your business and those of your customers - along with the full scope of data involved - is vital to determine which database is right for you. And with new technologies constantly evolving, ensure the following watchwords are on your checklist so you can rest assured you are making the best possible IT decision:
Watchword 1: SQL/NoSQL
To SQL or NoSQL - that is the question. Both SQL and NoSQL (Not Only SQL) databases have their place in the enterprise. But in today’s multi-format, content rich world where 80 per cent of data is unstructured (such as images, audio clips, videos and PDFs), will a SQL database designed for structured data bring the flexibility and agility your business requires? For example, a publisher we work with who used to take two years to launch a journal using its SQL database, can now launch three journals a year with much less effort using a NoSQL database.
Choosing the right NoSQL database is obviously key: many NoSQL databases offer high levels of flexibility, allowing customers to develop rich data models with advanced functionality. But open source variants do not also have all the enterprise-grade features your business may need, such as government-grade security, elasticity, high availability, disaster recovery with advanced replication and - the next watchword on your checklist - ACID.
Watchword 2: ACID
Whatever the size of your enterprise, your vital data assets are likely to require ACID compliance: ACID (Atomicity, Consistency, Isolation, Durability) refers to a set of properties that guarantee that database transactions are processed reliably. The business risks for non-ACID systems are huge, unless IT takes on the highly complex, extremely resource intensive commitment to build ACID compliance into its applications.
For mission critical applications – such as those involving personal, financial, medical or sensitive data – it’s imperative to have transactional consistency. A leading investment bank we work with processes over 100,000 complex trades each day that typically result in about 32 million live deals in their system at any one time. Imagine the chaos that would ensue if the bank’s data was not kept 100% accurate, consistent and current? And smaller firms also can’t afford to take any chances with their data. Any application that supports simultaneous clients requires transactions with ACID properties.
Watchword 3: Scalability
There are several aspects to scalability. In data terms alone you’ll need to consider, for example, how much data you will be adding to the database each day, where it’s coming from and what needs to happen to the data. If your data continues to accumulate and grow, you’ll need to consider how your database is going to handle this growth. How easily and affordably can you add new resources, such as servers or storage space? If there are significant peaks and troughs in your data usage, how easy is it to scale up and down in size?
You want a solution that can scale simply, cost effectively and efficiently; in today’s IT landscape, low-cost commodity hardware in any environment—cloud, virtualisation, on-premise or a combination of all three – makes this possible.
Watchword 4: Semantics
In an ever increasingly data-rich world, you need to be able to integrate and model complex data to reveal new relationships, patterns and trends.
Semantics makes it easier to discover these inferred facts and relationships, creating concepts and categories and providing context.
You can make better-informed decisions, reduce risk, and convey more accurate information by combining documents, data and linked data in a single architecture.
For the Olympics, the BBC needed to publish content across 2000 micro-sites, including pages for over 10,000 athletes, 200 teams, 400 disciplines, and dozens of venues. Using a dynamic semantic publishing (DSP) infrastructure based on the MarkLogic database, heterogeneous data (for example, tweets, images, video, and other data from the International Olympic Committee) was continuously and dynamically streamed into the Olympic website from a wide range of organisations, channels, and journalists. The solution included a content store built using MarkLogic’s Enterprise NoSQL database platform. This unified, interactive architecture was able to handle the crush of over 25,000 transactions per second, creating custom experiences for users who made 106 million requests for live and on-demand video.
Watchword 5: Bitemporal
Bitemporal is crucial for any business from a governance, risk and compliance perspective. Whilst the vast majority of information in applications is temporal (i.e. tracking data ‘as it was recorded’), the next iteration of this feature is bitemporal, which handles data along two different timelines, making it possible to rewind the information and discover “what did you know and when did you know it”. Think of it as a kind of tech time travel.
For legal firms it’s vital to understand what laws are applicable to specific legal documents at a given point in time. For companies providing services based upon processing legal documents, bitemporal functionality gives the essential ability to understand the validity of the documents on a specific date and time. For financial institutions, bitemporal provides organisations with the ability to address regulation requirements, clear audits and enhance risk management and business analytics while reducing costs.
Watchword 6: Search
Search, like the ability to query data, is an essential element for database software; You can gather every piece of data you have, but if you can’t search it, you can’t use it.
The advantage of a database where search has been “built-in” is that it enables organisations to turn petabytes of data stored across multiple existing systems into useful information and results. In contrast, a “bolted-on” search solution is likely to delay results, reduce the performance, accuracy and your ability to see the nuance of details in the data because the data needs to be shredded. The same issue applies to third party search capabilities.
So, whether you’re a multi-million dollar global enterprise, or an enterprising entrepreneur, make sure you can confidently tick off Search, Semantics, Bi-temporal, ACID, Scaleability and NoSQL watchwords from your database checklist to maximise your most important asset: your data.
Adrian Carr, Group Vice President, Global Commercial Sales, MarkLogic
Image credit: MarkLogic