Your database is a major agility risk factor. Here's how to fix it

We're willing to bet your database doesn't follow your already-implemented software best practices, but DBMaestro's Yaniv Yehuda is here to help you out.

It's a fact of business: when your competitor delivers relevant products, faster and with better quality, you're eventually going to lose market share. Of course, investing in sales and marketing campaigns to compensate for your product's shortcomings can work, but it's expensive and unreliable. And of course, you might find out that customers are moving to the superior product anyway.

This is exactly why companies need to be 'Agile'. They need to have better control of information, make quicker decisions, accelerate software delivery, and optimise quality control.

'Agile development' was born to deal with these challenges: the need to move quicker and deal with ever-changing requirements (since our target market and competition are never standing still), with best quality that can be assured. And this is usually accomplished with limited resources. Agile is what is expected from technology companies and IT divisions.

The need to take the Agile concepts to production (production environment, customer sites, etc.), and to link development with operations, has given rise to 'DevOps'.

The main goal of operations is to maintain stable and healthy applications, while the main goal of development is to continually innovate and provide applications that meet business and customer needs. Both are crucial to DevOps. While there is no doubt that change is the greatest enemy of stability, DevOps strives to understand and reconcile this conflict.

SDLC Best Practices

Software Development Life Cycle (SDLC) best practices did not drastically change when Agile and DevOps emerged, as these are steps of a software development evolution, rather than a revolution. However, these best practices are now much more important to implement and follow if you are traveling the fast lanes of the agile world. Furthermore, automation takes center stage in order to increase overall efficiency and reduce the risks inherent in frequent changes and releases.

Implementing Agile and DevOps approaches is part cultural change and part technological change. The cultural changes require more effective communication and collaboration between Dev teams and between Dev and Ops, while technological change is mainly about safely automating repeating tasks.

Here's my personal short list of things to keep an eye on:

Requirement management – A continuous process throughout a project of documenting, prioritising and tracking requirements, change requests, and trouble tickets, then controlling change and communicating to relevant stakeholders. All participants must clearly understand why changes are being made.

Development – The ongoing management of changes, using version control and change management, will ensure that it is clear who did what, when, and why. Task based development is a great way to correlate development work with requirement management.

Build – Link development changes with the requirements that initiated them, and build the next relevant release. Only agreed-upon changes should be implemented: nothing more, nothing less.

Testing – Quick development cycles challenge our ability to properly test our projects, as there is simply not enough time in each cycle. Regression testing and unit testing can effectively help focus on relevant changes, while making sure nothing else gets broken in the meantime.

Release – Release automation is a great way to make sure we create and test our change packages, and manually process them as little as possible. The more we automate, the more we can expect the same result from the same steps we take.

Operation – Roll back scenarios should be ready for each release. This is our last safety net.

What makes the database different?

Automation-based repeatability will drive safer and less error-prone deployments. The success of frequent changes cannot rely on a person's ability to remember everything that's already been done, or their ability to identify the full scope of what the current change might influence. The key is eliminating as much manual work as possible.

But, unlike other software components and code or compiled code, a database is not a collection of files. It is not something you can just copy and paste from development to testing and to production because it is a container of our most valued asset – the business data, which must be preserved. In most cases, database development is also done in a very different way than application code (.Net or Java development), as developers and DBAs are accessing and changing a shared resource, a central database, rather than a local copy on their workstation as they would do with application code.

The difference between database development and application code development is so great, that in time, silos were created.

While application developers are moving forward, adopting great new tools, practicing Agile and automation, database development is done using a less controlled process. More an art than engineering, database development is often disconnected from the entire SDLC processes. Tools, process, and best practices are not shared.

But, be careful. The database, being such a valued asset, can very easily become that wobbly wheel that destabilises the whole car.

How should database change be addressed?

We need to stop looking at the database as something different, just because it is constructed differently. Make sure all teams, code developers, DBAs, and database developers are following the same processes. Proven software best practices should be followed and applied to the database, as they are for code development.

At a bare minimum, the critical items from my previous "to do" list should include:

Agile development - Run short development cycles, and implement database changes as part of task based development, so code changes and database changes are done hand in hand, and can later be deployed together based on change requests.

Database change management – Implement database tools to make sure database changes are properly managed as part of SDLC, and not working outside of main development processes. Also, make sure each change is correlated to a change request, which will help manage the database as part of general development or change efforts.

Testing

Unit testing – Following change-focused testing and making sure nothing breaks as a result of changes will provide a solid safety net. There are tools for database unit testing, and some good ones are available for free.

Regression testing – Building proper automated regression tests requires big investments. My suggestion is to consider this as a long term goal, not a one-time project, and start accumulating automation tests for each new development or major change.

Automation – A major key for risk mitigation. The more we automate, the more we assure we can accurately reproduce the changes we introduce in development, test in testing environments, and repeat safely when going to production. Task based deployment and database impact analysis will leverage development knowledge and help achieve a healthy SDLC with minimum risk and maximum efficiency.

Summary

The database presents a real challenge when it comes to following best practices.

This is exactly how silos are created - a development department might follow different processes when dealing with code development vs. database development, or follow best practices for only some for its change efforts.

The best way to eliminate major risks is to stop looking at these differences from a technology point of view, and start making sure we find the right way – be it a process or a tool - to follow and enforce best practices and automation.

The database, being such a valued asset, must not become that wobbly wheel that will destabilise the car.

Yaniv Yehuda is the co-founder and CTO of DBmaestro