Autonomy and analytics: Driving innovation

I see self-directed teams re-emerging as the way to drive innovation and deliver great products.

I recently had the privilege to speak to Kevin Goldsmith, an engineer at Spotify, about how they’ve used autonomous teams to drive high velocity in their development process.

Spotify published a 'whitepaper' on how they’ve used this autonomous model to scale agile development to 600+ engineers, and it is definitely worth a read.

As I dug into the conversation a bit, Kevin began to stress that it isn’t the organisational model that makes all of this work. The fact they group into squads that have all the skills needed to deliver from idea to product release isn’t really what makes it all work.

Over and over again, Kevin stressed the importance of culture. High-velocity development is deep in the DNA. Data driven product decisions are something they are passionate about.

Continuous integration and quality are important to everyone in the organisation. The organisation model didn’t give birth to the velocity they achieve. Their culture and drive for velocity gave birth to the organisational model.

Being a long-time analytics and decision support guy, I immediately became interested in the role of managers in this model. If teams decide what they do, even decide to form new teams or merge existing ones, what is the role of a manager in this model? Do managers even exist?

As Kevin explained the model, it became clear that Chapter Leads, Tribe Leaders and the Agile Coaches all played manager roles, but not in a traditional ‘command-and-control’ sense. The roles were much more focused on coordination, mentoring and a lot of communication.

Sharing information about what other teams are doing, what specialised skills might be needed on other projects, and otherwise keeping everyone in the organisation in sync.

The model is compelling, and I found it fascinating that they eliminate decision bottlenecks by pushing work-effort decisions down to the teams.

So what about chaos? Well, the leads in their various roles do the communication work necessary to avoid that chaos, keeping the teams aware of and aligned with one another.

And without the decision bottlenecks, they move fast.

So how do they avoid a lot of bad decisions? Data. They are data driven. Think something will be awesome in the product? Don’t spend six months building it. Build the smallest piece that is testable, put it in the product, and roll it out to 1 per cent of your customers.

Measure to see if it works, and if it potentially harms other aspects of the products. Then let the data decide.

If it seems to work, continue to test on 3, 5, 20 per cent of your customers. And continue to flesh out the capability as you go. Once you can see that it is going to stick, you fully commit the change into the product. Who needs an uber product manager saying go when your customers can tell you?

In the analytics game, we’ve spent the last two-and-half decades focused on enabling the top 10 per cent of decision makers in organisations to make better decisions. But, as we shift to knowledge-work-based industries with highly educated, highly skilled workers, the model for scaling an organisation changes.

No longer is it enough to make the executives and senior managers smarter. You need to provide everyone in the organisation with the data they need to make more autonomous decisions.

Many businesses lack agility in their business model for the exact reason Spotify has identified and eliminated decision bottlenecks. A handful of people who are supposed to know all things at all times and make decisions based on them.

My bet is that autonomous work models will out-perform the command-and-control models of the 1900s. And my bet is that the analytics game will need to focus on delivering analytics everywhere.

Far beyond the C-Suite and the data scientists, information will need to show up in the context of every employee to enable them to make better decisions faster, and apply their own expertise from the bottom up to drive business agility.

Charles Caldwell is the Director of Solutions Engineering and Principal Solutions Architect for Logi Analytics.