Skip to main content

Winning customers online with successful experimentation

startup
(Image credit: Shutterstock.com / Pressmaster)

The results are clear – businesses can’t neglect experimentation. There is now a sea of tangible proof showing how businesses that experiment, often significantly improve their online metrics. This includes increasing page views and time spent on site, and ultimately improving the businesses bottom line. In today’s digital era, experimentation should be top of the agenda for any business operating online.

However, while many product teams are running experimentation programs, often they are far from as effective as they could be. Frequent issues include a lack of integration into wider business functions, limited numbers of variations being tested and a lack of leadership sponsors onboard. These all work to create businesses that are failing to harness the true potential of experimentation. Not only are they limiting their ROI, they are also quickly falling behind the competition. Fortunately, there is a clear path for businesses to follow that can help garner the best possible outcome from experimentation.

What does good experimentation look like?

Firstly, let’s talk about what good experimentation is, and how exactly it can help a business. Digital experimentation is the process of removing the guesswork from the customer journey. Adjustments to digital platforms, apps and processes shouldn’t be made on the hunches of a few key employees. They should be based on what customers actually want and need.

Experimentation allows this. Specifically, it enables multiple variations to be tested, in order to see the effect changes may have, on any aspect of the customer journey. This can range from the color of a website header, to the placement of a chatbot, to the process in which a dynamic asset loads. These different variations are then shown to real customers and tracked along with their effect on success metrics, like click through rates or time on site.

For example, Optimizely worked with TrustRadius to increase the effectiveness of their call to action (CTA) buttons. At the time their CTA buttons were getting lost as users scrolled down the page. The team at TrustRadius had some great ideas for how to improve the experience, but they wanted to see which would be most effective before making it permanent. They experimented, trying different locations and variations on the page to see which one produced the best result. From this they found the ideal location for their CTA button, which ended up doubling their click-through rates.

The comprehensive research from Professors at Harvard Business School and Duke’s Fuqua School of Business found that after just one year of testing, companies that experimented observed benefits in key metrics such as a 12 percent increase in page views and a four percent increase in time on site. This reflects Optimizely’s research which has shown that three-quarters of brands improved revenues by at least five percent as a result of experimentation.

Businesses that have a test-and-learn approach will outperform those that don’t. Decisions backed by data and experimentation will improve the customer journey far more effectively than those based on gut-feeling.

So what’s the problem?

It all seems pretty straightforward and a bit of a no brainer, however many businesses are still struggling to implement experimentation effectively. And there are a few common problems which crop up time and time again.

Firstly, experimentation works best when it is a foundational aspect of a business, not a siloed business function. If the majority of product decisions have already been made, and the product team can only test to make sure the already chosen change will work, the real value of customer insight is lost. Instead it should work the other way around. Experimentation lets customers tell businesses exactly what they want. With very little guesswork required.

Experimentation therefore should be open to all business functions. If marketing, or sales or any other function in the business, has an idea, they should be able to raise the change request and have the product team test this. These can even be small scale experiments that don’t affect the bulk of consumers, but are enough to produce some interesting data points. Future decisions can then be made off the back of this empirical data. This inclusive approach can help bring everyone on board to a test-and-learn mentality, meaning testing can be used more effectively to guide business decisions and add value.

This can, and should, extend all the way to the top. Senior executives and business leaders are the ones that have the power to drive forward business change. They also can lead a culture in which staff have more freedom and confidence to take risks, with big potential returns. However senior leadership will often want to follow the strategies that have worked well in the past, rather than experiment with what could work better in the future.

Business leadership can be the catalyst for a successful culture of testing, or be the obstacle holding it back. This is why it’s so important to have these stakeholders on board. Driving continuous experimentation and reporting successes or learnings, while offering senior leaders the opportunity to test their own ideas, can help bring them onboard. Working with a skilled partner can also help, as they can act as an experienced advocate for a holistic business model of testing. Once leadership is onboard, testing can really work to drive business growth.

Getting back to basics, many businesses that do test, miss out by not testing enough. This doesn’t necessarily mean the number of experiments, but instead the number of variations tested in each one. Businesses that A/B test are often only experimenting with one idea, using two variables. This was great ten years ago. Now, most effective businesses are those that run multiple variations in their experiments. Through this testing, they can explore new ideas, producing insightful data that would have otherwise be missed out on. Our research found that the benefits of testing clearly rose with the more variations added to each experiment. 25 percent of businesses that tested with only two variations in their experiments saw a significant uplift as a response. This increased to 44 percent when testing with five or more variations.

Another common excuse is a lack of resources, but there really isn’t much more effort in a four-variation experiment, versus a two variation one. The solution here is to test as often, with as many variations, as possible. Embrace the test-and-learn approach. The more data a business can have on how to improve their customer journey, the more effective it will become. Ultimately, this means higher conversion rates and increased revenue.

The science behind successful digital experience

Brands with the best digital customer journeys are those that make decisions with science, replacing the guesswork with evidence-based outcomes. They implement experimentation as a core principle of their business, getting executive support, and make sure they’re testing with as many variations as possible. By implementing experimentation not as a single solution, but a foundation of business culture, a business can make sure that they are maximizing each and every customer interaction, and in turn, winning in a world of online.

Jil Maassen, Lead Strategy Consultant, Optimizely

Through her daily work Jil facilitates transformation of mindsets, drives cultural shift of leadership & product teams, and encourages replacement of digital guesswork with evidence-based optimization. With an extensive background in the startup world as well as management consulting, Jil is originally from Germany and has lived and/or worked in 16 countries. She is currently based in Amsterdam and works as a Senior Strategy Consultant with Optimizely's global client base.