A/B Testing is like a box of chocolates

forrest-gump-originalA great, albeit fictitious, character once said.. Life is like a box of chocolates, you never know what you’re going to get. He could easily of been referring to web site testing and optimization results.

Best practice optimization campaigns can almost guarantee conversion increases. But every business is different, every web site is different and every product or service. The fundamental differences attracts a different client base then that of a competitor. Because of this digital marketing campaigns that prove effective for one customer base, business or vertical may not be effective for another.

Practices such as human computer interaction, user interface and user experience design also bring industry recognized best practice and established processes to developing effective customer experiences. Typically these practices are not directly aligned to conversion rate or on-line sales performance. Typical key performance indicators include generalized, usability, visual design or ease of use.

“Test and monitor Campaign vs Campaign performance for future prioritization efforts and budget decision making.”

By testing digital marketing campaigns and web site updates marketers can measure the performance of that change as it relates to a myriad of sales and engagement metrics such as conversion rate, time on site, page consumption, average order value and so on. Without testing the change against the default content, or what was there previously, a marketer is making changes to the web site without a yard stick to measure against.

Not all users are created equal

Just as every business is different every user is an individual and has different needs, wants and expectations of a business engagement. There is no such thing as a once size fits all or hard and fast rule to what works and what doesn’t in digital marketing.

A marketing campaign that is effective for one user, group or segment may be detrimental to another. By testing the performance of each marketing campaign against the various users and groups allows marketers insight into the individuals responsiveness in relation to that campaign and allows marketers to effectively target and personalize campaigns.

“You customers are individuals, but have common traits and behaviors. Testing will identify traits of significance for segmentation and personalization.”

A campaign that generates a 5% lift for new web site users may generate a -5% lift for existing or returning users. Overall the test results are flat, they cancel each other out. But by targeting New Users with that particular Campaign and targeting returning users with an competing or opposed campaign can theoretically garnish a combined lift of 10%.

20_Edison_wouldve_loved_new_light_bulb_law (1)Don’t be afraid of flat or loosing tests.

Thomas Edison developed many devices that greatly influenced life around the world, including the phonograph, the motion picture camera, but it was his most influential invention was the modern electric light bulb. During its development Edison made 1,000 unsuccessful attempts at inventing the light bulb. When a reporter asked, ‘How did it feel to fail 1,000 times?’ He responded, “I didn’t fail, I found 999 ways how not to make a light bulb.”

Not only did he He only needed to find one way to make it work and become one of the most influential inventors of the modern century. The same applies to optimization testing.

“You have not failed by running a loosing testing campaigns, you’ve actually learnt something very important and valuable – you’ve learnt what not do.”

By testing campaigns and changes to a small percentage of traffic, that way you can learn from potential mistakes before you make them. Monitor the different segments and user behavioral changes that respond differently to the norm, these are opportunities to target and personalized for increased over all Campaign performance.

There’s no such thing as a bad test idea, but ideas should be prioritized in terms of segment size, level of effort to execute and expected return on investment.

1901_Wind_TunnelMeasure expenditure, return on investment and prioritize accordingly

Without testing the outcomes of a digital marketing campaign a marketer can’t be assured that the campaign is increasing those important key performance indicators and metrics or actually detracting. Additionally by testing and quantifying the effectiveness of a marketing campaign marketers can make more informed decisions about budget allocation and prioritization of campaign efforts. Did that shiny new $2 million dollar marketing campaign bring an in an equally valued increase in sales? Was the return on investment worth the expenditure? Or was it out gunned by that simple $20,000 campaign that went viral?

“Test campaigns to a small percentage of traffic, that way you can learn from potential mistakes – before you make them.”

The Wright brothers are heralded as the fathers of flight, the success of the Wright Flyer I, their first plane to sustain flight can actually be attributed to anther invention of theirs, a wind tunnel which allows the brothers to test their various craft and wing designs. Optimization software with built-in testing capabilities are very much akin to that wind tunnel, allowing marketers to utilize test data to measure the effectiveness of their marketing campaigns.

The most mature digital marketing businesses don’t make any changes to their web site, or launch any digital marketing campaigns without testing them against their default experience in order to quantify the effectiveness. By truly understanding the impact of these campaigns these businesses can prioritize future campaigns for maximum impact, return on investment and increase in conversion rates and sales.

 “I didn’t fail, I found 999 ways how not to make a light bulb.”

Optimization practices such as A/B/n and Multivariate testing allow marketers to use hard data to educate and drive digital marketing campaign success and accurately measure return on investment from all digital marketing campaign activity. The integration of a effective testing regime into any online business is essential to effective marketing.

Optimization Testing is just like that box of chocolates, you never know what you going to get and results can be surprising. As Edison and the Wright brothers discovered great results can be achieved through scientific, data driven testing.


Leave a Reply

Your email address will not be published. Required fields are marked *