Thursday Testing: A/B is Plentiful, at Least Initially!

If you are in Silicon Valley, there is almost always a worthwhile event to attend and this past week was no exception. We had the opportunity to socialize, chat with, and listen to Avinash about testing (you know, the A/B and multivariate testing stuff :) ). The event I am referring to was “Thursday Testing” coordinated by Lily Chiu.  Thanks a bunch Lily, and thanks to the Omniture folks for sponsoring the event!

It’s always a pleasure to speak with Avinash. You’ll hear about analytics, online marketing, and also about what is going on in some of the world’s top brands.

I was able to take some notes that I wanted to share. OK, the notes are brief as I was typing on my iPhone and I think Avinash thought I was playing Sudoku too :) . Here you go:

Why aren’t more people testing and why is testing a rarity. Avinash mentioned a couple of things that made a lot of sense:

  • Vendors and consultants are sometimes at fault. In their effort to highlight the capabilities of their products and services, they inadvertently set a high barrier to entry. All they talk about is multivariate testing and the gazillion variations you can have.  While impressive, it might be overwhelming or intimidating to marketing managers and site owners. Simplifying what to test, starting with a couple of ideas, and setting realistic expectations might get the decision makers to grant you the go-ahead and get your testing effort off the ground.
  • The second point is that while it is easy to simplify things, it’s hard to come up with two good ideas to test (not just “let’s change the button color”). It does take creativity, knowledge of your audience, among other factors to come up with two very good ideas to test.

These were the two points that Avinash emphasized and I find very helpful. Then there was an interesting discussion about some experiments with 19 PhD’s (I’ll pass on this for now), and another discussion about embarrassing the highest level manager you can (this way they listen to you), but this one needs another post altogether, and Avinash talks about it on his blog, so you can read his comments on the topic and apply it at your own risk! :)

I would also add that looking for some easy wins, taking some risks, and finding a sponsor that will support your effort, will pay off big time in getting more out of your site, and also in setting/starting a testing program within your organization.

So to summarize:

  • If you/your organization is new to testing, start with A/B Testing
  • Be creative, take some risk, look for low hanging fruits and get some early results to get your testing program supported within your organization

For more information on testing, check out Google Website Optimizer, make sure you get a copy of Always Be Testing, and feel free to contact one of E-Nor’s consultants.

2 thoughts on “Thursday Testing: A/B is Plentiful, at Least Initially!”

  1. Hi there, thanks for the post!!

    It’s amazing how few people using web analytics software understand how important it is to do A/B (PHP or URL based) split testing.

    From my own experience I have come to know that usually the person who’s responsible for working with the web analytics program is not the same as the person who has designed the site. (especially if it’s an agency doing it for you). So this useful feature gets overlooked…:-(

    I think what makes split testing difficult is exactly this ”logistics” issue: it requires both knowledge of web analytics, html skills, and the insight in the sales purpose and strategy of the website that only the marketer has.

Leave a Reply

Your email address will not be published. Required fields are marked *


You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">