Traffick - The Business of Search Engines & Web Portals
Blog Categories (aka Tags) Archive of Traffick Articles Our Internet Marketing Consulting Services Contact the Traffickers Traffick RSS Feed

Wednesday, April 04, 2007

The Testing Revolution: Multivariate Landing Page Testing

Now that Google Website Optimizer is out of beta, more businesses will begin testing their landing pages to improve conversions.

As I was fairly familiar with the basic product features already, Google's Tom Leung and I had the chance to talk informally about some of the benefits of putting this product in many hands.

One issue I raised was how to weigh "advice sessions" and "clinics" and the like. The analogy might be a bit along the lines of American Idol... it would be very entertaining to see someone donning a Simon Cowell wig and blurting: "That's rubbish! That page will never convert! I mean just look at how small that search box is, and the abominable use of tables. And that paragraph about shipping. So trite. In short, I got nothing out of this and I'm wondering right now why I even bother to sit through this." So in short, Simon's usability advice could be hit-or-miss.

Compare that with the flipside: a distinctly un-Cowell-like Talent Optimizer that would input various pitches, intonations, arm lengths, dance gestures, wardrobe elements, and facial expressions into a virtual performer... and measure the correlations of each element to positive responses from the paying audience. ("Dr. Clark, it appears the optimal arm length for Celine Dion is a full foot shorter than we've been using! Egads! And look at that fingernail data! Midnight blue is kicking butt!")

Everything in its place. Just as we don't really quite want a Talent Optimizer (though record labels and boy band promoters probably have something close to that in the underground lab) judging American Idol, we need to move beyond Cowell-like subjectivity in our ecommerce efforts. A multivariate testing process is just that.

I managed to get a lengthy riff out of Leung on the reason Google doesn't recommend Taguchi optimization. That was something I noticed right away in the Website Optimizer literature, but Leung provided more color on it. The upshot is you really do need to test all potential combinations rather than a reduced combination set, because Taguchi makes poor assumptions around variable interactions. You can semi-Taguchify your process by hand. I won't bore you with the details but rest assured that whether you go with zero Taguchi, semi-Taguchi, or Taguchi on a Taco, this will likely be a step ahead of a simple A/B test, and many steps ahead of not testing at all.

Because testing is almost always better than not testing - and usually so much better that it even compensates for the risk of "messing with" a home page that ranks well organically - it's hard to see a significant downside.

Tom and I scratched our heads a bit trying to come up with an answer for the question: roughly speaking, can you make any serious errors running such tests? Setting aside technical snafus and things you might do to ruin your website by misinstalling code (your problem), basically the answer would be no. The biggest "error" would be to pick the wrong things to test - in other words, not improving as much as you could if you did it better.

The maximum number of variations the Optimizer product will allow for a single landing page test? Leung said 10,000. I recently completed a test that involved 16, and am running one with 24 now. For these particular tests, had I done them using 10,000 combinations, they'd reach statistical significance around the year 2258. Coincidentally, that's about the time they'll have finally perfected Celine Dion.

Labels: , , , , ,

Posted by Andrew Goodman




View Posts by Category

 

Speaking Engagement

See Andrew Goodman speak at eMetrics Chicago 2014

Need Solid Advice?        

Google AdWords book


Andrew's book, Winning Results With Google AdWords, (McGraw-Hill, 2nd ed.), is still helping tens of thousands of advertisers cut through the noise and set a solid course for campaign ROI.

And for a glowing review of the pioneering 1st ed. of the book, check out this review, by none other than Google's Matt Cutts.


Posts from 2002 to 2010


07/2002
08/2002
09/2002
10/2002
11/2002
12/2002
01/2003
02/2003
03/2003
04/2003
05/2003
06/2003
07/2003
08/2003
09/2003
10/2003
11/2003
12/2003
01/2004
02/2004
03/2004
04/2004
05/2004
06/2004
07/2004
08/2004
09/2004
10/2004
11/2004
12/2004
01/2005
02/2005
03/2005
04/2005
05/2005
06/2005
07/2005
08/2005
09/2005
10/2005
11/2005
12/2005
01/2006
02/2006
03/2006
04/2006
05/2006
06/2006
07/2006
08/2006
09/2006
10/2006
11/2006
12/2006
01/2007
02/2007
03/2007
04/2007
05/2007
06/2007
07/2007
08/2007
09/2007
10/2007
11/2007
12/2007
01/2008
02/2008
03/2008
04/2008
05/2008
06/2008
07/2008
08/2008
09/2008
10/2008
11/2008
12/2008
01/2009
02/2009
03/2009
04/2009
05/2009
06/2009
07/2009
08/2009
09/2009
10/2009
11/2009
12/2009
01/2010
02/2010
03/2010
04/2010

Recent Posts


Google Affirms the Vital Role of Marketing and Adv...

Twitter Ad Potential: Huge (Source: History, Users...

Twitter's Monetization Model: On the Mark, or Off-...

Don't Go to Google, TripAdvisor, or OurFaves for R...

Can Search Engines Sniff Out "Remarkable"?

Yusuf Mehdi's Too-Candid Comments About Abandoning...

Google AdWords: No More Last-Click-Attribution Blu...

Ad Network Confusion Mounts

Yelp, Extortion, Spam, and ... Oh, Just Stop It!

Go Deep With Our Paid Search Training at SES New Y...

Posts by Month


07/2002
08/2002
09/2002
10/2002
11/2002
12/2002
01/2003
02/2003
03/2003
04/2003
05/2003
06/2003
07/2003
08/2003
09/2003
10/2003
11/2003
12/2003
01/2004
02/2004
03/2004
04/2004
05/2004
06/2004
07/2004
08/2004
09/2004
10/2004
11/2004
12/2004
01/2005
02/2005
03/2005
04/2005
05/2005
06/2005
07/2005
08/2005
09/2005
10/2005
11/2005
12/2005
01/2006
02/2006
03/2006
04/2006
05/2006
06/2006
07/2006
08/2006
09/2006
10/2006
11/2006
12/2006
01/2007
02/2007
03/2007
04/2007
05/2007
06/2007
07/2007
08/2007
09/2007
10/2007
11/2007
12/2007
01/2008
02/2008
03/2008
04/2008
05/2008
06/2008
07/2008
08/2008
09/2008
10/2008
11/2008
12/2008
01/2009
02/2009
03/2009
04/2009
05/2009
06/2009
07/2009
08/2009
09/2009
10/2009
11/2009
12/2009
01/2010
02/2010
03/2010
04/2010

 


Traffick - The Business of Search Engines & Web Portals

 


Home | Categories | Archive | About Us | Internet Marketing Consulting | Contact Us
© 1999 - 2013 Traffick.com. All Rights Reserved