Traffick: May 22, 2005: Search Engine Enlightenment

Playing to "the algo," as we've argued here for some time, may soon be a thing of the past.

Yahoo's "Mindset" beta offers searchers the ability to customize their search as to commercial intent. Searching for commercial results? Slide the slider closer to the commercial side. Doing a more research-oriented search? Slider over to the right.

Google was the first to introduce sliders of this type. We felt that the significance of this wasn't so much the personalization experiment per se (users would tick off subject interests to "orient" themselves to the engine), but the whole experience of watching results bubble up and down depending on where one set the slider. For the average user, but maybe more importantly, as an educational tool for children learning to search, this could take the whole search experience to the next level.

The other important implication of a world where searchers see vastly different results depending on their own personalized algorithmic recipes, of course, is that search engine optimizers and especially hard-core search engine spammers can't reverse-engineer "the algo". It becomes harder to make generalized claims about "what search engines like." That would mean search marketing would really begin to be about deep marketing strategy, not just B.S. game-playing. A few really good cloakers might clean up, though. More likely, SEO's would claim that the best strategy would be to create multiple site types and multiple page types in order to do their job properly.

Amateur site optimizers and classic "white hat SEO's" might even be harder hit than the hard core optimizers under this scenario. Insofar as they tend to believe in a certain model for a well-optimized site -- one that might sit just to the right of the center of the commercial vs. research slider on the user's interface -- they'd be losing out on more of the traffic that came from eager buyers who slid the control all the way over to the "commercial" side. Few would use that as their default setting, but in those cases where folks really were looking for a commercial page, a "white hat" edict to "create plenty of useful content" might actually backfire, giving a site or page a profile that didn't match the user's commercial intent in making the query, thus pushing such listings well down the list of SERP's.

Currently, such "white hat" pages do very well in Google results. Arguably, that has been the whole purpose of Google's ongoing search quality initiatives leading up to, and especially after, the "Florida" index update of November 2003.

MSN Search was next to introduce "sliders," under the "results ranking" link in search preferences, but like Google's, it was experimental and didn't necessarily create any major useful dichotomies for searchers to sink their teeth into. Popularity and freshness are among the variables. You can play with it to see different results, which I sometimes do when I'm having trouble finding stuff.

Yahoo's Mindset initiative is a stunning and important contribution to a potentially exciting user experience for searchers. Top marks to Yahoo. We'll be watching for further development by all the top engines.

In playing around with the slider I notice that one friend's site, a well-known e-commerce site in its small niche, hits its high-water mark (a #1 ranking on his core term) when the slider is just one tick over towards the "research" side. Evidently, at least for now, Yahoo's technology interprets his site or its home page to be a slightly-white-hat type of thing. When I slide it farther over towards "research," he disappears off the first page, and others (including some commercial pages -- the machine learning should eventually take care of that) rise. When I slide the setting all the way towards "commercial," my friend's (highly commercial) site also drops off the list. I guess his site just isn't quite crassly commercial enough. Drat that blog!

The precariousness of first-page rankings with only minor user-driven adjustments on one parameter just underscores the silliness of client expectations (often fueled in the past by consultants overzealously selling SEO) of top-five results on core terms. Major companies feel they "should be" #1 or no worse than #3 on core terms. Niche leaders get jealous when a competitor at about the same level gets ranked one or two spots higher. Clients of SEO firms get disgruntled if the SEO firm "doesn't get them from #7 to #4" or worse, "allowed us to drop from #2 to #38." It's my sincere hope that down the road, there really will be no #2 or #38. There will be "#2 for Jim, #34 for Sheila, #95 for Naveen, and #10 for Ed." If Jim is your best potential customer anyway, then it's no great loss that your ranking for Sheila might bounce around in the 20's and 30's.

Someday, finally, everyone will look only at real campaign metrics within their web analytics reports: search referrals, paid search referrals, conversions to various actions, and so on. Month to month benchmarks of key metrics (including something as raw and un-search-ish as a simple chart of new customers acquired, on a simple bar graph for each month... doesn't that put it all in perspective) are far more important than "search rankings," of course. Just as in real life, "meals fed to family" is a better metric to look at than "red lights beaten." The two probably don't show a positive correlation, and depending on how aggressively you're trying to beat red lights, they could show a markedly inverse relationship.

Financial advisers have been facing pressures to "create results" for many years. But as we know, markets, like organic search rankings, will fluctuate. Depending on the investor, the best adviser might be the one who educates you, encourages diversification, insists that a certain percentage of your porftolio be in unfashionable value plays, and above all, protects your capital.

It will take years more of education before it's more widely understood that search marketing is about deep marketing strategy, and that search works for users in ways that defy control by marketers. You can steer intelligently to move things in the right direction, and spend a bit more to create better ROI on a search campaign. You can gain amazingly targeted customers at a rapid rate if you do it right. But search engines, as always, are working on ways to serve users, not just advertisers. That means that a first-page organic result is still a low-probability thing. Increasingly, any claims that there is a set methodology for achieving such results consistently should be met with scepticism.

For users, experiments like Mindset are all good.

Thursday, May 26, 2005

I first started seriously expounding on that whole long tail thing a little over three years ago, when I urged AdWords advertisers to "avoid insider thinking" in their selection of keywords.

Search engine optimizers have done this for much longer, since they've always looked at ways of ranking well on secondary phrases in industries where the top phrases are too competitive. A great way to continue to get lots of diverse search referrals to your site is still pretty basic: have a lot of great content. The diversity of words on those pages will act as a net that should capture odd, unexpected searches -- the phrases that only get typed in once or twice by anyone, ever.

So should we all start getting worried that more people are discovering the tail (props to Seth for the reference)? Nope. If you live by the tail today, or benefit from it, you're safe. The tail, like every other good idea in business, is safe from those who get into it only because it's a fad. Marketers who understand the tail and its benefits will watch as the fadsters come and go, because fadsters always misapply, misunderstand, and generally make mincemeat of any good idea.

In short, for the next little while, the tail becomes the head, as folks obsess about it, raise VC funds on the strength of it, and generally go ga-ga for it.

I mean, really. Take any general trend or fad, like say ASP-based productivity tools. Some like to call this "on demand business." Wall Street champions like to call Salesforce.com (a company never shy about promoting its stock) the "poster child for on demand" or something to that effect. Then again, I remember when Priceline.com had invented a new way of counting, or something.

I'm sure that when Google's stock tops $300, the last few bullish analysts will start beating the drum for Google's impressive and still-untapped mastery of the tail.

And some new e-commerce or auction-based services clearinghouse startup that will get squashed when eBay or a better competitor stomps on it will use the tail metaphor to hoodwink a few investors.

The tail has always been there. Big companies like Apple are studying it and figuring out how to design products using idiosyncratic advocates out in the field to look at unusual forms of emerging and highly differentiated consumer demand. It's a frightening and enormously cool thing that so many new businesses can benefit from it, and that so many customers can make so many more specific demands on business. It can't be ignored.

But unless your grasp of the tail fits in with a savvy business sense and customers you've been cultivating for awhile, be careful you aren't really grabbing for the head. You could get bitten.

Tuesday, May 24, 2005

Guess Gary's never read my AdWords Handbook? Those of us who do this for a living have been talking about the wide-ranging potential of AdWords-style auctions for years. Companies like VerticalNet and other vertical auction pioneers were exciting to analysts in their first wave because they offered opportunities for inventory to be sold at whatever price point made sense. When that principle carried over into search advertising, companies like Google and Yahoo went from marginal to highly profitable. That trend may continue as the principle plays out in various ad formats.

As the head of a competing contextual ad service (Quigo) recently told me, Doubleclick failed at this because they were only good at selling the inventory that was easy to sell. They did a terrible job of monetizing remnant inventory for their publisher partners. Not so for Google. Google is doing a fantastic job of it. So much so that a huge number of minor websites are using the system to generate mostly low-CPC ad revenues, and not a few larger publishers are doing the same.

"Junk" ad inventory is easier to sell when you have a platform that facilitates this. And high-priced inventory isn't compromised either, due to the auction system. This principle is now emanating out into the content space, and more are finally taking note of it now that Google is allowing publishers and advertisers to communicate more directly (buying and selling space at a site level).

Here's the odd paradox, though. From Day One, Google did such a good job of helping publishers get paid for their remnant inventory, they did a poor job of forging relations with larger publishers and with big advertisers who want specific media buys in prestige locations and are willing to pay more for it. The recent changes to the content targeting program are a belated recognition of the high-CPM ad market that needs to be treated with more care. Quigo, a key competitor of Google's and Yahoo's in the contextual ad-serving space, built a platform that does a better job of this. In some sectors, such as automotive, Quigo has also done a better job of convincing publishers and advertisers that prestige content needs its own auction. More on the exciting Quigo story soon.

(Hat-tip to Andy Beal)

It sounds like MSN Adcenter will toss factors like user interest (CTR) broken down by target demographic into the mix of ranking factors for paid search ads.

So it will be "more like Google's than Yahoo's" solution, but taking the complexity an additional step forward.

This will make it difficult for the advertiser to fully control their ad delivery or fully understand why an ad is placed in a certain spot (similar to Google's black box algo approach).

Two implications of this: (1) your head will hurt, so you'll hire me, and my head will hurt -- your pain is my pain; (2) it makes very little sense to dabble with the proprietary analytics solutions of a particular analytics vendor, given the importance of having impartial parties assess your campaign results. Third-party analytics vendors are going to be around for a long time, no matter how much sophisticated product the traffic vendors develop under their own umbrellae.

View Recent Posts

Speaking Engagement

Need Solid Advice?

Posts from 2002 to 2010

Speaking Engagement Need Solid Advice?

Posts from 2002 to 2010

Speaking Engagement

Need Solid Advice?