Monday, July 10, 2006
Two words: branded recommendations.
Opinionated categorization has been a key driver of web search. And now in the kerfuffle about the New York Times cloaking, and in lawsuits against Google, it's come to our attention that Google is actually editorializing when they choose to index and rank results in a certain way. Duh.
Mysterious processes of editorializing are going to lead to less compelling results. Algorithms that tap into "collective" wisdom are helpful, but easily attract spam schemes, and carry with them a "scientific cachet" that unnecessarily confuses users.
These search algorithms are indeed wonders of the modern world, don't get me wrong.
But it's evident that something akin to an Expert Council would be as useful and fun for users as the existing experimental methods that attempt to measure usefulness and relevancy -- and fight spam -- with forward-thinking mathematical models that look at how the whole world sees a given site or page. In many cases, there is so little data available that the returned results are somewhat arbitrary. A bunch of sites squabble over long tail search referrals by "optimizing" and fretting about why they don't have more of that valuable traffic in obscure search queries. And when most sites can't make it high on the trust meter, they get trumped by sites that seem to be just a bit higher. But is a Wikipedia entry or Yellow Pages backfill really the way to help users, in that case?
Arguably, people should know a search engine isn't good for everything, so the best they can expect on their obscure queries is a jumble of results and ads, and go find what seems best from there. But in practice, many people aren't that great about doing the finding. And in practice, they use a search engine for pretty much everything.
Engines have made great strides in customizing results -- with OneBox help, and more. But they could improve further. Essentially, they would think more like vertical portals in every vertical of substance. And they would selflessly point users towards the most appropriate "vortals" for their needs. Who's going to do the pointing, though? Anonymous editors hired by the SE companies? The ones who used to think foosball tables and pop were a great job perk? No sir! Users want to hear from their heroes.
Think of the problem search faces by pondering your user experience on a site whose job it is to share consumer opinions about products. (Think epinions, deja.com, and many more.) You'd be searching for information on the latest $500 Taylor Made driver, and all of a sudden there'd be all these 13-year kids in there claiming they're striping 300-yard drives. Yeah, sure, Taylor Made reps. :)
In spite of the fact that we live in ostensibly democratic societies, the "collaborative" part of collaborative filtering still isn't working. It's being gamed. The tail's wagging the dog (yep, I said it over six years ago, and I still think so... as do many of today's Wikipedia critics). And many online recommendations are losing legitimacy with the public. (This varies from community to community, of course. But I would argue that in those niche sites where you can truly believe your peers' recommendations, it's because they are known quantities, people you know as real people with verifiable claims and experiences.) Back in the ascendancy of the dmoz directory -- if you're a normal web user and not someone who worked there, put up your hand and tell me if you really wanted travel categories edited by some random enthusiast named "monkeybrayn".
The branded recommendations scheme could work in a couple of ways. One, search engines could actually go around and sign up a bevy of experts and celebrities and enlist them to participate in a recommendations scheme facilitated by technology. The expert's photo or logo could adorn results that show up in the OneBox, in the 3rd search results position, or in the right-hand margin in place of an ad. Compensation? No problem. As long as the rough parameters are disclosed on the search engine's site, people understand that experts are compensated but that they lose their status as experts if they don't give good advice. For lighter topics, who even cares about "relevancy" or "bias," since the picks would be "for entertainment purposes only."
The second method would be to develop metadata or other approaches that attempt to pin down ownership and authorship, so that real-world credentials matter more in weeding out spam. These kinds of checks and tests could be largely concealed to keep spammers off balance. There's no question search engines are doing some of these things today. Sadly, the "semantic web" concept which might have accelerated this trend still seems stalled in the garage.
Both approaches could work in tandem.
This is one of those areas where a company like Microsoft or Yahoo could have shown substantial leadership. What did they do instead? Built their own search algorithms and "MyWeb" schemes that attempted to out-Google Google, and which continued to overrely on anonymous humans and impersonal algorithms to rank-order "web pages". Meanwhile, Ask.com jettisoned its humanistic answer sets and natural language concepts (to say nothing of the butler) in favor of ever-more-impressive science that will be applied to an ever-more-spammy and ever-expanding in complexity web universe.
This isn't a debate about whether humans or computers do it better. A non-starter, to be sure. Computers aren't optional when it comes to search. But audiences are seeking guidance, and the idea of collaborative filtering is so powerful that it deserves to be refined and implemented in new ways, including ways that de-emphasize the "collaborative" aspect. Occam's Razor cuts through clutter and spam all in one go.
Ironically, it may be Google that understands this best, in spite of initially building their company around the world's fascination with PageRank.
The biggest hurdle to this, perhaps, may be the hubris that comes from building a company around 1,000 Ph.D.'s, buying up all the world's fiber, etc.
It now looks like partnerships with traditional media companies and relationships with traditional... people are the way you build a truly trusted media company. And those relationships are hard to build when so many in "new media" build their empires out of ripping and sharing little pieces of other people's content, releasing every new idea as if it were merely a product, rather than a product that requires a relationship.
View Posts by Category
Andrew's book, Winning Results With Google AdWords, (McGraw-Hill, 2nd ed.), is still helping tens of thousands of advertisers cut through the noise and set a solid course for campaign ROI.
And for a glowing review of the pioneering 1st ed. of the book, check out this review, by none other than Google's Matt Cutts.
Posts from 2002 to 2010