Traffick - The Business of Search Engines & Web Portals
Blog Categories (aka Tags) Archive of Traffick Articles Our Internet Marketing Consulting Services Contact the Traffickers Traffick RSS Feed

Monday, July 10, 2006

Fighting Search Engine Spam Part 2: My Baby Likes a Bunch of Authors (and Critics, and Scientists, and Celebrities, Designers, Analysts, Athletes...)

Two words: branded recommendations.

Opinionated categorization has been a key driver of web search. And now in the kerfuffle about the New York Times cloaking, and in lawsuits against Google, it's come to our attention that Google is actually editorializing when they choose to index and rank results in a certain way. Duh.

Mysterious processes of editorializing are going to lead to less compelling results. Algorithms that tap into "collective" wisdom are helpful, but easily attract spam schemes, and carry with them a "scientific cachet" that unnecessarily confuses users.

These search algorithms are indeed wonders of the modern world, don't get me wrong.

But it's evident that something akin to an Expert Council would be as useful and fun for users as the existing experimental methods that attempt to measure usefulness and relevancy -- and fight spam -- with forward-thinking mathematical models that look at how the whole world sees a given site or page. In many cases, there is so little data available that the returned results are somewhat arbitrary. A bunch of sites squabble over long tail search referrals by "optimizing" and fretting about why they don't have more of that valuable traffic in obscure search queries. And when most sites can't make it high on the trust meter, they get trumped by sites that seem to be just a bit higher. But is a Wikipedia entry or Yellow Pages backfill really the way to help users, in that case?

Arguably, people should know a search engine isn't good for everything, so the best they can expect on their obscure queries is a jumble of results and ads, and go find what seems best from there. But in practice, many people aren't that great about doing the finding. And in practice, they use a search engine for pretty much everything.

Engines have made great strides in customizing results -- with OneBox help, and more. But they could improve further. Essentially, they would think more like vertical portals in every vertical of substance. And they would selflessly point users towards the most appropriate "vortals" for their needs. Who's going to do the pointing, though? Anonymous editors hired by the SE companies? The ones who used to think foosball tables and pop were a great job perk? No sir! Users want to hear from their heroes.

Think of the problem search faces by pondering your user experience on a site whose job it is to share consumer opinions about products. (Think epinions, deja.com, and many more.) You'd be searching for information on the latest $500 Taylor Made driver, and all of a sudden there'd be all these 13-year kids in there claiming they're striping 300-yard drives. Yeah, sure, Taylor Made reps. :)

In spite of the fact that we live in ostensibly democratic societies, the "collaborative" part of collaborative filtering still isn't working. It's being gamed. The tail's wagging the dog (yep, I said it over six years ago, and I still think so... as do many of today's Wikipedia critics). And many online recommendations are losing legitimacy with the public. (This varies from community to community, of course. But I would argue that in those niche sites where you can truly believe your peers' recommendations, it's because they are known quantities, people you know as real people with verifiable claims and experiences.) Back in the ascendancy of the dmoz directory -- if you're a normal web user and not someone who worked there, put up your hand and tell me if you really wanted travel categories edited by some random enthusiast named "monkeybrayn".

The branded recommendations scheme could work in a couple of ways. One, search engines could actually go around and sign up a bevy of experts and celebrities and enlist them to participate in a recommendations scheme facilitated by technology. The expert's photo or logo could adorn results that show up in the OneBox, in the 3rd search results position, or in the right-hand margin in place of an ad. Compensation? No problem. As long as the rough parameters are disclosed on the search engine's site, people understand that experts are compensated but that they lose their status as experts if they don't give good advice. For lighter topics, who even cares about "relevancy" or "bias," since the picks would be "for entertainment purposes only."

The second method would be to develop metadata or other approaches that attempt to pin down ownership and authorship, so that real-world credentials matter more in weeding out spam. These kinds of checks and tests could be largely concealed to keep spammers off balance. There's no question search engines are doing some of these things today. Sadly, the "semantic web" concept which might have accelerated this trend still seems stalled in the garage.

Both approaches could work in tandem.

This is one of those areas where a company like Microsoft or Yahoo could have shown substantial leadership. What did they do instead? Built their own search algorithms and "MyWeb" schemes that attempted to out-Google Google, and which continued to overrely on anonymous humans and impersonal algorithms to rank-order "web pages". Meanwhile, Ask.com jettisoned its humanistic answer sets and natural language concepts (to say nothing of the butler) in favor of ever-more-impressive science that will be applied to an ever-more-spammy and ever-expanding in complexity web universe.

This isn't a debate about whether humans or computers do it better. A non-starter, to be sure. Computers aren't optional when it comes to search. But audiences are seeking guidance, and the idea of collaborative filtering is so powerful that it deserves to be refined and implemented in new ways, including ways that de-emphasize the "collaborative" aspect. Occam's Razor cuts through clutter and spam all in one go.

Ironically, it may be Google that understands this best, in spite of initially building their company around the world's fascination with PageRank.

The biggest hurdle to this, perhaps, may be the hubris that comes from building a company around 1,000 Ph.D.'s, buying up all the world's fiber, etc.

It now looks like partnerships with traditional media companies and relationships with traditional... people are the way you build a truly trusted media company. And those relationships are hard to build when so many in "new media" build their empires out of ripping and sharing little pieces of other people's content, releasing every new idea as if it were merely a product, rather than a product that requires a relationship.

Posted by Andrew Goodman




View Posts by Category

 

Speaking Engagement

I am speaking at SMX West

Need Solid Advice?        

Google AdWords book


Andrew's book, Winning Results With Google AdWords, (McGraw-Hill, 2nd ed.), is still helping tens of thousands of advertisers cut through the noise and set a solid course for campaign ROI.

And for a glowing review of the pioneering 1st ed. of the book, check out this review, by none other than Google's Matt Cutts.


Posts from 2002 to 2010


07/2002
08/2002
09/2002
10/2002
11/2002
12/2002
01/2003
02/2003
03/2003
04/2003
05/2003
06/2003
07/2003
08/2003
09/2003
10/2003
11/2003
12/2003
01/2004
02/2004
03/2004
04/2004
05/2004
06/2004
07/2004
08/2004
09/2004
10/2004
11/2004
12/2004
01/2005
02/2005
03/2005
04/2005
05/2005
06/2005
07/2005
08/2005
09/2005
10/2005
11/2005
12/2005
01/2006
02/2006
03/2006
04/2006
05/2006
06/2006
07/2006
08/2006
09/2006
10/2006
11/2006
12/2006
01/2007
02/2007
03/2007
04/2007
05/2007
06/2007
07/2007
08/2007
09/2007
10/2007
11/2007
12/2007
01/2008
02/2008
03/2008
04/2008
05/2008
06/2008
07/2008
08/2008
09/2008
10/2008
11/2008
12/2008
01/2009
02/2009
03/2009
04/2009
05/2009
06/2009
07/2009
08/2009
09/2009
10/2009
11/2009
12/2009
01/2010
02/2010
03/2010
04/2010

Recent Posts


Fightin' Search Engine Spam, Part I

Death of Common Culture?

Playing Knockoff

CPA the Real Reason for Google Checkout?

Thinking of Using Google Checkout? Here's What You...

Google, the Micropayments Savior?

Microsoft adCenter Labs: Anything New to See Here?...

Search Engine Results and Sociopolitical Controver...

I Knew Real Estate Was Getting Out of Hand in the ...

Jagger's Away from the Switch

 


Traffick - The Business of Search Engines & Web Portals

 


Home | Categories | Archive | About Us | Internet Marketing Consulting | Contact Us
© 1999 - 2013 Traffick.com. All Rights Reserved