Traffick - The Business of Search Engines & Web Portals
Blog Categories (aka Tags) Archive of Traffick Articles Our Internet Marketing Consulting Services Contact the Traffickers Traffick RSS Feed

Tuesday, August 16, 2005

Quality Matters


Danny's fed up with the search engine index size wars, and proposes that the biggies duke it out on a more important front: relevancy (or relevance as we like to call it over here).

He proposes that they all agree on some sort of standard test and to have an independent institute or consortium run the tests.

I don't entirely agree on the idea of a common definition of relevance. Search personalization, for example, can take on a variety of forms. In theory, every user would have a personalized set of results sitting in front of them.

Language itself shifts over time. Definitions of what is true often depend on what scientific camp you're in.

But yes, in an enlightened world, scientists do need to accept at least basic overlapping truths.

So it should be possible to start with baby steps. SE's probably won't agree to anything, internally or amongst themselves, that highlights things like SE index spam.

But it would be pretty easy to (a) take a broad basket of keywords and (b) some commonly-agreed benchmarks for what counts as a "spammy page" (even a scoring system); and have assessments done by (c) qualified reviewers to come up with a determination of how contaminated the major SE's are with spam, on a diverse basket of keywords, in the top 20 listings.

RustyBrick over on Search Engine Watch Forums tried something like this, but IMHO it was too open-ended. I propose a slightly different approach to it that didn't ask raters to determine which engine was most relevant, but rather, merely count how many pages in the top 20 on the sample keyword queries exceed a certain "spamminess score." We're talking about scraped pages, redirects, machine-generated gibberish pages... the real nasty stuff, which appears on a great many queries where it shouldn't. It wouldn't necessarily penalize sites for using spammy techniques, though. If someone's participating in a link farm, or cloaking, or keyword stuffing in the title tag, but the page the user sees is relevant to their query and likely to lead to a desired result (gaining real insight from original content, making a purchase from the type of vendor they were probably looking for, joining a forum, etc.), then the page shouldn't be counted as spammy. Actually, SE's have been thinking along those lines, too. How often have you seen someone using outdated optimization techniques like keyword stuffing in titles and tags, and yet the page itself would have ranked OK anyway, and the SE's actually do rank it well without penalizing it? SE's rightly look past a lot of the stuff we might consider "spammy," as long as the page is relevant.

This type of exercise wouldn't require us to split hairs in defining relevance. It would give us a base to work from that virtually any sentient, rational being would agree on. If snippets of gibberish content are stolen from multiple sources to create a junk page whose only purpose is to generate a bit of AdSense revenue, etc., then that's obvious spam. Users aren't seeking pages of stolen gibberish content... ever. Nor are they wanting a redirect to a casino site when they type "fantasy football statistics 2004."

In other words... in the parlance of applied social sciences, we need to "operationalize" relevance so coders can actually do their jobs consistently.

Danny, Rusty, if you like this idea, count me into the working group. We could hash out a scoring system on what counts as a "spammy page," choose a broad (but confidential, to avoid gaming by the SE's) keyword basket, and round up coders to assess the major engines. This would give us a real "SE spamminess index" as opposed to a highly subjective "relevancy score." It would get published in the Wall Street Journal before you know it, alongside some of those other famous SEM indexes they've been kicking around lately.

Who's in?

Posted by Andrew Goodman




View Posts by Category

 

Speaking Engagement

See Andrew Goodman speak at eMetrics Chicago 2014

Need Solid Advice?        

Google AdWords book


Andrew's book, Winning Results With Google AdWords, (McGraw-Hill, 2nd ed.), is still helping tens of thousands of advertisers cut through the noise and set a solid course for campaign ROI.

And for a glowing review of the pioneering 1st ed. of the book, check out this review, by none other than Google's Matt Cutts.


Posts from 2002 to 2010


07/2002
08/2002
09/2002
10/2002
11/2002
12/2002
01/2003
02/2003
03/2003
04/2003
05/2003
06/2003
07/2003
08/2003
09/2003
10/2003
11/2003
12/2003
01/2004
02/2004
03/2004
04/2004
05/2004
06/2004
07/2004
08/2004
09/2004
10/2004
11/2004
12/2004
01/2005
02/2005
03/2005
04/2005
05/2005
06/2005
07/2005
08/2005
09/2005
10/2005
11/2005
12/2005
01/2006
02/2006
03/2006
04/2006
05/2006
06/2006
07/2006
08/2006
09/2006
10/2006
11/2006
12/2006
01/2007
02/2007
03/2007
04/2007
05/2007
06/2007
07/2007
08/2007
09/2007
10/2007
11/2007
12/2007
01/2008
02/2008
03/2008
04/2008
05/2008
06/2008
07/2008
08/2008
09/2008
10/2008
11/2008
12/2008
01/2009
02/2009
03/2009
04/2009
05/2009
06/2009
07/2009
08/2009
09/2009
10/2009
11/2009
12/2009
01/2010
02/2010
03/2010
04/2010

Recent Posts


Miva's Almost Out of the Woods

Google Sandbox = Purgatory

The Slow Erosion of User Confidence

Stonecutters' Power Diminishes: Internet to Blame

Is Google Getting Into Behavioral Targeting, or No...

Secondary Sources of Targeted Traffic: All Part of...

At Least My Mom Loves Me

China Week, Cont'd

Google Authorized Resellers in China

146,000 Backlinks Woohoo!

 


Traffick - The Business of Search Engines & Web Portals

 


Home | Categories | Archive | About Us | Internet Marketing Consulting | Contact Us
© 1999 - 2013 Traffick.com. All Rights Reserved