Traffick - The Business of Search Engines & Web Portals
Blog Categories (aka Tags) Archive of Traffick Articles Our Internet Marketing Consulting Services Contact the Traffickers Traffick RSS Feed


Searchonomics: A New Era in Human-Guided Web Search?

Legend has it that a major plank in Ronald Reagan's economic platform - the Laffer Curve - was sketched out for him on a napkin. On this foundation, Reagonomics became the ovverriding theme of a decade of US fiscal policy. A vast array of reforms flowed from some basic reasoning that essentially argued that if you put a luxury tax on boats, sales will decline, and the luxury boat industry will dry up.

A lot of powerful ideas would appear to originate this simply. 1998 was a year in which everyone in the Internet business decided that there was room to improve on Yahoo! - in particular, its directory, which, critics charged, couldn't scale with the growth of Internet content. The overworked team of Yahoo! editors let "link rot" set in, and ignored many submissions.

Can't anyone be a "surfer"?

It's telling that Yahoo!'s job descriptions refer to directory editors as "surfers." If you're thinking "just about anyone could do that," you're not far from wrong. And in fact, there is a huge thirst amongst heavy Internet users to get more involved in rating sites, offering advice, publishing articles, and ranking products. A host of innovative business models are appearing on the scene today, promising to give a platform, and sometimes some pecuniary rewards, to these legions of enthusiasts and experts. This new type of knowledge worker may potentially fuel the next leap forward in information delivery on the Internet.

One team of inventors, led by Steve Thomas, a software developer and aficionado of cognitive science, sketched out some principles for the ultimate web search resource "in a one-page concept paper" in February of 1998. Barely larger than the proverbial napkin, this paper kicked off a process of planning and development for a new system of "Searchonomics" under the company name Wherewithal.

Founding principles

Thomas had been Netscape's Product Manager, Platform - responsible for Java/Javascript, DHTML, and inventing something called "layers." With Netscape since shortly after its IPO in 1995, he left in 1998 to start Wherewithal. Along with co-founder and CIO Darren Skinner, Thomas sketched out key principles for a new web search resource to end all resources:

  • Scalability. Observing Yahoo! and other human-managed directories, Thomas and his team believed that most of them were designed with assumptions of the "small scale Internet" in mind. After all, Yahoo! began as Jerry Yang's Guide to WWW, a helpful guide to Internet content in different subject areas. It wasn't all that hard to build. Set up an arbitrary taxonomy, get a few people working on it, and reasonable coverage of the web was possible. As the Internet expanded, this model, in Wherewithal's view, became obsolete.

  • Human element. One thing Yahoo! did have going for it was the judgment employed by category editors. Robots can be inflexible, and do a poor job of acting as "gatekeepers" and judges of what's good and what isn't. The raw knowledge available on the Internet needs to be digested by "informediaries" if it is to be useful.

  • The possibility of multiple taxonomies. If you think about a Yahoo! category, or for that matter, the category structure in any human-edited directory or web guide (eg. Looksmart, Open Directory, or About), the choice of how to organize the resources in a given field - which to highlight, which to downplay, which sites to accept, and which to reject - is made by a single editor. Even if there is a committee system in place in an editorial organization, the end user only sees one category structure and one opinion as to what resources about sports cars (for example) are most useful. Such an arbitrary method for a medium which is supposed to be about flexibility, self-publishing, even chaos! The Wherewithal plan needed to come up with a way for the end user to substitute their favorite editor (a so-called Second Opinion) in place of one which they felt wasn't suitable for their needs. This builds accountability into the system. There is no such thing as a "bad" editor (the users will determine that), and therefore command-and-control administration of the editorial personnel isn't needed.

  • A sustainable incentive system. Why would anyone be a category owner in the first place? The work of infomediaries has value. Over time, some analysts predict, the compensation of such infomediaries will adjust to closer reflect the value of that work - in part, through choices made by the infomediaries themselves. We're well past the days when we simply volunteered advice in Usenet newsgroups without any expectation of reward. The act of volunteering advice or pointing people towards useful resources isn't going to go away, but most infomediaries and consultants today have a sharper sense of how soon to put their "clients" on the "meter." In a manner similar to About, which pays its guides a percentage of the revenues generated from their About.com Guide Sites, Wherewithal's Searchonomics concept pays editors a portion of the revenues from targeted advertising in their directory categories. Editors who attract more users through the high quality of their work, or simply through the act of editing in more marketable categories, stand to make more income.
Newhoo Built a Valuable Asset Quickly

As Thomas tells it, Wherewithal became aware of the advent of Newhoo (now Open Directory) about six months into the life of their own project. Wherewithal was diligently working on the back end - the technical and operational system which would give the project legs. By contrast, Newhoo was rushed to market to take advantage of the ready pool of volunteers who had a bone to pick with Yahoo's link rot and limited scalability. Good move - before long, Newhoo had been acquired by Netscape for a tidy sum.

A key thing that the design of Newhoo failed to take into account - at least as far as its work force was concerned - is that, as Thomas puts it, "the ultimate directory of Internet content is a very valuable thing... indeed one of the most valuable things there is." Therefore, the "compensation model needed to be perfect," and a key part of the plan had to be to tie the payouts to the flow of incoming revenues. The thought of not paying editors at all didn't factor into Wherewithal's Searchonomics plan.

To push the Laffer Curve analogy perhaps a bit too far, imagine if you were employed as an editor in a hierarchical republic whose rate of taxation was effectively 100%. Editing activity, in the long run, might drop close to zero as incentive to perform services was greatly diminished, and as contributors opted for citizenship in a republic with a lower tax rate. Now imagine if the rate were pushed close to 0%. Advertising revenues? They're all yours! Great, you, as an editor, get to take home more loot, but the operation isn't fiscally sustainable. The guys who keep the lights on and the engines well oiled suddenly disappear, and you're back to complete anarchy. Wherewithal needed to find a way of conveying the right percentage of advertising revenues to its potential labor force of infomediaries.

Calling all humans

Wherewithal makes a compelling pitch to current Open Directory editors that they might consider upgrading from their less-than-sweatshop pay scale at ODP. In fact, a large part of Wherewithal's current strategy has involved targeting ODP editors, and even allowing them priority in signing up on Wherewithal's system so they can "reclaim" their old categories. This has led to some allegations that Wherewithal intends to make "unauthorized" use of ODP data in contravention to the ODP license. Wherewithal maintains that it is posting ODP attribution appropriately on its web site. ODP data is, indeed, used in different ways by different licensees, including the Google Directory, which doesn't provide ODP attribution on every page, but only insofar as the license requires it.

Thomas and his team have run some interesting tests of the Open Directory's claim to have 30,000 active category editors. In their analysis, there are only 9,000 ODP editors who "actually have a category," and only about 5,000 have done more than one or two edits. A central core of 500 do the bulk of the editing, many observers believe. While humans may do it better, there are fewer humans working for ODP than previously believed. Yahoo! and Looksmart have even fewer than 500.

Thomas believes that the Wherewithal system provides a more robust platform for editors to shine and to be paid well for linking users to the content they're seeking, particularly in targeted commercial sectors. Thus he believes that it will only require 2,000 editors to rival ODP's database in terms of quality. The hope, of course, is to find many more than 2,000.

Some kid with a 486 in the basement?

David Prenatt, Wherewithal's recently-hired Chief Evangelist and a former Open Directory Project editor (see " Life After the Open Directory Project"), was even more optimistic than his boss. "In three months," asserts Prenatt, "our database will make ODP look like some kid with a 486 in a basement."

The fact that editors may earn money for their toils - while important - is not the most impressive feature of Wherewithal. The scalability of the project is compelling. ODP claimed to solve the scale problem, but ran into problems of personnel administration as well as the problem of quality control that is bound to crop up in a volunteer project. Wherewithal, by contrast, allows anyone to own a category, and doesn't have to "replace" bad editors with good ones. End users can, in essence, set up their own customized directories by picking and choosing their preferred category owners. Wherewithal intends to play virtually no centralized role as an arbiter of good content or good editing. "Our model is to stay out of the way, provide the platform, and make sure the computers don't crash," summarizes Thomas. Spoken like a true advocate of laissez-faire.

Another plus associated with the Searchonomics system is its flexibility. Parts of the directory could be plugged into a vertical portal site. The ads could even be turned off if the site wanted to find its own volunteer editors or pay a higher fee for the use of the directory.

T minus $5 million, and counting

Of course, many of these features exist only at the design stage, and await full implementation of the project. Like Reaganomics or a rocket launch, the blueprint might look nice, but you never know how things will shake out until the mission is underway in real time, affected by the steering actions and interactions of those most unpredictable of actors, real people at multiple levels (end user, editor, engineer, site owner, ad rep, advertiser), and the economy they create. Wherewithal is an angel-funded startup with eleven staff, and is currently putting together its proposal for a Series A round of venture capital funding.

To attract the volume of end-user traffic that would generate the needed ad revenues to pay the category owners (one of several "chicken and egg" questions facing Wherewithal), it would seem clear that Wherewithal will, at some stage of its development, find itself portal infrastructure deals - to propagate itself far and wide as companies like Looksmart, Dmoz, and Quiver have sought to do. No traffic, no revenues. No revenues, no company.

Really big computers

Getting back to basics: will this help us find information? In Thomas' view, the sheer number of potential editors - tens of thousands or even millions - that can be built into the Wherewithal system without it cracking under the strain make it the ultimate Internet research tool. One indicator that this startup is dead serious about the scope of its ambitions is that Wherewithal already has *really big computers*. The company's web site contains an impressive description of the design of the search engine technology which powers the system, and the use of dynamic HTML and other means to improve speed. Wherewithal engineers reportedly sought out the number of searches performed on Yahoo! per second at peak times, added a zero, and went to work designing their system to handle a heavy load.

Listening to Thomas' careful description of the various operational and scientific principles behind the project, it's clear that this wasn't in the same league as many business models in the pay-you-to-surf realm, many of which were one-dimensional and unable to keep their promises. Indeed, from Day 1, the goal was to design a web search technology which would embrace the chaos and promise of the Internet. Wherewithal's hands-off, no-hierarchy approach would, paradoxically, offer a means of taming that chaos and fulfilling that promise.

From scribbles on napkins to an Internet revolution called Searchonomics? Stranger things have happened.

Speaking Engagement

I am speaking at SMX West

Need Solid Advice?        

Google AdWords book


Andrew's book, Winning Results With Google AdWords, (McGraw-Hill, 2nd ed.), is still helping tens of thousands of advertisers cut through the noise and set a solid course for campaign ROI.

And for a glowing review of the pioneering 1st ed. of the book, check out this review, by none other than Google's Matt Cutts.


Posts from 2002 to 2010


07/2002
08/2002
09/2002
10/2002
11/2002
12/2002
01/2003
02/2003
03/2003
04/2003
05/2003
06/2003
07/2003
08/2003
09/2003
10/2003
11/2003
12/2003
01/2004
02/2004
03/2004
04/2004
05/2004
06/2004
07/2004
08/2004
09/2004
10/2004
11/2004
12/2004
01/2005
02/2005
03/2005
04/2005
05/2005
06/2005
07/2005
08/2005
09/2005
10/2005
11/2005
12/2005
01/2006
02/2006
03/2006
04/2006
05/2006
06/2006
07/2006
08/2006
09/2006
10/2006
11/2006
12/2006
01/2007
02/2007
03/2007
04/2007
05/2007
06/2007
07/2007
08/2007
09/2007
10/2007
11/2007
12/2007
01/2008
02/2008
03/2008
04/2008
05/2008
06/2008
07/2008
08/2008
09/2008
10/2008
11/2008
12/2008
01/2009
02/2009
03/2009
04/2009
05/2009
06/2009
07/2009
08/2009
09/2009
10/2009
11/2009
12/2009
01/2010
02/2010
03/2010
04/2010

 


Traffick - The Business of Search Engines & Web Portals

 


Home | Categories | Archive | About Us | Internet Marketing Consulting | Contact Us
© 1999 - 2013 Traffick.com. All Rights Reserved