Spiders vs. Directories

First of all, for those who may not be familiar with the difference between spidered engines and directories, a quick primer (all you veterans who are rolling your eyes can skip forward to the next section). Spidered engines use an automated agent called a spider to crawl through the web, grabbing key information from pages and storing it in a huge database called an index. When you perform a search, the engine does a query on the database, or index, and the results you see are pulled directly from it. The order of the returned results depend on that engine’s algorithm, or the magic formula it uses on the collected bits of information to determine what ranks number one and what ranks 24,536th.

With a directory, a warm blooded human (although this is debatable with some of the editors I’ve encountered) visits sites, decides which to include in the directory and compiles a directory listing, generally consisting of the title of the site and a short description. The directory is divided into main topic areas and a huge number of sub-topics. The editor has final say over which topic or sub topic you’ll be listing under. Generally they favour a very specific sub topic that could be several levels in from the main topic area. Although most directories let you suggest titles, descriptions and the appropriate category, the final decision rests with the editor.

Why Are Directories Important?

Directories are important primarily because one, Yahoo, has consistently garnered a huge chunk of all search portal traffic. It was the first and it’s still the biggest. Depending on the source of traffic monitoring, Yahoo is credited with one third to almost 60% of all search traffic. That means well over 100 millions searches are launched on Yahoo daily.

Are Directories Good Search Tools?

Not really. By their very nature, directories tend to give a pretty superficial over view of the web. Keyword searchability is limited to the words found in the title, the description and the category under which a site could be found. It’s pretty hard to give an accurate view of the content of a 5000 page site in 20 words or less. For this reason, directory results are usually supplemented by results from a spidered engine. Yahoo uses Google to provide supplement search results. LookSmart’s directory results can be found paired with spidered results on MSN, AltaVista and Exite. Open Directory results back up spidered results on Netscape Search, AOL Search, Google, Lycos, HotBot and DirectHit.

But the human mind tends to like the categorized approach directories take. It allows the searcher to browse and drill down through topics and sub topics, even if he’s not sure exactly what he’s searching for. For this reason, directories will always provide a significant portion of search traffic.

Please Support Your Favorite Directory

Like the spidered search engines, directories ran up against a brick wall when it came to turning a buck. Banner advertising revenues just couldn’t pay the bills. So in 1999 they started looking for alternative forms of revenue. Yahoo was the first to offer the Express Submission, in November of 1999. For $199 US, you could be sure that a Yahoo editor would actually review your site for inclusion in the index. Prior to the Express Submission, it could take months and repeated submission before you happened to hit an editor with enough time to actually take a look at your site. Since then, LookSmart has followed suit with their own Express Submission charge and a recently announced paid placement program. Open Directory doesn’t have any submission fees, probably because it proudly promotes itself as a volunteer driven directory and accepting money for work done by volunteers may taint this altruistic image.

Ranking the Directories

As with the engines, I’ll try to give you straight advice about the best place to spend your online submission budget. Here’s the ratings:

	Include this in your budget, it's a necessity
	If you can't afford it now, plan on trying it in the next 6 months
	Try it if you have all the major bases covered and you can do it at minimal cost
	I wouldn't bother, but it's your money
	If you liked this one, I've got some Beachfront property in Saskatchewan you're going to love!

Yahoo: The Search Portal Juggernaut

Like it or not, you can’t ignore Yahoo when it comes to search engine traffic. Depending on your target audience, Yahoo could very well produce half of all the traffic you’ll get from all search portals together. Unlike LookSmart and Open Directory, Yahoo’s directory results don’t appear on other search portals, but then they don’t have to. Not when Yahoo is perpetually one of the 5 busiest sites on the Web. For this reason, the $199 Express Submission charge is a must. It’s a one time charge that will pay you back many times over.

But before you plunk down (or plug in) your credit card, make sure you’re ready for the submission. Getting into the Yahoo index is still no guarantee that you’ll be found. It’s VITALLY important that you do a thorough keyword analysis and have a very good idea of your top 4 or 5 keyword phrases. Then, it’s a matter of diligently searching out the right categories to be found under. The right keywords have to appear in the right places in your title, description and categories, or you won’t show up under any keyword searches, which represents the vast majority of searches launched on Yahoo.

It’s very important that this be done correctly the first time. It’s not even the $199 charge. It’s the fact that if you are included in Yahoo with a poor title or description, it’s virtually impossible to get them to change it. Add to this the fact that the title and description have to be carefully worded so they don’t appear to be stuffed with keywords or you’ll have a Yahoo editor gleefully slashing away your keyword rich copy.

Bottom line: sweat bullets over your Yahoo submission or have a reputable positioning firm do it for you. It’s the most important bit of search engine marketing you’ll ever do.

Yahoo Express Submission…

LookSmart: Worthy Competition to the Yahoo Crown

LookSmart has done an excellent job of signing up strategic partners to carry it’s directory listing. You’ll find LookSmart on AltaVista, MSN, iWon and Excite, as well as other sites such as CNN.com. LookSmart managed to sign on major players by offering them a cut of any submission fees gathered through their site. This allowed them to do an end run around Open Directory, who could only offer free directory listings.

LookSmart claims to cover 73% of all online traffic (similar to Goto’s claim of 75% coverage) but you have to understand that that doesn’t guarantee a LookSmart directory listing will give you 75% total online visibility. It just means at some point, 73% of the people on Net will visit a site that happens to carry LookSmart listings. It’s a pretty loose number to hang a claim on, but that’s marketing for you.

Initially, LookSmart matched Yahoo’s Business Express Submission program. Recently, LookSmart’s jacked their price up significantly. For $299, you can put your submission on the fast track and have it reviewed within 2 business days. For half that amount ($149) you’ll still receive a visit from a LookSmart editor, but it might take up to 8 weeks. Unless you’re in a huge rush, save yourself the $150 and go this route.

Another feature that LookSmart has recently added is the Express Modify service. As I said with Yahoo, it can be a royal pain changing a description once you’re in a directory. LookSmart has recently recognized this and used it as an additional source of revenue. For $199, you can change your description and have it reviewed within 5 business days. LookSmart also allows you to add your listing to another category for $149. I would expect Yahoo to sit up and take notice of this addition. Don’t be surprised if they add a similar program soon.

In addition, LookSmart is selling the top three spots on it’s search listing page under the title “Featured Listings”. Unlike Goto, LookSmart’s listing aren’t sold on a straight bid basis, so you won’t find yourself bumped out of top spot the very next day after your bid is placed. Unfortunately, LookSmart requires a $2500 monthly minimum for a LookListings account.

LookSmart Listings
Basic Paid Submission
Express Paid Submission
LookListings

Open Directory

Open Directory is, according to it’s site, “The Largest Human Edited Directory of the Web”. It relies on a huge army of volunteer editors. This directory supplies directory listings to Netscape Search, AOL Search, Direct Hit, Hotbot, Google and Lycos.

The good news about Open Directory is that there are no Express Submission fees. Submitting your URL is still free. The bad news is that depending on your volunteer editor, it can take quite a while to be included and the quality of the editors is rather inconsistent. Often you’ll be scratching your head in bewilderment when you see what this helpful volunteer did to your carefully constructed description. You might also find your self in a completely inappropriate category. With this directory, you don’t pay your money, but you still take your chances.

While Open Directory doesn’t really qualify under the paid submission topic, it is important in your overall search engine marketing program so I’ve included it here.

Free Submission to Open Directory (Can’t beat the price!)

Next Time Around

As the search engine landscape changes, people are wondering why hire a search engine optimization company when you have to pay the engines anyway. Next time, straight from our home office in Kelowna, BC, I’ll give you the top ten reasons why you still need an SEO firm.