Latest Bytes on Search Engine Marketing

By David Gikandi
Posted Saturday, October 23, 2004

The search engine world is always changing. They are always up to something new, using new technology, coming up with new requirements and rules, and, of course, running into new glitches. As a web site owner, you need to know exactly what they are up to at any one moment so that you can get your site well placed in their indexes. After all, search engines deliver the bulk of the traffic most sites get, so you cannot afford to ignore them. So let us have a look at what exactly is going on with various engines and how we can us this knowledge to our marketing benefit.

Tip: Use unique IP addresses for each site

Consider using a unique domain name and IP addresses for your doorways. Once in a while, it can happen that you go too far in your efforts to use various kinds of gateway pages (doorways, information frames, etc) to get higher placements on search engines. For example, you may be using too many keywords that are too alike and thereby ending up with duplicate pages or a cluster of high rankings for the same keyword. AltaVista is especially watchful for such mistakes. The penalties for such an error are usually just the penalization of the offending page, or at worst case the search engine may block any new submissions from the offending site. To make sure that such a mistake will have no negative effect on your web site even if your gateway pages do spam an engine due to improper use, it is highly recommend that you get a unique domain name that has its own IP address (those numbers that look like 206.67...789 that your hosting service tells you point to your domain name). Get two domain names, one for your site and one for your web promotion material such as doorways. For best effect, ensure that these domains are hosted by different hosting services. That guarantees that each domain name will definitely have a unique IP address. If you want to stick to one hosting service, ask your web hosting service to make sure that each domain name is assigned an IP address that is unique to it. While you are at it, get your hosting service to ensure that (a) the IP address given to you is not shared by other sites on their server and (b) it has not been used before by a previous site that was on their server. Any of these situations can cause you to get penalized by the engines by spamming done by the sites that share your IP address or the one that used to own it.

Another good reason to have two domains is that AltaVista and Infoseek seem to have a limit on the total number of pages you can have indexed from one site. Infoseek supposedly limits this to 600 pages per domain. AltaVista's limit is much higher, if they do have one. So having two domains and splitting your pages among them can be a good idea if you have many pages.

Finally, for most of the major engines, having links to your main site from another domain adds to your rankings. So your main site will rank better if the doorways pointing to it are from a different domain than from within it. Yet another good reason to get a new domain name and hosting account just for your doorways.

While you are at it, try to get a long domain name for your doorways that is made up of your most important keywords. For example, if your site is about real estate in New York, you might want to get the name (www.realestatenewyorkapartmentshouses.com) or something like that. Keywords in the domain add to higher rankings.

How-to: How to submit to the search engines

For all the major search engines, it is highly recommended that you only submit your home pages and hallway pages (hallways are pages that contain links to all your site's pages so search engines can find them). Let the search engine crawl and find your other pages. Make sure that you wait at least 30 minutes in between each submission (e.g. if you want to submit a home page and two hallway pages to Hotbot, submit the home page, wait at least 30 minutes, submit the hallway page, wait again, then the next hallway page). For best results, submit these pages manually using the search engine's submission page. Do not submit your other site pages. This is in line with the latest recommendations issued by the search engines. In fact, on some engines you may get tagged for spam review or get extra submitted pages ranked lower if you submit anything other than the home and hallway pages. For all engines, re-submit your home and hallway pages once every fortnight to ensure that they remain indexed.

Tip: Infoseek submission

Infoseek's Add URL function had been down for quite a while but is now back up. However, it seems to now take about 5 - 7 weeks for newly submitted sites to get indexed. Also, it now only accepts submissions for the home page and not any other sub page. Fortunately their crawling seems to be good and eventually the spider will come to your home page and follow all links it finds on the home page to index sub pages. So forget about submitting other pages on your site - just submit the home page and make sure that all other pages on your site can somehow be reached from the home page by the crawler.

Infoseek also uses channels as a search option. If you merely submit to the usual Add URL link for the general search engine, you wont be found in the channels. So browse the Channels on Infoseek and find one suitable for your site and submit your home page there.

Tip: Excite delays and subdirectories issue

Excite is still, in our experience, the slowest of the major engines when it comes to getting new pages indexed. It takes months to get pages indexed at times. We hope this is going to improve soon. We are sure a negative image is not what they want so we are betting that they are working on fixing this. The point to note is that Excite seems to be indexing every few months, talking in pages that were previously submitted before. So you should go ahead and treat it as if it works - submit your site as usual and re-submit every fortnight. When they do index again, at least you will be in the queue and get indexed.

Excite also seems to drop pages in subdirectories, favoring those pages in the root directory. It is highly recommend that you have your doorways and most other pages in the root directory (that's where your home page is) and make sure you have your own domain name. If you cannot do that, get a separate domain name and hosting account just for your doorways (even better!) so that you can have them in the root directory.

Tip: Lycos

Nothing much new hear but one small tip. The best way to get into Lycos is to submit your site to Lycos as usual AND submit to (www.dmoz.org), the Open Directory Project that Lycos and many other engines use as a source of results on certain searches. Lycos also uses the FAST engine for some of its results so you should also submit your site to the FAST engine (at (www.alltheweb.com).)

Tip: Hotbot

Hotbot is still mysteriously dropping pages off their index for no apparent reason. Other than that, they are doing fine. Here is a tip from (www.searchengine-news.com). Their newsletter said that the fastest way to get indexed by Hotbot at the moment is to submit to Canada.com at (http://canada.com/search/web/addurl.asp). Try it! You should also submit to (www.dmoz.org), the Open Directory Project, which Hotbot uses.

Hotbot is also the first engine to index dynamic pages (pages created on the fly from a database or CGI script). To have your dynamic pages indexed, if you have any, simply submit your dynamic page with all the arguments added onto the URL (e.g. (www.mysite.com/page.asp?catalog=5369&state=florida)) or submit a static page with a bunch of dynamic URLs you need indexed.

Note: AltaVista "Too many URLs problem"

AltaVista recently installed an ultra-tight and aggressive spam filter which according to numerous reports is causing a very large problem to thousands of web site owners. The spam filter is often improperly tagging many pages as spam. The good news is that your site is not dropped off their index if you do get tagged, but the offending pages are given low ranking and you are blocked from submitting any new ones. So it isn't a very grave situation and especially because way too many webmasters have been affected unfairly this situation will likely get better soon. For most webmasters, simply avoiding the known spam techniques such as hidden text etc and abstaining from submitting more than one page a day at the most should keep you clear off their spam filter. If you feel you have been cut off from new submissions unfairly, simply use AltaVista's feedback form and let them know and they will put you back up.

Tip: DirectHit, LookSmart and Open Directory Project submissions

Several search engines such as HotBot and Lycos use DirectHit and the Open Directory Project as a major source of their search results. It has been found advantageous, for this reason, to also submit your site directly to DirectHit at (http://www.directhit.com/util/addurl.html) as this gets you a better position much faster because DirectHit updates its engine much more timely than the engines that use it such as Lycos and Hotbot. You should also submit to (www.dmoz.org), the Open Directory Project, as this is the only way to get into certain results served by an increasing number of search engines (over 20 major search engines and directories use it). LookSmart is also being used by a very large number of search engines and directories so be sure to submit there too at (www.looksmart.com).

Suggestion: Automatic submission tools plus getting listed on as many places as possible

Automatic submission tool (software or web based) are extremely useful and timesaving when it comes to submitting your site to several hundred search engines and directories. By all means, use them. But make sure that you do not use them for Infoseek and AltaVista if you are submitting more than just one page. These two engines have high-speed submission detection that can get your submitted page penalized. How this works is that they watch for more than one submission per, say, 30 minutes from any one site. So if you submit two pages from your site to Infoseek or AltaVista in less than 30 minutes, that's not good for you.

On the other hand, you should use these tools and get yourself listed on as many places on the web as possible - directories, search engines, free-for-all link pages, About.com, etc. It is very important, especially for Infoseek rankings, that you have as many incoming links into your site from other sites as possible, especially from larger sites such as Yahoo and About.com.

Final Note: Delayed index updates

Most of the major engines are now taking between 30 and 90 days to refresh their databases, although their information pages claim shorter periods than that. What this means is that a page you submit today and any pages on your site linked from it will appear on search results in 30 to 90 days. Excite is especially slow. The Internet community believes that this situation will definitely change because no engine wants to have old information in its database - its bad customer service to its users. So they will work at correcting these delays.

This is a time of great opportunity, and marketing your site through search engines is one of the world's greatest advertising opportunities. Where else can you reach such vast numbers of people worldwide at practically no cost to you? So it does pay to keep a close tab on the engines and use them wisely but ethically to your advantage.

If you need any more information on this topic, please see the Additional Resources section at (http://www.positionweaver.com).

David Gikandi (support@positionweaver.com), CEO at SearchPositioning.com (http://www.positionweaver.com).

About the Author
None