New InSites into the FAST Search Engine (Part 2)
By: Robin Nobles
|Robin Nobles, Director of Training, Academy of Web Specialists, (www.academywebspecialists.com) has trained several thousand people in her online search engine marketing courses (www.onlinewebtraining.com) and is the content provider for (GRSeo) Search Engine Optimizer software (www.se-optimizer.com). She also teaches 3-day hands on search engine marketing workshops in locations across the globe with Search Engine Workshops (www.searchengineworkshops.com).|
"When you register for version III, the FAST spider crawls your entire site, up to 500 pages. In addition to including those pages in the FAST index, we host an index of your site's pages and provide you with a Search My Site tool bar that you can paste into your HTML. This allows site visitors to search only the contents of your site. In the next version, we will provide more control to the subscriber over the site search rankings, but Web search rankings will always be determined by FAST."PartnerSite IV: Designed for sites up to 500 pages and provides bulk inclusion via XML but no site search.
"We have developed our own DTD (Document Type Definition) to integrate XML feeds. XML indexing is actually how we currently index database pages. We have the partner create an XML feed according to our DTD that has the 'page to fetch.' This essentially tells our crawler not to explicitly try to crawl the page, but rather, just 'fetch,' the page's contents."Editor's Note: Document Type Definition defines the legal building blocks of an XML document. See the DTD School for more information: www.w3schools.com/dtd/default.asp
For the latest information on the different versions of PartnerSite, visit:
North American Customers: searchservices.lycos.com/searchservices/
International Customers: www.lycos.de/
Lycos is only one of Fast's pay inclusion program partners in a list that is still growing. For the complete partner list. go to: www.fastsearch.com/products/internet/partnersite_partners.asp
Next subject . . .
What about free submissions to FAST? Are they a thing of the past?
According to Baker,
"Free submit will continue to operate. There will be no effect on pages (already) in the index. The inclusion service is simply a guarantee that your pages will be indexed regularly and are guaranteed a reservation in the FAST index. The trick with free submit is the amount of spam we receive through that channel. It makes it difficult to keep up with all of the submissions. PartnerSite provides an economic incentive to NOT submit spam."
How quickly does FAST index free submissions?
Mr. Baker answered,
"Usually within 2-6 weeks. However, 95% of what comes through free submission is spam, and that's what causes the bottleneck. Also, there's no guarantee on the refresh rate of those sites that we pick up through free submit. Unfortunately, it's the 5% of the submitters that are spammers that ruin it for the rest of us. The extent that people go through to spam the index is truly amazing."
Is there a penalty for submitting your site through their free Add URL?
"Not really. Freshness does effect rank, but only marginally. I have never seen freshness boost ranking more than 1 or 2 spaces."
What does FAST consider spam? As stated earlier, FAST estimates that an amazing 95% of the submissions through their free Add URL page are spam.
Baker further explained,
"Unfortunately this is the case. We believe there are approximately 30 million crawlable servers globally, two-thirds of which have been blacklisted as spam servers."Whew! Think about it: 20 MILLION crawlable servers globally are blacklisted as spam servers. If this is really true, it explains why the engines have collectively gone to such great lengths in their efforts to curtail spam.
At the Dallas Search Engine Strategies Conference in November, FAST introduced their new spam policies. Baker explained that according to FAST, spam comes in three different categories:
- Page Spam, which consists of any measures to boost ranking, such as link farms, etc.
- Spam stuffing, such as keyword stuffing, invisible text, etc.
- Offensive content, which is not so much spam, but is something that we detect and flag as such.
And, if you're so inclined, you can report spam to FAST by emailing email@example.com.
The ultimate fear of a search engine marketer: your site has been banned. What recourse is there for a site that gets on Fast's spam list and becomes banished from the engine? Is this forever?
"This really is handled on a case by case basis. We have worked with sites that have contacted us and informed us that they have cleaned up their act. Obviously, this is very time consuming, so the combination of PartnerSite, the spam guidelines, and not having your site hosted on a banned server should enable a site to get in the index through a variety of means if they think they have cleaned up their act."Translated: If your site is hosted on one of the 20 million servers tagged for spamming, you'd best move it to a different server before you initiate the kiss-and-makeup ritual with FAST.
Baker further explained, "What really matters is the server where the submission is coming from. So many servers have been completely blacklisted due to the proliferation of spam. I suggest 'know thy neighbor.' The crawler will take care of the rest."
This again underscores the importance of your site having its own unique IP address to insure against problems caused by an unruly site sharing the same IP.
(Continued in Part 3. Click here.)
© Copyright 2002 Robin Nobles. All rights reserved.
The author assumes full responsibility for the contents of this article and retains all of its property rights. MarcommWise publishes it here with the permission of the author. MarcomWise assumes no responsibility for the article's contents.