CRAWLERS


Positioning your website on the first page…

of search results happens from the actions of "search engine indexing robots" - also known as "Crawlers", "Robots" or "Spiders." A Crawler is an automated program, a simple browser to some degree, that "reads" your website.

Based upon a number of criteria, these programs build an index based upon the predominant content of your site. While the criteria used varies and many of the parameters are constantly changine - the level of experience gained since the early 1990s has made these Crawlers fairly good… but far from perfect. Part of the difficulty lies with the complexity of the job they are attempting (literally, artificial intelligence) - and the abuse, misuse and misleading actions of some "less than professional" web marketing firms that exploit known anomalies to gain website traffic.

When creating a search optimized website…

it is always important to keep certain things in mind.

First, recognize that search engines have a mission! They are designed (and constantly refined) to provide good service to the user. They do this by retrieving those websites most relevant to the information, products or services the user is seeking.

By succeeding at this mission, the user has a good experience with the search engine and, thus, is more likely to return and keep using that search engine. For the search engine, this translates to more traffic, happier advertisers and increased revenue for both the search engine AND their customers (advertisers).

When you launch a search for specific information, the search engine returns a list of candidate websites it feels are most relevant to what you are seeking. These "search results" are shown, in order, by most relevant to least relevant (to your search phrase(s)). What makes a search engine decide who goes near the top of the list… and who goes last? We would suggest that you bring this point up when you contact one of our search placement counselors. But for the moment - the BIG criteria are the quality and validity of the content, placement of the content, correct use of buried (invisible) tags, the length of time the site has been in existence, link popularity of the site - and the quality and quantity of those links. There are hundreds of other criteria as well - but much is based upon these primary website search optimizing criteria.



Call (949) 307-3013 for a 'no obligation' consultation...




Crawling...
Surface Crawl - eventually happens automatically for most websites. Looks at < TITLE > tag and first thousand characters, or so, of the website Home page.

Deep Crawl - Usually a "paid for" service... but not always. Looks deeper (5,000+ char) into every page on the website. Does cross table verifications, looks for external referring links in other related websites, etc.