Search engines return two sets of results, for the most part. One section, often called “Sponsored Links” are adverts paid for by the advertiser and usually payment is made each time that someone clicks on the advert. The other section is free listings. This is the list of sites that, in the opinion of the Search Engine, are most likely to satisfy your request.
There’s a whole industry of people dedicated to pushing up the rank of their client companies in the free listings, by fair means or foul. Here’s a fair means, that seems to be little understood and rarely used, even though it is free.
The major search engines offer a way to tell them about the pages in your site. The information may be as simple as telling them that you have pages, or as complex as telling the search engines how often those pages change. In the rest of this article, I describe where you can find out more about these free services.
Google has a whole suite of tools based around the “sitemap” file. You can find these services under “Google Webmaster Tools“. You’ll need a Google Account in order to use these and you’ll need to prove that you control the web site, by adding a file or a tag to a web page at the top of the site.
Google’s sitemaps let you describe how frequently web pages are updated. The tools let you see what proportion of your site is crawled and whether any probem pages have been found. It also warns you of poor practices that may lead to your site being penalised.
Yahoo!
Another authenticated service, using a Yahoo! Account, again with authentication of site ownership, Yahoo!Search Site Explorer.
Like the Google tools, this allows you to look at your own sites and see whether there are crawling and indexing problems. There’s also an API, so you can write programs to interact with the service.
