Search Engine Robots (Bots)

Spiders, also known as a crawlers or robots, are the part of a search engine that locate and index every page on the Internet that the webpage is a possible answer to a searcher’s query. Successful [[search engine optimization]] depends on crawlers finding many or all a website's pages. The most prolific and important spiders come from Google: googlebot, Yahoo: slurp, and MSN: msnbot. Bots find websites and put them into their index by going through links from other websites. Because of this, it's not necessary to submit to search engines, which is a common mistake.