Don't Block Search Engine Crawlers

edit

More articles

 By Michael Cottam on January 4, 2011        

edit Make sure your important pages can rank

---

A great page on your website will never rank well in search results - and be discovered by more people - if the search engines don't know it's there in the first place.

First, some background: Search engines use computer programs called "crawlers" (or “spiders” or “robots”) to discover web pages and add them to their indexes. Google calls its crawler “Googlebot,” and lots of SEO (search engine optimization) professionals call it that, too.

Read the rest of this article on AboutUs.com blog.