Thursday, March 22, 2007
How Fast is Google Crawling Your Blog?
by Paula Neal Mooney
The other day I was so excited to write about Google's new AdSense Referrals Betaprogram because I caught it right when it was announced to the public.
I published the post, pinged it on Google's blog search ping search, then searched for it thru Google Blog Search.
Nothing at first.
Pretty soon after, I received an email for my Google Alert set up on http://paulamooney.blogspot.com -- letting me know that Google had indexed my blog page.
I re-checked a Google blog search on "Adsense Referrals Beta" and there was my blog post. It made it in at the number two position for relevance, right under Darren Rowse, Mr. ProBlogger.net himself.
How did I get Google to pick up my info so fast? Besides the aforementioned publishing and pinging, I relate this to two other factors:
1 - Having a sitemap listed in Google Webmaster Tools for my blog
Visit Google Webmaster Tools if you haven't already, click on the Google Webmaster Tools (including sitemaps) link and then "Add Site".
Follow the instructions to verify your blog site and add a sitemap. Now, I must admit, it took me FOREVER to figure out how to get my blog verified properly.
Only when I read Hakuna Matata's instructions -- gee think that's his real name? -- did I finally get http://paulamooney.blogspot.com to show up with a litte '1' under the sitemap page.
My other blog address listing, http://www.paulamooney.blogspot.com is also verified and with the '1' under sitemap, but it contains errors. Only when I followed Hakuna Matata's instructions on adding my atom.xml feed as my sitemap, did I have success.
2 - Achieving a normal (instead of slower) crawl rate for Googlebot
After you've set up your blog on Google Websmaster Tools, check and see what your crawl rate is by locating the "Diagnostic" tab then clicking the Crawl Rate link on the left-hand side.
Initially, my blog was set to the dreaded Slower rate: "A slower crawl will reduce Googlebot's traffic on your server, but we may not be able to crawl your site as often."
Some time after figuring out how to get my blog properly verified and a good sitemap attached, I went back and found out I was given the option to have it crawl at the Normal, recommended crawl rate.
From what I've read, updating your blog often will help flip your crawl rate into normal mode if it's slower, along with the sitemap and verification steps described above.
This is one reason I'm posting this puppy on my dad's nice and fast and lovely computer while I'm off on a vacation-of-sorts. I love having my normal crawl rate!
Hope yours is at least normal now too...
Tags: normal crawl rate ,slower crawl rate ,choose a new crawl rate ,get indexed faster ,google crawl rate ,google indexed ,get indexed