Sitemaps in robots.txt File

Give your carefully created sitemap a mention in your robots.txt file. You simply use this format:
Sitemap: http://www.mysite.com/sitemap.xml
You can repeat this line many times, if you have many sitemaps (eg, a video sitemap as well as your page sitemap).

The main advantage of this specification is that all sorts of bots will now know about all of your pages. You may have verified your site with online tools (such as Google Webmaster Tools) and specified your sitemaps there, but now everyone can know about them!

For more information, see stiemaps.org.


Comments

It's quiet in here...Add your comment

Related Reading

Web Development Survey!
Which web technologies would you like to see the back of, and why?