Yes, Robots.txt can be ignored by bots and yes, it's not secure: everyone could see the content of this file entering http://www.yourwebsite.com/robots.txt. Nevertheless, well-considered robots.txt helps to deliver your relevant content to search engine bots and omit low-priority pages in SERP. At first glance giving directories is an easy task but any kind of management requires your attention and forethought. Let’s go through the most common Robots.txt mistakes and find out how to establish a constructive dialogue with Googlebot.
Winter Spring is Coming!
Recently Google Webmaster Central Office announced a series of SEO updates with a hashtag #springiscomning in Twitter. News about the new way Google will treat pagination affected SEO community greatly. Let the dust settle and get to the bottom of this news.