Creating Robots.txt File in a WordPress Blog
Robots.txt is a way through which search engines can identify which pages of your blog are allowed to be spidered or indexed. It is different from meta name=”robots” tag, which tell search engines or robots only about those individual pages on which these tags are placed. There is a huge difference between these two.
You can find tons of information about the robots.txt file at RobotsTXT.Org, if interested or else I’ll be explaining it here in simple terms.
General rule of thumb is to have your robots.txt file at the root of your domain i.e., domainname.com/robots.txt.
A perfect example of WordPress blog robots.txt file is as below
Check What Matt Cutts is Saying about Robots.Txt File
Share This Article
Get Free Email Alerts
And, Next Confirm Your Email Subcription