If Google etc… doesn’t find a meta robots tag it will index the webpage and follow all it’s links. You will find many websites use the code below and it indicates the webmaster doesn’t understand basic SEO because it’s the default:
<meta name="robots" content="index, follow">
I’ve been working in SEO for over 15 years, I’ve built hundreds of websites with millions of webpages, all my income is basically from websites and I NEVER use the above robots meta tag and would NEVER advise using it. It is pointless adding an index, follow meta robots tag, you DO NOT need to add it just in case!
If you care about performance SEO which is part of the Google ranking algorythm you don’t add unnecessary code just in case!
I’ve added it to this SEO tutorial for clarity, but if you go ahead and use I’m embarrassed for you!
The default index, follow code is also equal to:
<meta name="robots" content="INDEX, FOLLOW">
<meta name="robots" content="Index, Follow">
The case is ignored.
If you only include one option the default is used for the other option, so
<meta name="robots" content="index">
is equal to the default index, follow.
<meta name="robots" content="follow">
is also equal to the default index, follow.
You will also find some websites list all as an option:
<meta name="robots" content="all">
I don’t believe search engines recognize the all option, but it doesn’t matter since if a robots meta tag isn’t valid, it’s ignored and index, follow is used.
It’s no different to having a made up robots meta tag like this:
<meta name="robots" content="IndexMePlease">
Google etc… will treat the above as equal to index, follow. :-)
- meta name=”robots” content=”index, follow”
- meta name=”robots” content=”noindex, follow”
- meta name=”robots” content=”index, nofollow”
- meta name=”robots” content=”noindex, nofollow”
Continue Reading Meta Robots