Home  Find Movie Maker Transitions, Titles & Credits, Transitions and more  Multimedia Download Center Home
 Free Multimedia File Downloads Index
Multimedia Resources Home
Free Birthday Songs / Music
Free Background Image Tiles
Free Sound Effects
Free TV Theme Songs
Free Computer Wallpapers
Free Icons Collections
Windows Movie Maker
Free Computer Clipart Images
Free Website Templates
Free Fonts Downloads
Mega-Paks File Archives
Free Website Page Graphics
Articles, Tutorials & Trivia
Free Cool Tools Online
Free Midi Karaoke Songs Free MIDI Karaoke Songs

 Nifter Media Group
Nifter Media on YouTube
Casey Anthony Case Videos
Link to Nifter.com

  Free Online Masters Courses
WAHM - Work at Home Mom
Webmaster Business Course
NetWriting Course
Service Business / Sellers
Local Business Course





 Video, Movie Title Screen Backgrounds - Movie Maker and others



 See the Breakup Transitions Pack Preview Video



 Preview the BrightRight Effects Pack here...



 Our most exciting package yet - Cool Zoom Effects

Robots Meta Tag Element - Attributes, Uses and Other Information

Home > Free Multimedia Download Center > Information, Articles, Tutorials > The Robots Meta Tag Element




The Robots Meta Tag Element - Metadata Elements (Meta Tags)



Before jumping into this article it would only be purdent to mention that there is an alternative to using the Robots Meta Tag, which must be added to every web page it is necessary to use it on. The alternative is to use a robots text file (robots.txt) We recommend you look over the Wikipedia article on the Robots Text File if you want to learn more. Robots.txt at Wikipedia can be found here.

Not everyone has permission to access their web server and upload a robots.txt file. The use of a free web site host is an example of where this can happen. There may be other reasons where it is a good idea to use the Robots Meta Tag also, regardless of whether you are using the robots.txt file or not. In any case, this article will help you understand more about this metadata element, an important bit of knowledge for any webmaster.

About the Robots Meta Tag
The Robots Meta Tag is a useful element whose purpose is to disallow specific actions by web robots and spiders. For example, you wouldn't want a page that users have paid a fee to access appearing on Google search results pages. And even when you have disallowed indexing of the page, you still don't want a robot following links on the page that lead to other members-only areas. The Robots Meta Tag will help you control this type of access plus other actions that are commonly performed by web robots.
The Robots Meta Tag is written like this with no regard to letter case:
<meta name="robots" content="ATTRIBUTES_HERE">
A comma delimited list of the terms you choose to use goes in the area that we have labelled ATTRIBUTES_HERE in the above example. Common terms for the Robots Meta Tag include the following:
all   index   noindex   follow   nofollow
Other terms can be added as part of the comma delimited (separated) list as shown in the table at the end of this article.

    Explanation of Terms / Attributes
    The word service (or services) is used here to signify the organization(s) using the robot that is searching the web page. Examples: Google, Bing, Yahoo etc.

  • Term: all
    Use: <meta name="robots" content="all">
    Action: All services may index the page. (Include in their database.)

  • Term: index
    Use: <meta name="robots" content="index">
    Action: The web page may be indexed by the service.

  • Term: noindex
    Use: <meta name="robots" content="noindex">
    Action: The web page may not be indexed by the service.

  • Term: follow
    Use: <meta name="robots" content="follow">
    Action: Links on the page may be followed by the robot.

  • Term: nofollow
    Use: <meta name="robots" content="nofollow">
    Action: Links on the page may not be followed by the robot.

Don't Spend Time for Nothing
The default action for a robot is to index each page it finds and follow all links on that pages to find more pages. If no Robots Meta Tag exists, the default action is taken. Note that the default action is identical to using the Robots Meta Tag like this:
<meta name="robots" content="index, follow">
In other words, using index or follow provides no different result than if you didn't use them at all. Many people include one of these terms in robots tags that are using both, but it isn't necessary. In fact, if you want a robot to handle a web page in the default manner, you don't need to include the Robots Meta Tag at all. Take a look at this example:
The tag
     <meta name="robots" content="index, nofollow">
will have the same effect as this tag:
     <meta name="robots" content="nofollow">
The default action (index, follow) is taken when any of the following conditions are found:
  • There is no content on the web page.
  • The Robots Tag is missing.
  • No terms are specified. Example: <meta name="robots" content="">
  • The term all is found within the comma delimited list.
    When the term "all" is found, it automatically over-rides any other terms that have been included. For example, "noindex, all, nofollow" will be handled the same way as "all" would be when used as a solitary attribute. Here's the same example written in full:

    This Meta Tag, with "all" included in the comma delimited list
         <meta name="robots" content="noindex, all, nofollow">
    has the same effect as:
         <meta name="robots" content="all">

Contradictory attributes allow the robot to disregard the Robots Meta Tag and handle the page in whatever manner it pleases (however it has been programmed to handle this situation.) An example would be the following:
<meta name="robots" content="noindex, follow, noindex">

Using the Robots Meta Tag Properly
We will end this section of the article with some more examples of the Robots Meta Tag used in ways that conform to the standard for this tag. An explanation of the actions that would be taken by the robot follow. We have also included comparison tags to help you complete your understanding of this tag.
  1. <meta name="robots" content="noindex">
    The robot may not index this page. Since no other terms exist, they are assummed to be allowed. For example, all links on the page will be followed because the nofollow attribute is not present.

  2. <meta name="robots" content="nofollow">
    The robot may index this page but may not follow links from this page.

  1. <meta name="robots" content="noindex, follow">
    This tag has the same effect as #1 above.

  2. <meta name="robots" content="index, noindex, nofollow">
    This tag has the same effect as #2 above.

As we mentioned earlier it is quite common for people to write the Robots Meta Tag with two terms included, even though one of them represents the default action of robots.
Example: <meta name="robots" content="noindex, follow">
(Follow is a default action and therefore not necessary to include.)
This is often done because it gives the coder a clear picture of the actions he being allowed and/or disallowed. How you put your Robots Meta Tags together is up to you; just remember to keep the format proper and do not duplicate attributes. Doing so will ensure that most web robots respect your wishes. Not all, but the important ones will, and that's what counts.



Robots Meta Tag Elements - Which are Used by Who?
This table provides a list of Robots Meta Tag elements and the search services that include them in their indexing process.

Note that all of these services use the main robots tag elements, noindex, nofollow and noarchive, a less common but useful attribute in specific situations.

Attribute Googlebot Slurp MSNBot Teoma
NoIndex Yes Yes Yes Yes
NoFollow Yes Yes Yes Yes
NoArchive Yes Yes Yes Yes
NoSnippet Yes No No No
NoODP Yes Yes Yes No
NoYDIR No Yes No No
NoImageIndex Yes No No No
NoTranslate Yes No No No
Unavailable_After Yes No No No


Free website content, articles, tutorials, facts and trivia Articles, Information and Tutorials - Free Website Content
Download from a selection of free computer file downloads Free Multimedia Download Center


 Articles, Information and Tutorials Index



 Click for banners.  Help us provide free downloads by adding a link on your website




Partners    Privacy Policy    About    Top of Page    Search    Contact    Home

Download Center    Nifter's Nuts



 Link to or exchange links with Nifter.com



 Top Rated Web Hosting - Unlimited Websites, Bandwith, Email, Subdomains...



 This Web Site Rated and Approved by SafeSurf



   Nifter Media Group  Home Copyright © 2009 - Nifter.com - All Rights Reserved