The Great Search Engine Hypocrisy

The clearest, most consistent opinion voiced by the major search engines to webmasters is this…

Show the bots what you show your users.

It is a fairly simple proposition that has been applied to the vast majority of grey and black hat search technologies: cloaking, ip-delivery, doorway pages, and keyword stuffing. Essentially every type of search spam, even link spam tangentially, relates to this premise.

So let’s look at the latest incarnations of search engine hypocrisy waged by both Google and Yahoo.

1. The NoFollow Tag.

Originally, webmasters were held accountable for each and every outbound link. The industry even developed a name for the unfavorable sites which webmasters should avoid linking: “bad neighborhoods”. While websites were a target for spammers trying to promote “bad neighborhood” sites (as they still are today, despite growing nofollow use), the principle remained in tact: show the bots what you show your users.

The NoFollow tag grossly violated this long held principle: empowering the webmaster to tag content that will be openly displayed to users without regard, but flagged for the bots as untrustworthy. Rather than preventing sites which display untrustworthy/unsavory links from being promoted via high rankings, these sites are now exempted by using a tag that the unsuspecting site visitor never sees. There is little reason for a user to believe that the results in Google will lead to a page with trustworthy links.

Subsequently, what has occurred is exactly what the search marketing industry had predicted. Massive user-generated sites have begun using NoFollow well beyond its original intents, marking all external links using the rel=nofollow tag.

2. The NoContent Tag.

Yahoo has recently proposed a NoContent Tag. It is the logical extension of NoFollow. If we are going to allow webmasters to present links in a different way to the bots than to users, why not allow them to do the same with content?

Some have responded that this is not cloaking because “Yahoo will still see your entire page”. However, “Yahoo! will not use the terms contained in that section as information for finding the page or for the abstract of that page in search results.” And even though Yahoo! explains that it is not cloaking, there really is no other word for it.

In actuality, it is cloaking in reverse. Rather than showing the bots extra content, webmasters show the users extra content. While perhaps not to the level of sneaky redirecting practices and IP Delivery, the conclusion is straight-forward.

Webmasters using NoFollow and Robots-NoContent are sending conflicting messages about the trustworthiness of their content to the bots and to the users. NoFollow, and soon NoContent, is being abused as a way for webmasters to get away with showing untrustworthy content.

The Solution.

The crazy thing about it all is that solutions to link spam do exist which do not violate this principle. Rather than promoting or developing this type of technology, the Search Engines create quick fixes that hurt the User. A simple combination of the tools below is more than enough.

  1. Akismet
  2. LinkSleeve
  3. Captchas
  4. Admin Moderation
  5. Community Moderation

We use Akismet + Linksleeve + Admin Moderation. Effective as of this morning, TheGoogleCache dropped the nofollow tag from all of its comments. Why should we use it? We hand-approve every comment that makes it onto our pages. If we don’t trust the link, we won’t post the comment.

No tags for this post.


  1. Scott G
    May 5, 2007

    great article, the hypocrisy is getting out of control and the Webmaster Guidelines for each search engine should be considered gray, and not black and white, seeing this is how search engines are choosing to implement them.

    I too removed ‘no-follow’ from my comments and am extremely disappointed about Yahoo’s new implementation of the ‘no-content’ attribute. Grossly irresponsible if you ask me!

    Search engines need to stop trying to help us code our sites, or prevent spam… it doesn’t work. Solutions mentioned above have helped out more.

  2. Detoam
    May 22, 2007

    I agree with both of You. I don’t use the NOfollow as I approve each and every link on my sites and directories. What I am wondering though is what are search engines themselves doing to improve. So far i have heard nothing but whining from Google and Yahoo. Which in turn proves to me their own inability to handle it. So they pass the buck onto us. If they are so concerned about spam and bad neighbourhoods why aren’t they doing anything “real” to help stop that.

Submit a Comment

Your email address will not be published. Required fields are marked *