Hiding Text From SE Crawlers

Much attention is given on how to get text into search engine indexes. Yet, there are good reasons why certain text should not be indexed by SE crawlers.

Why hide selective words and phrases from spiders and crawlers?

Perhaps you wrote an exposé Your web page may benefit by obfuscating the words

For another example, a product review page would not want context-sensitive ads displayed for the product being reviewed. The ads may take the click away from the affiliate link. Obfuscating certain key words can make that less likely.

Consider:

1. Page rank. A web page is less likely to be deemed a bad neighborhood without bad neighborhood words.

2. Keyword ranking. The positions and repetitions of key words and phrases may be used to automatically establish what a web page is about. Too many key words of an unrelated subject may cause the scanning software to assign an incorrect subject to the web page.

   Example: An article about the Amish that goes on and on about their horses might incorrectly be listed as being about horses instead of about the Amish. Obfuscating most "horse" words and words naming horse breeds reduces the likelihood of that incorrect assignment.

3. AdSense and other context ad delivery services. These services generally deliver ads compatible with the subject of the web page. The "Keyword ranking" example above also fits here.

4. Search engine results pages. SERPs contain the URLs of pages with the words being searched for. By hiding specific key words and phrases from SE crawlers, a web page is less likely to be in the results pages for those words.

   Example: If it is undesirable to be found when someone searches for from SE crawlers.

When specific key words and phrases are hidden from crawlers and spiders, the hidden text is unavailable for indexing, ad delivery, keyword ranking, or any other purpose.

How To Hide Text From Search Engines

JavaScript can be used to make key text disappear for spiders and crawlers. It's nowhere in the source code. Yet, the text is visible for humans in the browser window and can be selected just like regular text.

Spiders and crawlers that don't scan JavaScript for content won't see any of the JavaScript code. The rest will see only a base64-encoded version of the text, not the text humans see in their browser.

For example, the text "" is "V2lsbCBCb250cmFnZXI=" in the source code.

Here is a JavaScript text obfuscation generator. It generates JavaScript to replace the key words or phrases to be obfuscated.

How To Use The Content Obfuscation Generator

1. Type the word or phrase to be obfuscated into the generator's top text box.

2. Click the button.

3. Copy the JavaScript from the bottom text box and paste it into the web page in place of the words being obfuscated.

The obfuscation JavaScript will print the words on the web page. Yet, the words are not available in the source code.

Only whole words should be generated. If a word is preceded or followed by punctuation marks, obfuscate the punctuation, also.

The reason is the script tag causes a space to be rendered by the browser. Obfuscating partial words would insert a space within the word.

The JavaScript

Considering that crawlers and spiders are getting better and better, obfuscating words by printing them with document.write() may now or soon be insufficient protection.

Using the JavaScript provided by the Content Obfuscation Generator can remove the obfuscated words entirely from the source code. What is not there is not seen.

For browsers that might have JavaScript turned off, the generated obfuscation JavaScript can be followed immediately with:

<noscript>[turn JavaScript on]</noscript>

You now have the means control which words and phrases you do not want SE crawlers to index.

Will Bontrager