How do you block any web pages from the computer?

Why would I block specific Web pages from crawlers/spiders?

  • I have the option to block specific pages of my website from crawlers/spiders. Can doing this to a few pages (not the main of course) hurt my position in a search. I thought it might be a good idea to block the Contact Us page and a few others like Login and Help pages. All I could find is the info below from Go Daddy. Any thoughts? * To avoid crawlers such as spam bots trying to collect your private information for spamming purposes. * To avoid traffic overload by spiders accessing your site at high speeds or bots that index your scripts. * To protect information that you want to remain private such as addresses or contact information, printer-friendly pages that contain duplicate content, testing or experimental pages, and advertisement links.

  • Answer:

    The reasons you have mentioned are correct. It is okay to block Contact us and Login page. However, blocking Help page is not recommended because help page has lot of keywords and information that people might be looking for in search engines.

Sacred at Yahoo! Answers Visit the source

Was this solution helpful to you?

Just Added Q & A:

Find solution

For every problem there is a solution! Proved by Solucija.

  • Got an issue and looking for advice?

  • Ask Solucija to search every corner of the Web for help.

  • Get workable solutions and helpful tips in a moment.

Just ask Solucija about an issue you face and immediately get a list of ready solutions, answers and tips from other Internet users. We always provide the most suitable and complete answer to your question at the top, along with a few good alternatives below.