Do Search Engines Index Areas that are password protected?

I am reading my SEO for dummies book and I am a little unclear as to if search engine spiders (or whatever they call them) index the pages that are in password protected folders like members area.

Any clarification would be appreciated. Thanks.

To the best of my knowledge SE bots cannot get past an .htaccess file which is the traditional way of protecting a members area's pages.

Some subscription services, such as New York Times and a few others specifically enable IP addresses for Googlebot and other search engine spiders that grant access to otherwise protected areas so that the content can be indexed.

I suspect these are special deals between Google and the information providers, because if you did this otherwise, you might be accused of cloaking and get banned.

Other than those arrangements, I agree with Tom... as far as I know, there's no way a spider can get past an .htaccess file.

Ok, that is really good to know. My book did not really cover that very well. So basically I want to spend my time optimizing the public areas of the site as best as possible, and not worry about the insides. ;o)

VERY GOOD TO KNOW.

Thanks to all for posting. You just saved me A LOT of time.

- Michael

SEO is not important for your protected area but a good quality structure with easy navigation is extremely important and helps keep members. Remember, its way easier to keep an existing member than it is to gain a new one.

cheers,
Luke

In fact, you should exclude your member's area in your robot.txt file. The last thing you want is your member's area appearing in Google's index.

If you need help with this let me know.

Michael

Quote:

---

Originally Posted by Dzinerbear

In fact, you should exclude your member's area in your robot.txt file. The last thing you want is your member's area appearing in Google's index.

If you need help with this let me know.

Michael

---

Thank you very much. I might. There is a good chapter it looks like on how to set up the robot.txt file, so when I am done with that chapter, if I am stuck, I will send you a note. Thanks for your kind offer, it will take me a few weeks to get that far. I am leaving today for Vegas for AVN, and some needed rest away from the snow. ;o)

SNOW:whip: