Hey David

Well the first reason (and probably most logical) for the reason many members area's dont get spidered is due to the .htaccess most sites have in place. No point in spidering something if nobody can get into the document 'freely'

The other thing is that may paysites (i know the Stud Money sites do) have some form of spider redirection in place such as a robots.txt file of referrer restrict which actually stops the search engine spiders going past a certain point of the site itself.

You can find a bit more information on the robots.txt protocol at the following url:

http://www.gaywidewebmasters.com/art...ext-file.shtml

Hope that answers your question for you :thumbsup:

Regards,

Lee