Robots.txt & .htaccess

Hi,

over in a German forum (http://www.pornhulk.net/) we plan to share our robots.txt and .htaccess lists for blocking IPs and Crawlers/Spiders.

Would anybody of you mind to contribute? :D

DEVELISH

I beleive there is an article in that area of the community forums about blocking leeching robots and applications, feel free to share the list of referral agents contained within it :)

Regards,

Lee

Thanks Lee,

I will share the UAs and IPs with y'all.

Over at pornhulk we are talking about building a shared database.

PLEASE contribute your robots.txt and .htaccess block-lists

DEVELISH