-
Life is a dick and when itīs get hard---just fuck it...
Robots.txt & .htaccess
Hi,
over in a German forum (http://www.pornhulk.net/) we plan to share our robots.txt and .htaccess lists for blocking IPs and Crawlers/Spiders.
Would anybody of you mind to contribute?
DEVELISH
-
You do realize by 'gay' I mean a man who has sex with other men?
I beleive there is an article in that area of the community forums about blocking leeching robots and applications, feel free to share the list of referral agents contained within it
Regards,
Lee
-
Life is a dick and when itīs get hard---just fuck it...
Thanks Lee,
I will share the UAs and IPs with y'all.
Over at pornhulk we are talking about building a shared database.
PLEASE contribute your robots.txt and .htaccess block-lists
DEVELISH
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
Bookmarks