I have two php scripts which send out mail to a group and are executed by site admin from a password ptotecetd menu, but the scripts keeps getting activated by bots scarping the site.
I have tried to set a robots.txt file in the domain root as follows
User-agent: *
Disallow: /Monibot/ # This is an infinite virtual URL space
Disallow: /.index_cron.php
Disallow: /de/component/customfilters/
BrowserMatchNoCase "Monibot" bots
Order Allow,Deny
Allow from ALL
Deny from env=bots
User-agent: *
Disallow: /
The robots.txt has no effect,
Next I set the two php scripts permission to 600 with no effect.
What I want to do is turn off the whole domain to bots can anyone suggest how to do this effectively.