Block bad bots

Recently my website has been crawling to a stop with 508 errors.

And then i realised there’s a stupid bot keep crawling my website non-stop

And that stupid bot is called ‘Semrush’ and below is the access log entry - - [05/Dec/2016:16:07:55 +0800] "GET /xyz.html HTTP/1.1" 301 - "-" "Mozilla/5.0 (compatible; SemrushBot/1.1~bl; +"

And to block this bot all you have to do is to add the following entries to your .htaccess file

BrowserMatchNoCase Semrush bad_bot

Order Deny,Allow
Deny from env=bad_bot

deny from 46.229.164.
deny from 46.229.168.

The above codes will block both browser agent and as well as IP address

Similar Posts:

VN:F [1.9.22_1171]
Rating: 6.2/10 (9 votes cast)
Block bad bots, 6.2 out of 10 based on 9 ratings