From c75c8e0cc252eaa3c55654b7988748ffd64aeaa1 Mon Sep 17 00:00:00 2001 From: Alex 'AdUser' Z Date: Sun, 18 Feb 2018 12:20:25 +1000 Subject: [PATCH] * ban noisy bot --- theme/site/robots.txt.ep | 3 +++ 1 file changed, 3 insertions(+) diff --git a/theme/site/robots.txt.ep b/theme/site/robots.txt.ep index 52c2953..937705d 100644 --- a/theme/site/robots.txt.ep +++ b/theme/site/robots.txt.ep @@ -11,5 +11,8 @@ Disallow: * User-Agent: MJ12bot Disallow: * +User-Agent: SemrushBot +Disallow: * + User-agent: WebIndex Disallow: *