diff options
| author | Markus Heiser <markus.heiser@darmarit.de> | 2023-04-30 09:49:26 +0200 |
|---|---|---|
| committer | Markus Heiser <markus.heiser@darmarit.de> | 2023-04-30 09:49:26 +0200 |
| commit | 823c490c84a04447a2bc0198d0b168c6b81d6ce2 (patch) | |
| tree | 2699b9f42a2db0577a4ea1c919e01f4c1af5c24c /searx | |
| parent | 9967fbb7d14170f443c6cfb6938d444ecff3f442 (diff) | |
[mod] limiter: block requests from PetalBot
Block requests from PetalBlock. Normally robots.txt is enough to stop
PetalBlock from making requests [1]. However, if SearXNG is offered below a
path (example.org/search), then the robots.txt is not available in the root
paths of the domain / subdomain.
[1] https://webmaster.petalsearch.com/site/petalbot
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Diffstat (limited to 'searx')
| -rw-r--r-- | searx/plugins/limiter.py | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/searx/plugins/limiter.py b/searx/plugins/limiter.py index 72be3391..46c82f58 100644 --- a/searx/plugins/limiter.py +++ b/searx/plugins/limiter.py @@ -37,6 +37,7 @@ block_user_agent = re.compile( # unmaintained Farside instances + r'|' + re.escape(r'Mozilla/5.0 (compatible; Farside/0.1.0; +https://farside.link)') + + '|.*PetalBot.*' + r')' ) |