holysoles/bot-wrangler-traefik-plugin

Bot user agents sources in different JSON structures

开放

#55 创建于 2026年3月17日

 (1 条评论) (0 个反应) (0 位负责人)Go (1 个派生)auto 404
help wanted

仓库指标

星标
 (13 个星标)
PR 合并指标
 (PR 指标待抓取)

描述

Hey! I've finally updated my plugins and I've found more interesting stuff for this project! When looking for sources, I've found these two project which also distribute bot user agents information in JSON format, but with a completely different structure from the ai.robots.txt project. These are certainly relevant to bot blocking but I'll let you judge if they are worth the effort of working on more parsing.

https://github.com/monperrus/crawler-user-agents https://github.com/arcjet/well-known-bots

贡献者指南