# robots.txt: generated by scripts/apply-rights.py from scripts/rights/rights.json (edit that, not this file). # Normal search engines and every other agent stay allowed (an empty Disallow allows everything). # AI training and scraping crawlers are disallowed below. The list is current at 2026-11; add a new bot to `ai_crawlers` in rights.json and re-run apply-rights.py. # Only compliant crawlers honour this file. The reservation of text-and-data-mining rights is also in # /.well-known/tdmrep.json and at /data-licence.html (EU Directive 2019/790 Art. 4(3)). User-agent: Googlebot Disallow: User-agent: Bingbot Disallow: User-agent: * Disallow: # AI crawlers (current at 2026-11) User-agent: GPTBot User-agent: ChatGPT-User User-agent: OAI-SearchBot User-agent: ClaudeBot User-agent: Claude-Web User-agent: anthropic-ai User-agent: CCBot User-agent: Google-Extended User-agent: GoogleOther User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Bytespider User-agent: Amazonbot User-agent: Applebot-Extended User-agent: Meta-ExternalAgent User-agent: FacebookBot User-agent: cohere-ai User-agent: Diffbot User-agent: ImagesiftBot User-agent: Omgilibot User-agent: Timpibot User-agent: YouBot User-agent: DuckAssistBot User-agent: AI2Bot Disallow: /