🎃 Generate llms.txt and llms-full.txt from the repo docs so AI agents can read them

Start with scripts/adapter-count.mjs for the --check pattern and repository-root resolution, and with .github/workflows/ci.yml lines 48-75 for the existing generated-file checks. Build scripts/generate-llms-txt.mjs to write llms.txt and llms-full.txt deterministically, then add scripts/generate-llms-txt.test.mjs. Done means the generator runs twice with no diff, --check fails after a doc edit and passes after regenerating, and the tests pass.

Part of the October Challenge #997.

Why

AI assistants and coding agents increasingly look for an llms.txt file (the llmstxt.org proposal): a short Markdown index of a project's documentation with links, plus an optional llms-full.txt with the full text in one file. An agent asked "how do I add a SOAP connector to AnythingMCP?" can then read the right page instead of guessing. This repository has about 40 Markdown documents under docs/ (connectors, client integrations, deployment, operations, API reference, the tool definition format) but no llms.txt. The anythingmcp.com/llms.txt linked from README.md:268 belongs to the website and covers the site and its guides, not these docs. AGENTS.md tells coding agents how to work on the code; llms.txt is about using and operating the product.

Self-hosted vs Cloud

• Self-hosted: no runtime change. The two files live in the repository and are read from GitHub. They are not copied into the Docker image (the runner stage copies only build output and start.sh, Dockerfile:127-146) and instances do not serve them.
• Cloud (DEPLOYMENT_MODE=cloud): no change. Do not put the files into packages/frontend/public/: proxy.ts:42 lets every path containing a dot through without sign-in, so they would be served by every self-hosted instance and by Cloud, which is not the goal.
• Who can use it: anyone reading the public repository. Nothing is per user or per organization.

What to build

1. Generator scripts/generate-llms-txt.mjs: plain Node, no new dependency, in the style of scripts/adapter-count.mjs and scripts/regenerate-catalog.mjs. It writes llms.txt and llms-full.txt at the repository root. With --check it exits 1 when the committed files differ from what it would write, like adapter-count.mjs --check.

2. llms.txt in the llmstxt.org format: # AnythingMCP, a one-paragraph > summary (reuse the description of the root package.json), then ## sections of - [Title](url): description lines:

   • Getting started: README.md, docs/deployment.md, docs/guides.md.
   • Connectors: docs/connectors/*.md, docs/tool-definition.md.
   • AI clients: docs/integrations/*.md.
   • Operations and security: docs/operations/*.md, docs/sso.md, docs/scim-entra-setup.md, docs/license-faq.md.
   • Reference: docs/api-reference.md, docs/mcp-resources.md, docs/knowledge-graph.md.
   • Contributing: .github/CONTRIBUTING.md, AGENTS.md.
   • ## Optional: docs/guides/*.md, docs/ROADMAP.md, docs/ADOPTERS.md.

   Titles come from each file's first # heading, descriptions from its first paragraph (first sentence, about 160 characters at most), with an override map in the script for files whose first paragraph is not a good summary. Links are absolute raw URLs on main (https://raw.githubusercontent.com/HelpCode-ai/anythingmcp/main/<path>), which agents can fetch as plain Markdown.

3. llms-full.txt: the same files in the same order, each introduced by a separator heading with its path and source URL. Rewrite relative links ((connectors/rest.md), (../README.md#…)) to absolute GitHub URLs, and drop images and HTML-only blocks. For README.md:1-35, keep the title and the summary sentence and drop the banner, the star-history <picture> and the badge rows. docs/ is about 300 KB today, which is fine for one file.

4. Deterministic output: sorted file lists, no timestamps, LF line endings, so --check is stable.

5. CI: one step in the backend job of .github/workflows/ci.yml, next to "Verify the quoted adapter count and license wording" (:57-58): node scripts/generate-llms-txt.mjs --check. Add scripts/generate-llms-txt.test.mjs to the node --test line that already runs scripts/validate-adapters.test.mjs (:51-52). Changes under .github/ get extra review, so keep the workflow diff to these lines.

6. Pointers: next to the website line in README.md:268, add one for the repository's own llms.txt, and the same in README.de.md, README.ja.md and README.zh-CN.md (:286). In .github/CONTRIBUTING.md and AGENTS.md, add one line: after changing docs, run node scripts/generate-llms-txt.mjs and commit the result.

Where to look

• docs/ and docs/guides.md (the current human-written index).
• scripts/adapter-count.mjs: --check pattern and repository-root resolution.
• .github/workflows/ci.yml:48-75: existing generated-file checks.
• README.md:1-35 (HTML header), :268 (website llms.txt link).
• packages/frontend/src/proxy.ts:42: why the files must not go into public/.

Acceptance criteria

• node scripts/generate-llms-txt.mjs writes both files; running it twice produces no diff.
• Every link in llms.txt points to a file that exists in the repository (checked by the test).
• llms-full.txt contains no relative Markdown links and no <picture> or <img> HTML (checked by the test).
• --check fails after a doc is edited and passes after regenerating (checked by the test).
• node --test scripts/generate-llms-txt.test.mjs passes and the CI step is green.
• README lines and contributor notes added.

Out of scope

• The website's anythingmcp.com/llms.txt, which is maintained with the website.
• content/guides/ (website guides in several languages, synced by scripts/sync-guides-to-website.sh) and the translated READMEs.
• Serving the files from the app, or exposing the docs as an MCP resource.

Size

M (1–2 days)

How to claim

Comment "I'd like to work on this" and we'll assign you. Rules in #997.