Multilingual sites: which language is `/llms.txt`, and how do agents find the others?

Start by reading the existing v2 subpath and link-notation sections, then review the cited OpenCode webfetch implementation for the language-handling context. Add the three proposed notes about per-language indexes, root-index discovery, and hreflang links without introducing new filenames or discovery mechanisms.

Summary

v2 formalizes placing an llms.txt at any subpath, covering the pages under that path, with agents using the most specific file where more than one applies. That rule already makes /ja/llms.txt a natural fit on sites with language-prefixed paths. No new mechanism is needed.

What is still undefined is the language dimension itself:

1. Which language should the root /llms.txt of a multilingual site be written in?
2. How does an agent that starts at the root learn that other language versions exist?
3. How should the language of a target be expressed on rel="describedby" and rel="alternate" links?

This issue proposes short notes on top of what already exists. It does not propose a new filename, a new notation, or a new discovery mechanism.

Today: implementations diverge

With nothing specified, the decision rests with each site, and in practice sites differ.

• One bilingual (Japanese/English) developer documentation site serves a single /llms.txt at the root; /ja/llms.txt returns 404. The index lists only English page URLs, even though per-page Markdown versions exist in both languages. The Japanese content is there, but the index doesn't reach it.
• Another site places its index at a locale-bearing path (of the form https://example.com/<locale>/llms.txt). The file's own text — its headings and the per-link descriptions — is nonetheless in English.

Neither violates the spec. With nothing specified, both choices are reasonable. From an agent's point of view, though, neither the language of an index nor the location of other language versions can be known in advance.

The consumption side hard-codes the language too

The webfetch tool of the open-source coding agent OpenCode builds its request headers like this:

packages/core/src/tool/webfetch.ts (commit SHA: https://github.com/anomalyco/opencode/blob/941e71dbbb94ea5b32226c2845585992dadb361f/packages/opencode/src/tool/webfetch.ts )

const acceptHeader = (format: Format) => {
  switch (format) {
    case "markdown":
      return "text/markdown;q=1.0, text/x-markdown;q=0.9, text/plain;q=0.8, text/html;q=0.7, */*;q=0.1"
    case "text":
      return "text/plain;q=1.0, text/markdown;q=0.9, text/html;q=0.8, */*;q=0.1"
    case "html":
      return "text/html;q=1.0, application/xhtml+xml;q=0.9, text/plain;q=0.8, text/markdown;q=0.7, */*;q=0.1"
  }
  return "*/*"
}

const headers = (format: Format, userAgent: string) => ({
  "User-Agent": userAgent,
  Accept: acceptHeader(format),
  "Accept-Language": "en-US,en;q=0.9",
})

Accept (the format dimension) is selectable by the caller, while Accept-Language (the language dimension) is fixed to en-US with no way to change it. Even when the user is asking in Japanese and the target is Japanese documentation, English is always requested.

With no guidance to follow, defaulting to one's own primary language seems a natural choice for an implementer. (This too is a single implementation observed at a single point in time, and I don't claim it generalizes.)

Proposal

Three short notes, each sitting on top of an existing mechanism.

1. Say that the subpath rule covers per-language indexes.

A single sentence noting that sites with language-prefixed paths can place a per-language index, e.g. /ja/llms.txt. This already follows directly from v2's "covers the pages under its path, most specific wins" rule — but because it isn't stated, implementers aren't doing it.

2. Give the root index a path to the other languages.

For the root /llms.txt of a multilingual site, recommend either (a) including links to the per-language indexes, or (b) using per-language H2 sections. (a) needs only the existing link notation; (b) is handled by existing parsers as-is.

3. Mention that links can carry a language.

Links in a Link: header can express the language of the target via the hreflang attribute (RFC 8288). It would be enough to show that this works for both describedby and alternate:

Link: </ja/llms.txt>; rel="describedby"; hreflang="ja", </en/llms.txt>; rel="describedby"; hreflang="en"

Notes

• I'm not proposing a filename change, a new discovery mechanism, or making Accept-Language negotiation mandatory. (Negotiation has known operational drawbacks — cache complexity, uneven CDN support — so combining it with path-based separation seems more realistic.)
• This relates to the point raised in #79. A comment there suggests link alternate would solve it (referencing #86), but alternate corresponds to the Markdown version of a page, and does not apply to the site-level index that v2 assigns to describedby. Because the index functions as an entry point to a site, the effect of its language being undefined remains separate from the per-page case.

Offer

If there's agreement on the direction, I'm happy to open a PR for the relevant section. As a user of bilingual documentation sites, I can also help gather implementation examples and feedback from the Japanese-speaking developer community.