/llms.txt reads every page on each request: no cache as for the sitemap
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 55/100
- Issue type
- Feature
- Clarity
- Clearly specified
- Activity status
- Active
- Tech stack
- javascript
- Domain
- performance
Research direction
Start by reading the existing /sitemap.xml cache path, especially hypha_getSitemapEntries() and hypha_getSitemapLastModified(), then trace the /llms.txt builder from issue #680. Add cache handling for the summary and lines in data/llms.cache, keyed by the root URL, and keep translated headings per request. Done means /llms.txt reuses cached page data and refreshes it when the cache key changes.
Written by the indexing model from the issue text.
Description
/llms.txt (#680) reads every public page on each request: hypha_getSitemapLastModified() loads the file of every text page and article, and every article is loaded a second time for its title and excerpt. /sitemap.xml reads the same files, but keeps its entries in data/sitemap.cache until hypha.xml or a page changes; /llms.txt has no cache.
A load test against a test copy of a site with about 1,200 public pages, 20 seconds at 40 concurrent requests:
| Address | Requests/s |
|---|---|
/sitemap.xml (cached) |
95 |
| a text page | 79 |
/llms.txt |
3.9 (p95 about 9 s) |
So a handful of clients asking for /llms.txt, as AI crawlers do, keeps the server busy (see also #411).
Fix: hypha_getSitemapCached() takes the cache logic of hypha_getSitemapEntries() (key: the times of hypha.xml and data/pages), for any build function. /llms.txt keeps its summary and lines in data/llms.cache, with the root URL in the key; the translated headings are added per request.
🤖 Generated with Claude Code
- Dominant language
- JavaScript
- Stars
- 5
- Forks
- 9
- Avg merge
- 18h 49m
- Merged PRs (30d)
- 13
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from PlanBCode/hypha
-
Component: User interface Impact: Low Level: Easy Pagetype: Peer reviewed article Type: Bug Value: Visitors
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Impact: Medium Level: Easy Type: Bug Value: Users
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Impact: Low Level: Easy Pagetype: Festival Type: Bug Value: Coders
Difficulty 1/5 Under an hour Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Impact: Medium Level: Easy Pagetype: Mailinglist Type: Bug Value: Users
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
Maintainers usually reply within 1 day
-
Impact: Low Level: Easy Type: Bug Value: Coders
Difficulty 1/5 1-3 hours Newbie friendliness 85/100
PlanBCode/hypha#586 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
sebastienros/jint#4449 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
remotion-dev/remotion#12957 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Ebony-Vinyl/dsh-our-free-model#759 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
RocketChat/EmbeddedChat#1416 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
yetone/magpie#5265 · 1 comment ·
Maintainers usually reply within 1 day