Serve a robots.txt that references the sitemap
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 58/100
- Issue type
- Feature
- Clarity
- Mostly clear
- Activity status
- Active
- Tech stack
- javascript
- Domain
- backend
Research direction
Start by locating how Hypha serves generated files and how test mode is detected; the issue does not name those entry points or tests. Review the sitemap work in #375 and test-mode behavior in #404. Done means robots.txt points to the generated sitemap, disallows system and edit URLs, and disallows all crawling in test mode.
Written by the indexing model from the issue text.
Description
There is no proper robots.txt that points crawlers to the sitemap. Combined with the missing sitemap (see #375), indexing of new articles takes weeks.
Proposal
- Generate
robots.txtwith aSitemap:line pointing to the generated sitemap - Disallow system and edit URLs
- Disallow everything in test mode (see #404)
Related: #375, #404
Origin: meeting on the future of Hypha (De Stadsbron).
- Dominant language
- JavaScript
- Stars
- 5
- Forks
- 9
- Avg merge
- 18h 49m
- Merged PRs (30d)
- 13
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from PlanBCode/hypha
-
Component: User interface Impact: Low Level: Easy Pagetype: Peer reviewed article Type: Bug Value: Visitors
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
Maintainers usually reply within 1 day
-
Impact: Medium Level: Easy Type: Bug Value: Users
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Impact: Low Level: Easy Pagetype: Festival Type: Bug Value: Coders
Difficulty 1/5 Under an hour Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Impact: Medium Level: Easy Pagetype: Mailinglist Type: Bug Value: Users
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
Maintainers usually reply within 1 day
-
Impact: Low Level: Easy Type: Bug Value: Coders
Difficulty 1/5 1-3 hours Newbie friendliness 85/100
PlanBCode/hypha#586 · 1 comment ·
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
Maintainers usually reply within 1 day
-
help wanted Idea 💡 wdio-cli
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
webdriverio/webdriverio#16361 · 2 comments ·
Maintainers usually reply within 1 day
-
clientV2: Application Info header separators show as missing-character boxes without a CJK fontOpenbug new-client
Difficulty 1/5 1-3 hours Newbie friendliness 88/100
NUWCDIVNPT/stig-manager#2285 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
Fei2411/DeepSeek-Bot#922 · 1 reaction ·