Add canonical links to Markdown page copies

Open Beginner friendly
  • Is something wrong?
#512 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
73/100
Issue type
Feature
Clarity
Clearly specified
Activity status
Active
Tech stack
django, python
Domain
backend, search

Research direction

Start by reading apps/llms_txt/mixins.py around line 46 and inspect markdown_response to see how it builds the response headers. Check how the /markdown/ copies and llms-full.txt are served, then compare their headers with the proposed behavior. Done when Markdown copies send a canonical link to their HTML pages and llms-full.txt sends X-Robots-Tag: noindex.

Written by the indexing model from the issue text.

Description

Problem statement

/markdown/ page copies return 200 with no canonical or robots directive. They appeared in search results in place of the HTML page, and so did llms-full.txt. The site has 152 of these URLs.

Proposed solution

Add a Link: <html-url>; rel="canonical" response header in markdown_response (apps/llms_txt/mixins.py:46). This keeps the copies available to agents. Use X-Robots-Tag: noindex on llms-full.txt, which has no single HTML equivalent.

Alternative solutions

X-Robots-Tag: noindex on the Markdown routes. A canonical is a better fit, as it passes signals to the HTML page.

Additional context

Search results came from web search result lists with no confirmed rank order. Related: #486, #498.

Found in an AI-assisted SEO audit of the live site (Claude Code, 2026-10-30), reviewed by @thibaudcolas.

Working on this

Contribute to the discussion to help this move forward. Explain your use cases, discuss possible solutions, research how to adapt existing code, or compatibility concerns.

View our contributing guidelines. Add a comment to the issue to explain what approach you are taking.

Dominant language
Python
Stars
36
Forks
44
Avg merge
2d 22h
Merged PRs (30d)
9

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from wagtail/guide

All issues in wagtail/guide

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.