django/djangoproject.com

Use noindex meta tag or header, not robots.txt, to block untranslated docs pages

Open

#877 建立於 2019年3月2日

在 GitHub 查看
 (8 留言) (1 反應) (0 負責人)Python (2,001 star) (1,100 fork)user submission
good first issue

描述

As far as I can tell, we're currently using robots.txt to block untranslated docs pages from being indexed by Google. Looking through Search Console, it appears that is not the correct approach and these pages are still being indexed:

That warning links to: https://support.google.com/webmasters/answer/7440203#indexed_though_blocked_by_robots_txt

Which suggests using the noindex meta tag or header instead: https://support.google.com/webmasters/answer/93710

貢獻者指南

技術棧
djangopythonhtmlcss
領域
frontenddocumentationsearch
議題類型
bug
難度面向新貢獻者的預計實作難度，1 表示很小改動，5 表示專家級工作。
2
預計時間有經驗貢獻者完成調查、實作、測試並準備 pull request 的粗略時間範圍。
1-3 hours
活動狀態議題目前的可參與程度：新鮮、活躍、陳舊、阻塞或等待維護者輸入。
stale
清晰度議題是否清楚說明預期改動、驗收標準和下一步。
clear
前置要求
Django basicsHTML meta tagsHTTP headers
新手友善度1-100 的估計分數，表示該議題對首次貢獻者的友善程度。
70
研究方向
Investigate which Django templates or views serve the untranslated docs pages. Likely the base template or a custom middleware needs to be modified to include a noindex meta tag or X Robots Tag header. Refer to the Google Webmaster guidelines linked in the issue. Check existing templates in the repository for similar patterns.