django/djangoproject.com

Use noindex meta tag or header, not robots.txt, to block untranslated docs pages

Open

#877 创建于 2019年3月2日

在 GitHub 查看
 (8 评论) (1 反应) (0 负责人)Python (2,001 star) (1,100 fork)user submission
good first issue

描述

As far as I can tell, we're currently using robots.txt to block untranslated docs pages from being indexed by Google. Looking through Search Console, it appears that is not the correct approach and these pages are still being indexed:

That warning links to: https://support.google.com/webmasters/answer/7440203#indexed_though_blocked_by_robots_txt

Which suggests using the noindex meta tag or header instead: https://support.google.com/webmasters/answer/93710

贡献者指南

技术栈
djangopythonhtmlcss
领域
frontenddocumentationsearch
议题类型
bug
难度面向新贡献者的预计实现难度，1 表示很小改动，5 表示专家级工作。
2
预计时间有经验贡献者完成调查、实现、测试并准备 pull request 的粗略时间范围。
1-3 hours
活动状态议题当前的可参与程度：新鲜、活跃、陈旧、阻塞或等待维护者输入。
stale
清晰度议题是否清楚说明期望改动、验收标准和下一步。
clear
前置要求
Django basicsHTML meta tagsHTTP headers
新手友好度1-100 的估计分数，表示该议题对首次贡献者的友好程度。
70
研究方向
Investigate which Django templates or views serve the untranslated docs pages. Likely the base template or a custom middleware needs to be modified to include a noindex meta tag or X Robots Tag header. Refer to the Google Webmaster guidelines linked in the issue. Check existing templates in the repository for similar patterns.