django/djangoproject.com
Use noindex meta tag or header, not robots.txt, to block untranslated docs pages
開放
#877 建立於 2019年3月2日
good first issue
倉庫指標
- 星標
- (2,001 顆星)
- PR 合併指標
- (平均合併 7天 20小時) (30 天內合併 25 個 PR)
描述
As far as I can tell, we're currently using robots.txt to block untranslated docs pages from being indexed by Google. Looking through Search Console, it appears that is not the correct approach and these pages are still being indexed:
That warning links to: https://support.google.com/webmasters/answer/7440203#indexed_though_blocked_by_robots_txt
Which suggests using the noindex meta tag or header instead: https://support.google.com/webmasters/answer/93710