django/djangoproject.com

Use noindex meta tag or header, not robots.txt, to block untranslated docs pages

Aberta

#877 aberto em 2 de mar. de 2019

 (8 comentários) (1 reação) (0 responsável)Python (1.100 forks)user submission
good first issue

Métricas do repositório

Stars
 (2.001 estrelas)
Métricas de merge de PR
 (Mesclagem média 7d 20h) (25 fundiu PRs em 30d)

Description

As far as I can tell, we're currently using robots.txt to block untranslated docs pages from being indexed by Google. Looking through Search Console, it appears that is not the correct approach and these pages are still being indexed:

That warning links to: https://support.google.com/webmasters/answer/7440203#indexed_though_blocked_by_robots_txt

Which suggests using the noindex meta tag or header instead: https://support.google.com/webmasters/answer/93710

Guia do colaborador