django/djangoproject.com

Use noindex meta tag or header, not robots.txt, to block untranslated docs pages

Open

Aperta il 2 mar 2019

Vedi su GitHub
 (8 commenti) (1 reazione) (0 assegnatari)Python (2001 star) (1100 fork)user submission
good first issue

Descrizione

As far as I can tell, we're currently using robots.txt to block untranslated docs pages from being indexed by Google. Looking through Search Console, it appears that is not the correct approach and these pages are still being indexed:

That warning links to: https://support.google.com/webmasters/answer/7440203#indexed_though_blocked_by_robots_txt

Which suggests using the noindex meta tag or header instead: https://support.google.com/webmasters/answer/93710

Guida contributor

Tech stack
djangopythonhtmlcss
Dominio
frontenddocumentationsearch
Tipo issue
bug
DifficoltàQuanto dovrebbe essere impegnativa per un nuovo contributor.
2
Tempo stimatoTempo stimato per completare e verificare uno scope piccolo.
1-3 hours
Stato attivitàQuanto è probabile che la issue sia ancora attiva e reviewable.
stale
ChiarezzaQuanto chiaramente la issue descrive problema, scope e risultato atteso.
clear
Prerequisiti
Django basicsHTML meta tagsHTTP headers
Adatta ai principiantiQuanto la issue è adatta a contributor alla prima esperienza.
70
Direzione di ricerca
Investigate which Django templates or views serve the untranslated docs pages. Likely the base template or a custom middleware needs to be modified to include a noindex meta tag or X Robots Tag header. Refer to the Google Webmaster guidelines linked in the issue. Check existing templates in the repository for similar patterns.