Saving a content type with no changes reindexes all of its content and re-sends it on the next push publish

Abierto
#38,569 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
42/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
java
Área
backend, search

Línea de trabajo

Start in ContentTypeAPIImpl.transactionalSave, where perms.resetPermissionReferences runs for existing types, then follow it into PermissionBitFactoryImpl.resetPermissionReferences and ContentTypeFactoryImpl.dbSaveUpdate. Use FieldAPIImpl.save's early return for unchanged fields as the pattern to copy. Done means an unchanged save queues no reindex entries and leaves mod_date untouched, while adding a field or changing permission inheritance still reindexes, with an integration test added to a MainSuite.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

OKR : Customer Support Team : Maintenance Type : Defect
Problem Statement

Saving an existing content type always queues every item of that type for reindexing, even when nothing in the definition changed. The same save also updates the type's mod_date, so the next push publish of any content of that type carries the content type along as a dependency. The receiving server saves it again, reindexes everything again, and passes the change on to the next environment.

In a customer's production environment, an automated CI/CD process saved the unchanged File Asset type. A routine single-file publish then carried it to production, which reindexed about 236,000 items at once. The reindex ran for about 2.5 hours and overloaded the search cluster, and uncached pages timed out for over 2 hours. Nobody chose to push the content type, and the definition hadn't changed.

Cause in code:

  • ContentTypeAPIImpl.transactionalSave (line 1185) calls perms.resetPermissionReferences(contentTypeToSave) whenever the type already exists, whether or not anything changed.
  • PermissionBitFactoryImpl.resetPermissionReferences (lines 2572-2573) then calls addStructureReindexEntries(type) for a ContentType, which queues every item of that type.
  • ContentTypeFactoryImpl.dbSaveUpdate (lines 479-481) always sets mod_date to now. Push publish compares that date when deciding whether to include the type as a dependency.
  • The push receiver (ContentTypeHandler.saveOrUpdateContentType) saves through the same typeAPI.save path, so the same thing happens on each environment along the chain.

For comparison, FieldAPIImpl.save already returns early when a field is unchanged ("No changes made to field").

Related, not in scope here: BulkProcessorListener.handleSuccess (lines 329-330) clears the whole ES/OS query cache after every successful reindex batch. That's what turned this reindex into an outage.

Steps to Reproduce
  1. On an instance with a large number of items of one content type (File Asset works well), note how many items are in the reindex queue and the type's mod_date.
  2. Save that content type without changing anything: open it in the Content Types screen and click Save, or send PUT /api/v1/contenttype/id/{idOrVar} with its current definition.
  3. Every item of that type is now queued for reindexing, and mod_date has changed.
  4. Push publish any single item of that type to a receiver that already has the type. The bundle includes the content type as a dependency, and the receiver queues all of its items of that type for reindexing too.
Acceptance Criteria
  • Saving an existing content type whose definition, fields, field variables and permission inheritance are unchanged does not queue a reindex of its content.
  • A save with no changes does not update the type's mod_date, so the next push publish doesn't include the type.
  • Changes that really affect indexing (adding a field, changing a field's indexed flag, changing permission inheritance) still queue a reindex.
  • A push-published content type that matches the receiver's copy doesn't cause a reindex on the receiver.
  • Integration test covering an unchanged save (no reindex entries, mod_date unchanged), registered in a MainSuite.
dotCMS Version

Current Evergreen release. The code path is present on main at e972c83f86.

Severity

Critical - System unusable

Links
Lenguaje dominante
Java
Estrellas
972
Forks
493
Merge medio
2 d 12 h
PR fusionados (30 d)
228

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de dotCMS/core

Todos los issues de dotCMS/core

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.