Saving a content type with no changes reindexes all of its content and re-sends it on the next push publish
Los mantenedores suelen responder en 2 días
Nadie ha tomado este issue todavía.
Evaluación
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Aptitud para principiantes
- 42/100
Línea de trabajo
Start in ContentTypeAPIImpl.transactionalSave, where perms.resetPermissionReferences runs for existing types, then follow it into PermissionBitFactoryImpl.resetPermissionReferences and ContentTypeFactoryImpl.dbSaveUpdate. Use FieldAPIImpl.save's early return for unchanged fields as the pattern to copy. Done means an unchanged save queues no reindex entries and leaves mod_date untouched, while adding a field or changing permission inheritance still reindexes, with an integration test added to a MainSuite.
Escrito por el modelo de indexación a partir del texto del issue.
Descripción
Problem Statement
Saving an existing content type always queues every item of that type for reindexing, even when nothing in the definition changed. The same save also updates the type's mod_date, so the next push publish of any content of that type carries the content type along as a dependency. The receiving server saves it again, reindexes everything again, and passes the change on to the next environment.
In a customer's production environment, an automated CI/CD process saved the unchanged File Asset type. A routine single-file publish then carried it to production, which reindexed about 236,000 items at once. The reindex ran for about 2.5 hours and overloaded the search cluster, and uncached pages timed out for over 2 hours. Nobody chose to push the content type, and the definition hadn't changed.
Cause in code:
ContentTypeAPIImpl.transactionalSave(line 1185) callsperms.resetPermissionReferences(contentTypeToSave)whenever the type already exists, whether or not anything changed.PermissionBitFactoryImpl.resetPermissionReferences(lines 2572-2573) then callsaddStructureReindexEntries(type)for aContentType, which queues every item of that type.ContentTypeFactoryImpl.dbSaveUpdate(lines 479-481) always setsmod_dateto now. Push publish compares that date when deciding whether to include the type as a dependency.- The push receiver (
ContentTypeHandler.saveOrUpdateContentType) saves through the sametypeAPI.savepath, so the same thing happens on each environment along the chain.
For comparison, FieldAPIImpl.save already returns early when a field is unchanged ("No changes made to field").
Related, not in scope here: BulkProcessorListener.handleSuccess (lines 329-330) clears the whole ES/OS query cache after every successful reindex batch. That's what turned this reindex into an outage.
Steps to Reproduce
- On an instance with a large number of items of one content type (File Asset works well), note how many items are in the reindex queue and the type's
mod_date. - Save that content type without changing anything: open it in the Content Types screen and click Save, or send
PUT /api/v1/contenttype/id/{idOrVar}with its current definition. - Every item of that type is now queued for reindexing, and
mod_datehas changed. - Push publish any single item of that type to a receiver that already has the type. The bundle includes the content type as a dependency, and the receiver queues all of its items of that type for reindexing too.
Acceptance Criteria
- Saving an existing content type whose definition, fields, field variables and permission inheritance are unchanged does not queue a reindex of its content.
- A save with no changes does not update the type's
mod_date, so the next push publish doesn't include the type. - Changes that really affect indexing (adding a field, changing a field's indexed flag, changing permission inheritance) still queue a reindex.
- A push-published content type that matches the receiver's copy doesn't cause a reindex on the receiver.
- Integration test covering an unchanged save (no reindex entries,
mod_dateunchanged), registered in a MainSuite.
dotCMS Version
Current Evergreen release. The code path is present on main at e972c83f86.
Severity
Critical - System unusable
Links
- Lenguaje dominante
- Java
- Estrellas
- 972
- Forks
- 493
- Merge medio
- 2 d 12 h
- PR fusionados (30 d)
- 228
Preparar el entorno
Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.
- Sin Dockerfile ni archivo de Docker Compose
- Tiene una plantilla de pull request
- Leer la guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Más de dotCMS/core
-
dotCMS : UVE OKR : Customer Support Team : Maintenance Type : Defect
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Los mantenedores suelen responder en 2 días
-
OKR : Customer Support Team : Maintenance Type : Defect
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 2 días
-
Opening content from Content Search replaces the "Search" breadcrumb in the new Edit Content screenAbiertodotCMS: New Edit Contentlet OKR : Customer Support Team : Falcon Type : Defect
Dificultad 2/5 1-3 horas Aptitud para principiantes 78/100
Los mantenedores suelen responder en 2 días
-
Escape the Binary field hint in the legacy content editor so an apostrophe doesn't break the fieldAbiertoOKR : Customer Support Team : Maintenance Type : Defect
Dificultad 2/5 1-3 horas Aptitud para principiantes 79/100
Los mantenedores suelen responder en 2 días
-
dotCMS : Permissions OKR : Customer Support Team : Maintenance Type : Defect
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Los mantenedores suelen responder en 2 días
Todos los issues de dotCMS/core
Issues similares
-
enhancement
Dificultad 2/5 1-3 horas Aptitud para principiantes 70/100
it-at-m/Wahllokalsystem#3846 ·
Los mantenedores suelen responder en 2 días
-
Area/Connector module/solace Priority/Low Type/Bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 77/100
ballerina-platform/ballerina-library#9849 ·
Los mantenedores suelen responder en 1 día
-
Dificultad 1/5 Menos de una hora Aptitud para principiantes 65/100
Vakore/ZappierGames#327 ·
-
bug
Dificultad 2/5 1-3 horas Aptitud para principiantes 82/100
Los mantenedores suelen responder en 1 día
-
Radix number block generates NaN after switching a negative or decimal number to binary, octal or hexPosiblemente ocupada @matthewalberto la tomó hoy. Abierto
Dificultad 2/5 1-3 horas Aptitud para principiantes 64/100
mit-cml/appinventor-sources#4246 · 1 comentario ·
Los mantenedores suelen responder en 1 día