Saving a content type with no changes reindexes all of its content and re-sends it on the next push publish

Aperta
#38,569 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
4/5
Tempo stimato
3-5 giorni
Idoneità per principianti
42/100
Tipo di issue
Bug
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
java
Ambito
backend, search

Direzione di ricerca

Start in ContentTypeAPIImpl.transactionalSave, where perms.resetPermissionReferences runs for existing types, then follow it into PermissionBitFactoryImpl.resetPermissionReferences and ContentTypeFactoryImpl.dbSaveUpdate. Use FieldAPIImpl.save's early return for unchanged fields as the pattern to copy. Done means an unchanged save queues no reindex entries and leaves mod_date untouched, while adding a field or changing permission inheritance still reindexes, with an integration test added to a MainSuite.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

OKR : Customer Support Team : Maintenance Type : Defect
Problem Statement

Saving an existing content type always queues every item of that type for reindexing, even when nothing in the definition changed. The same save also updates the type's mod_date, so the next push publish of any content of that type carries the content type along as a dependency. The receiving server saves it again, reindexes everything again, and passes the change on to the next environment.

In a customer's production environment, an automated CI/CD process saved the unchanged File Asset type. A routine single-file publish then carried it to production, which reindexed about 236,000 items at once. The reindex ran for about 2.5 hours and overloaded the search cluster, and uncached pages timed out for over 2 hours. Nobody chose to push the content type, and the definition hadn't changed.

Cause in code:

  • ContentTypeAPIImpl.transactionalSave (line 1185) calls perms.resetPermissionReferences(contentTypeToSave) whenever the type already exists, whether or not anything changed.
  • PermissionBitFactoryImpl.resetPermissionReferences (lines 2572-2573) then calls addStructureReindexEntries(type) for a ContentType, which queues every item of that type.
  • ContentTypeFactoryImpl.dbSaveUpdate (lines 479-481) always sets mod_date to now. Push publish compares that date when deciding whether to include the type as a dependency.
  • The push receiver (ContentTypeHandler.saveOrUpdateContentType) saves through the same typeAPI.save path, so the same thing happens on each environment along the chain.

For comparison, FieldAPIImpl.save already returns early when a field is unchanged ("No changes made to field").

Related, not in scope here: BulkProcessorListener.handleSuccess (lines 329-330) clears the whole ES/OS query cache after every successful reindex batch. That's what turned this reindex into an outage.

Steps to Reproduce
  1. On an instance with a large number of items of one content type (File Asset works well), note how many items are in the reindex queue and the type's mod_date.
  2. Save that content type without changing anything: open it in the Content Types screen and click Save, or send PUT /api/v1/contenttype/id/{idOrVar} with its current definition.
  3. Every item of that type is now queued for reindexing, and mod_date has changed.
  4. Push publish any single item of that type to a receiver that already has the type. The bundle includes the content type as a dependency, and the receiver queues all of its items of that type for reindexing too.
Acceptance Criteria
  • Saving an existing content type whose definition, fields, field variables and permission inheritance are unchanged does not queue a reindex of its content.
  • A save with no changes does not update the type's mod_date, so the next push publish doesn't include the type.
  • Changes that really affect indexing (adding a field, changing a field's indexed flag, changing permission inheritance) still queue a reindex.
  • A push-published content type that matches the receiver's copy doesn't cause a reindex on the receiver.
  • Integration test covering an unchanged save (no reindex entries, mod_date unchanged), registered in a MainSuite.
dotCMS Version

Current Evergreen release. The code path is present on main at e972c83f86.

Severity

Critical - System unusable

Links
Lingua principale
Java
Stelle
972
Fork
493
Merge medio
2g 12h
PR unite (30g)
228

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di dotCMS/core

Tutte le issue di dotCMS/core

Issue simili

Altre issue su Java

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.