Saving a content type with no changes reindexes all of its content and re-sends it on the next push publish
I maintainer di solito rispondono entro 2 giorni
Nessuno ha ancora preso questa issue.
Valutazione
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Idoneità per principianti
- 42/100
Direzione di ricerca
Start in ContentTypeAPIImpl.transactionalSave, where perms.resetPermissionReferences runs for existing types, then follow it into PermissionBitFactoryImpl.resetPermissionReferences and ContentTypeFactoryImpl.dbSaveUpdate. Use FieldAPIImpl.save's early return for unchanged fields as the pattern to copy. Done means an unchanged save queues no reindex entries and leaves mod_date untouched, while adding a field or changing permission inheritance still reindexes, with an integration test added to a MainSuite.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Descrizione
Problem Statement
Saving an existing content type always queues every item of that type for reindexing, even when nothing in the definition changed. The same save also updates the type's mod_date, so the next push publish of any content of that type carries the content type along as a dependency. The receiving server saves it again, reindexes everything again, and passes the change on to the next environment.
In a customer's production environment, an automated CI/CD process saved the unchanged File Asset type. A routine single-file publish then carried it to production, which reindexed about 236,000 items at once. The reindex ran for about 2.5 hours and overloaded the search cluster, and uncached pages timed out for over 2 hours. Nobody chose to push the content type, and the definition hadn't changed.
Cause in code:
ContentTypeAPIImpl.transactionalSave(line 1185) callsperms.resetPermissionReferences(contentTypeToSave)whenever the type already exists, whether or not anything changed.PermissionBitFactoryImpl.resetPermissionReferences(lines 2572-2573) then callsaddStructureReindexEntries(type)for aContentType, which queues every item of that type.ContentTypeFactoryImpl.dbSaveUpdate(lines 479-481) always setsmod_dateto now. Push publish compares that date when deciding whether to include the type as a dependency.- The push receiver (
ContentTypeHandler.saveOrUpdateContentType) saves through the sametypeAPI.savepath, so the same thing happens on each environment along the chain.
For comparison, FieldAPIImpl.save already returns early when a field is unchanged ("No changes made to field").
Related, not in scope here: BulkProcessorListener.handleSuccess (lines 329-330) clears the whole ES/OS query cache after every successful reindex batch. That's what turned this reindex into an outage.
Steps to Reproduce
- On an instance with a large number of items of one content type (File Asset works well), note how many items are in the reindex queue and the type's
mod_date. - Save that content type without changing anything: open it in the Content Types screen and click Save, or send
PUT /api/v1/contenttype/id/{idOrVar}with its current definition. - Every item of that type is now queued for reindexing, and
mod_datehas changed. - Push publish any single item of that type to a receiver that already has the type. The bundle includes the content type as a dependency, and the receiver queues all of its items of that type for reindexing too.
Acceptance Criteria
- Saving an existing content type whose definition, fields, field variables and permission inheritance are unchanged does not queue a reindex of its content.
- A save with no changes does not update the type's
mod_date, so the next push publish doesn't include the type. - Changes that really affect indexing (adding a field, changing a field's indexed flag, changing permission inheritance) still queue a reindex.
- A push-published content type that matches the receiver's copy doesn't cause a reindex on the receiver.
- Integration test covering an unchanged save (no reindex entries,
mod_dateunchanged), registered in a MainSuite.
dotCMS Version
Current Evergreen release. The code path is present on main at e972c83f86.
Severity
Critical - System unusable
Links
- Lingua principale
- Java
- Stelle
- 972
- Fork
- 493
- Merge medio
- 2g 12h
- PR unite (30g)
- 228
Preparare l'ambiente
Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.
- Nessun Dockerfile né file Docker Compose
- Ha un modello di pull request
- Leggi la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Altre issue di dotCMS/core
-
dotCMS : UVE OKR : Customer Support Team : Maintenance Type : Defect
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 2 giorni
-
OKR : Customer Support Team : Maintenance Type : Defect
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 2 giorni
-
Opening content from Content Search replaces the "Search" breadcrumb in the new Edit Content screenApertadotCMS: New Edit Contentlet OKR : Customer Support Team : Falcon Type : Defect
Difficoltà 2/5 1-3 ore Idoneità per principianti 78/100
I maintainer di solito rispondono entro 2 giorni
-
Escape the Binary field hint in the legacy content editor so an apostrophe doesn't break the fieldApertaOKR : Customer Support Team : Maintenance Type : Defect
Difficoltà 2/5 1-3 ore Idoneità per principianti 79/100
I maintainer di solito rispondono entro 2 giorni
-
dotCMS : Permissions OKR : Customer Support Team : Maintenance Type : Defect
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 2 giorni
Issue simili
-
enhancement
Difficoltà 2/5 1-3 ore Idoneità per principianti 70/100
it-at-m/Wahllokalsystem#3846 ·
I maintainer di solito rispondono entro 2 giorni
-
Area/Connector module/solace Priority/Low Type/Bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 77/100
ballerina-platform/ballerina-library#9849 ·
I maintainer di solito rispondono entro 1 giorno
-
Difficoltà 1/5 Meno di un'ora Idoneità per principianti 65/100
Vakore/ZappierGames#327 ·
-
bug
Difficoltà 2/5 1-3 ore Idoneità per principianti 82/100
I maintainer di solito rispondono entro 1 giorno
-
Radix number block generates NaN after switching a negative or decimal number to binary, octal or hexForse già presa @matthewalberto l’ha presa oggi. Aperta
Difficoltà 2/5 1-3 ore Idoneità per principianti 64/100
mit-cml/appinventor-sources#4246 · 1 commento ·
I maintainer di solito rispondono entro 1 giorno