Search results can stay empty for up to an hour after a reindex batch because the query cache is cleared before the index refresh

Abierto
#38,549 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 2 días

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
58/100
Tipo de issue
Error
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
elasticsearch, java
Área
backend, search

Línea de trabajo

Start with ContentletIndexOperationsES.java:167, BulkProcessorListener.java:328-330, and ContentFactoryIndexOperationsES.cachedIndexSearch at line 134 to trace when the batch is acknowledged, the index refreshes, and query results are cached. Then inspect ReindexThreadTest.test_normal_pause_unpause_cycle_still_indexes_content and add an integration test that reproduces the stale empty result without clearing the cache in the test. Done means a search after a batch acknowledgment does not keep serving a result cached before the document becomes searchable.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

Team : Platform
Problem Statement

After the reindex worker indexes a batch, a content search can return no results for up to an hour even though the content is in the index.

The worker sends its bulk requests with refresh NONE (ContentletIndexOperationsES.java:167), so new documents become searchable only at the next index refresh. When the batch is acknowledged, BulkProcessorListener.handleSuccess (BulkProcessorListener.java:328-330) clears ESQueryCache and OSQueryCache. That happens before the refresh. A search made between the acknowledgement and the refresh misses the new documents, and ContentFactoryIndexOperationsES.cachedIndexSearch (:134) caches that empty result for cache.esquerycache.seconds (3600 s by default). Nothing clears the cache again until the next batch is acknowledged, so on a quiet instance the empty result is served for up to the full hour.

Steps to Reproduce
  1. Set refresh_interval to -1 on the content indices, so the window between acknowledgement and refresh stays open.
  2. Save a contentlet with the DEFER index policy and wait for the reindex queue to drain.
  3. Search for it with ContentletAPI.search (+identifier:<id>): no hits, and the empty result is cached.
  4. Force a refresh (POST /_refresh) and search again: still no hits. A read that does not use the query cache (ContentletAPI.createScrollQuery) finds the document.

This was reproduced in an integration-test run on main (59b89f6fe1); it is the cause of the intermittent ReindexThreadTest.test_normal_pause_unpause_cycle_still_indexes_content failure.

Acceptance Criteria
  • A search made after a reindex batch is acknowledged never keeps serving a result cached before the documents became searchable. Options include clearing the query caches after the refresh instead of at acknowledgement, or not caching results computed between the two.
  • An integration test covers it without clearing the cache from the test.
dotCMS Version

main (59b89f6fe1); the same code is in 26.11.09-01.

Links
  • Found while fixing CI test flakes; ReindexThreadTest now reads the index without the query cache so the test no longer depends on this behaviour.
Lenguaje dominante
Java
Estrellas
972
Forks
493
Merge medio
2 d 12 h
PR fusionados (30 d)
228

Preparar el entorno

Abrir en Codespaces

Inicia el contenedor de desarrollo del proyecto en tu navegador, con tu propia cuenta de GitHub.

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de dotCMS/core

Todos los issues de dotCMS/core

Issues similares

Más issues de Java

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.