Search results can stay empty for up to an hour after a reindex batch because the query cache is cleared before the index refresh

未关闭
#38,549 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

维护者通常 2 天内回复

还没有人认领这个 Issue。

评估

难度
4/5
预计耗时
3-5 天
新手友好度
58/100
Issue 类型
缺陷
描述清晰度
描述清楚
活跃度
活跃
技术栈
elasticsearch, java
领域
backend, search

调研方向

Start with ContentletIndexOperationsES.java:167, BulkProcessorListener.java:328-330, and ContentFactoryIndexOperationsES.cachedIndexSearch at line 134 to trace when the batch is acknowledged, the index refreshes, and query results are cached. Then inspect ReindexThreadTest.test_normal_pause_unpause_cycle_still_indexes_content and add an integration test that reproduces the stale empty result without clearing the cache in the test. Done means a search after a batch acknowledgment does not keep serving a result cached before the document becomes searchable.

由索引模型根据 Issue 内容生成。

描述

Team : Platform
Problem Statement

After the reindex worker indexes a batch, a content search can return no results for up to an hour even though the content is in the index.

The worker sends its bulk requests with refresh NONE (ContentletIndexOperationsES.java:167), so new documents become searchable only at the next index refresh. When the batch is acknowledged, BulkProcessorListener.handleSuccess (BulkProcessorListener.java:328-330) clears ESQueryCache and OSQueryCache. That happens before the refresh. A search made between the acknowledgement and the refresh misses the new documents, and ContentFactoryIndexOperationsES.cachedIndexSearch (:134) caches that empty result for cache.esquerycache.seconds (3600 s by default). Nothing clears the cache again until the next batch is acknowledged, so on a quiet instance the empty result is served for up to the full hour.

Steps to Reproduce
  1. Set refresh_interval to -1 on the content indices, so the window between acknowledgement and refresh stays open.
  2. Save a contentlet with the DEFER index policy and wait for the reindex queue to drain.
  3. Search for it with ContentletAPI.search (+identifier:<id>): no hits, and the empty result is cached.
  4. Force a refresh (POST /_refresh) and search again: still no hits. A read that does not use the query cache (ContentletAPI.createScrollQuery) finds the document.

This was reproduced in an integration-test run on main (59b89f6fe1); it is the cause of the intermittent ReindexThreadTest.test_normal_pause_unpause_cycle_still_indexes_content failure.

Acceptance Criteria
  • A search made after a reindex batch is acknowledged never keeps serving a result cached before the documents became searchable. Options include clearing the query caches after the refresh instead of at acknowledgement, or not caching results computed between the two.
  • An integration test covers it without clearing the cache from the test.
dotCMS Version

main (59b89f6fe1); the same code is in 26.11.09-01.

Links
  • Found while fixing CI test flakes; ReindexThreadTest now reads the index without the query cache so the test no longer depends on this behaviour.
主要语言
Java
星标
972
派生
493
平均合并
2 天 12 小时
30 天内合并 PR
228

环境准备

在 Codespaces 中打开

在浏览器里用你自己的 GitHub 账号启动这个项目的开发容器。

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

dotCMS/core 的其他 Issue

查看 dotCMS/core 的全部 Issue

相似的 Issue

更多 Java Issue

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。