Feature Request: Implement a Self-Healing Sync Queue (Journal) for transient connection failures

Open
  • Is something wrong?
#4,226 2 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

@faisalahammad is already working on this.

Since Aug 7, 2026.

  • #4349 by @faisalahammad — open

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Stale
Tech stack
elasticsearch, php
Domain
backend, search

Research direction

Start by tracing ElasticPress's existing read-fallback logic and the write-operation entry points for post inserts, updates, and deletes. Done means transient connection failures persist operations in a journal and a recurring WP-Cron job retries them until the Elasticsearch index is synchronized.

Written by the indexing model from the issue text.

Description

enhancement large
Is your enhancement related to a problem? Please describe.

Hello,

Thank you for your feedback on issue #4224.

Following up on that, I'd like to suggest a feature to make the plugin more robust when the ES server is temporarily unreachable. This would solve the root problem of #4224 and replace the manual snippet suggested there. That snippet feels like a workaround rather than a robust solution, and it becomes tedious to run manually across many sites.

Here's my idea: ElasticPress should keep a persistent journal (or queue) for write operations that fail due to connection issues.

This can happen for many reasons:

  • A planned reboot of the ES/OpenSearch server for updates.
  • A brief, intermittent network interruption.
  • Transient DNS lookup failures or routing problems on the WordPress server's side.

I am aware that ES/OpenSearch is designed for high availability via clusters. However, this doesn't account for the client-side network (the WordPress server). As you can imagine, that network's reliability is an unknown, and intermittent issues are often the hardest to debug.

My proposal is to extend the existing read-fallback logic (using MySQL for search) to write operations as well:

  1. When a post is to be inserted, updated, or deleted, EP attempts the operation.
  2. If the connection to the ES server fails (e.g., timeout, connection refused, 503 error), EP should not discard this operation.
  3. Instead, it should add the operation (e.g., {action: 'delete', post_id: 123}) to a persistent journal (e.g., a custom table or a persistent option).
  4. A recurring WP-Cron job then checks (e.g., every 5 minutes) if the ES server is reachable and if the journal contains pending jobs.
  5. If both are true, it processes the jobs from the journal, retrying them until they succeed, effectively making the sync "self-healing."

This would make the plugin significantly more robust and provide a much better Quality-of-Life experience for all users: self-hosters and elasticpress.io customers alike.

While your own hosting is no doubt highly available, you cannot control the reliability of your customers' WordPress server networks. This feature would transparently handle those intermittent issues, likely reducing support tickets related to "out-of-sync" indexes caused by temporary network blips.

Thank you for considering this proposal and for your great work on the plugin!

Designs

No response

Describe alternatives you've considered

No response

Code of Conduct
  • I agree to follow this project's Code of Conduct
Dominant language
PHP
Stars
1.3k
Forks
325
Avg merge
2d 11h
Merged PRs (30d)
24

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from 10up/ElasticPress

All issues in 10up/ElasticPress

Similar issues

More PHP issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.