feat: As a user, I want a Valkey config_provider for the data-plane role, so that I can run a decoupled data plane without etcd

Aperta
  • Qualcosa non va?
#13,751 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 2 giorni

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
5/5
Tempo stimato
Più di una settimana
Idoneità per principianti
45/100
Tipo di issue
Funzionalità
Chiarezza
Abbastanza chiara
Stato di attività
Tranquilla
Stack tecnologico
lua, redis
Ambito
api, backend, databases

Direzione di ricerca

Inizia leggendo i provider di configurazione esistenti in apisix/core/config_yaml.lua e apisix/core/config_xds.lua, quindi esamina la gestione dei provider in apisix/cli/schema.lua, apisix/cli/file.lua, apisix/cli/ops.lua e apisix/init.lua. Aggiungi il provider Valkey del data plane e la relativa copertura della configurazione, crea t/config-center-valkey/, aggiungi il servizio Valkey in ci/pod/docker-compose.common.yml e collega il gruppo di test in .github/workflows/build.yml; il lavoro è completato quando il percorso di lettura e watch basato su Valkey supera i relativi test di integrazione.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

enhancement
Description

As a user running Valkey/Redis-compatible infrastructure, I want to run APISIX data-plane nodes that sync their configuration from Valkey, so that I can operate a decoupled data plane without also running an etcd cluster.

Context

APISIX already supports multiple config_provider options for the data-plane role (etcd, yaml, json, xds). This proposal adds one more — valkey — following the same read-only sync pattern that json and xds established. It is purely additive: etcd remains the default and nothing about the existing providers changes.

Many teams already standardize on Valkey/Redis for their infrastructure (APISIX's own redis-backed plugins are widely used against Valkey today, e.g. #13584, #12282). Letting a data plane sync config from Valkey removes etcd as a hard dependency for those deployments.

Why Valkey can satisfy the config-provider contract

The historical concern with non-etcd config stores (see the FAQ) is the need for resumable watch streams, prefix reads, and atomic updates. Valkey now provides primitives that map cleanly onto these:

  • Change notification → Valkey Streams (XREADGROUP with consumer groups): resumable, at-least-once delivery from the last processed ID, which is exactly the "watch stream" semantic etcd provides. This is stronger than the yaml provider's 1s polling.
  • Prefix / directory reads → one Hash per collection ({apisix}:routes, {apisix}:services, ...): HGETALL returns all items in a collection atomically.
  • Revision tracking → an application-managed counter key ({apisix}:revision), INCRed on each change, used to synthesize the conf_version / modifiedIndex that core.config consumers rely on.
  • Hash-tag {apisix} keeps keys co-located on one shard so the design is Valkey-cluster-safe.

Proposed design (data-plane, read-only)

  • New provider module apisix/core/config_valkey.lua implementing the existing core.config read contract (new, get, fetch_created_obj, close, init, init_worker, plus values / values_hash / conf_version / clean_handlers and filter / checker / item_schema handling) — the same surface config_yaml.lua and config_xds.lua satisfy today.
  • Connect via the existing api7-lua-resty-redis-connector dependency (already used by the redis plugins).
  • Add "valkey" to the data-plane config_provider enum + a valkey connection schema (host, port, password, database, tls) in apisix/cli/schema.lua, with the corresponding edits in apisix/cli/file.lua, apisix/cli/ops.lua, and the config_ready_check allowlist in apisix/init.lua.
  • Config gets written to Valkey by an external process (or a Valkey-backed standalone controller); the data plane only reads and watches. Admin API writes are explicitly out of scope for this issue — this is data-plane sync only, matching how xds works.

Example configuration:

# conf/config.yaml
deployment:
  role: data_plane
  role_data_plane:
    config_provider: valkey
    valkey:
      host: "valkey.example.com"
      port: 6379
      password: "secret"
      database: 0
      tls:
        enabled: true

Testing

  • New t/config-center-valkey/ directory mirroring t/config-center-yaml/ and t/config-center-json/, using --- yaml_config blocks to set the data-plane role + config_provider: valkey.
  • Add a valkey/valkey service to ci/pod/docker-compose.common.yml and wire t/config-center-valkey into the corresponding test group in .github/workflows/build.yml.

Scope note

This is deliberately scoped to the data-plane read path — the pattern with clear precedent. Extending the option to the Admin API / control-plane write path is a larger, separate discussion (it would need a write-side storage abstraction, since the Admin API calls core.etcd directly today) and is not part of this proposal.

I'd like to gauge maintainer interest before writing code, per CONTRIBUTING.md. Does this direction sound acceptable, and are there constraints I should account for in the design? I'm happy to implement it. Thank you!

Lingua principale
Lua
Stelle
17.3k
Fork
3k
Merge medio
4g 16h
PR unite (30g)
69

Preparare l'ambiente

Apri in Codespaces

Avvia il container di sviluppo del progetto nel browser, con il tuo account GitHub.

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/apisix

Tutte le issue di apache/apisix

Issue simili

Altre issue su Lua

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.