Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

Use Redis Sentinel for Caching/Async Queries

Open
  • Is something wrong?
#783 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
55/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
kubernetes, redis
Domain
backend, distributed-systems

Research direction

Start by tracing the existing Redis and Celery configuration introduced by PR 724, including the superset_config.py override and CeleryConfig entry points. Compare it with RedisSentinelCache and the sentinel:// workaround shown here; done means Redis Sentinel can be configured for both caching and async-query broker/results without manual overrides, including the master name.

Written by the indexing model from the issue text.

Description

customer-request

Description

The PR https://github.com/stackabletech/superset-operator/pull/724 added celery worker support using Redis following the standard Superset example: https://superset.apache.org/admin-docs/configuration/async-queries-celery/

  • This only works for RedisCache not RedisSentinelCache
  • The schema used is always redis:// instead of sentinel://
  • The master-name cannot be configured for sentinel nodes

So Redis in HA mode is not available without overrides (not tested).

Possible untested fix

configOverrides:
  superset_config.py:
    FILE_FOOTER: |
      from flask_caching.backends.rediscache import RedisSentinelCache
      _SENTINELS = [("redis-sentinel-0.redis-sentinel-headless", 26379),
                    ("redis-sentinel-1.redis-sentinel-headless", 26379),
                    ("redis-sentinel-2.redis-sentinel-headless", 26379)]
      _MASTER = "mymaster"
      _PW = os.environ["CELERY_RESULTS_BACKEND_DATABASE_PASSWORD"]

      RESULTS_BACKEND = RedisSentinelCache(
          sentinels=_SENTINELS, master=_MASTER, password=_PW, db=0,
          key_prefix="superset_results",
          # sentinel_password=..., only if the sentinels themselves require auth
      )

      _URL = ";".join(f"sentinel://:{_PW}@{h}:{p}" for h, p in _SENTINELS) + "/0"
      class CeleryConfig(object):
          broker_url = _URL
          result_backend = _URL
          broker_transport_options = {"master_name": _MASTER}
          result_backend_transport_options = {"master_name": _MASTER}
          imports = ("superset.sql_lab", "superset.tasks.scheduler")
          worker_prefetch_multiplier = 10
          task_acks_late = True
          task_annotations = {"sql_lab.get_sql_results": {"rate_limit": "100/s"}}
      CELERY_CONFIG = CeleryConfig

This is just a workaround and will need adaptation whenever the redis replicas change.

Issue checklist

This is a simple checklist of things to bear in mind when creating a new issue.

  • Describe the use-case: As far as possible, use the pattern "As a [type of user], I would like [feature/functionality] to be able to do [specific action]." This helps identify the feature and the problem it addresses.
  • Indicate importance and urgency: Use a scale (e.g., low, medium, high) to indicate the level of importance and urgency.
  • Work-around: If there is a known work-around, describe it briefly.
  • Environment: Describe the environment where the issue occurs (e.g., SDP version, K8S version, etc.).
Dominant language
Rust
Stars
35
Forks
4
Avg merge
12h 20m
Merged PRs (30d)
10

Getting set up

We have not checked this project's setup files yet. Start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from stackabletech/superset-operator

All issues in stackabletech/superset-operator

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.