quantum-elixir/quantum-core

multiple apps in libcluster, quantum does not identifies which node to select

Open

#605 opened on Mar 5, 2024

View on GitHub
 (2 comments) (0 reactions) (0 assignees)Elixir (2,408 stars) (153 forks)batch import
enhancementhelp wanted

Description

Hi there, I have few applications running in cluster which able to call each other Assume I have apps api-server, intergation-server I have quantum used in app intergation-server I have few nodes of api-server and few nodes intergation-server in intergation-server config, I have settings in config

config :intergation-server, IntergationServer.Scheduler,
timeout: :infinity,
  overlap: false,
  global: true,
  jobs: [
  {"1 * * * *", ... } <- Every hour 
  ...
 ]

Nodes visible in :intergation-server:

Node.self
:"app@**.**.**.**1"
iex(app@10.21.73.141)7> Node.list
[
:"app@**.**.**.**2"  # <- node of :intergation-server
:"app@**-**-**-***.api.pod.cluster.local", # <- node of :api-server
 :"app@**-**-**-***.api.pod.cluster.local", # <- node of :api-server
 :"app@**-**-**-***.api.pod.cluster.local" # <- node of :api-server
 ]

I can see job (for application intergation-server) is triggering once in few hours, I assume nodes of another application (api-server) included in when Quantum.RunStrategy.Random select node to run

Node list gets all nodes available: https://github.com/quantum-elixir/quantum-core/blob/main/lib/quantum/run_strategy/random.ex#L46

Contributor guide

Tech stack
elixir
Domain
backend
Issue type
bug
DifficultyEstimated implementation difficulty for a new contributor, from 1 for very small changes to 5 for expert-level work.
3
Estimated timeA rough time range for an experienced contributor to investigate, implement, test, and prepare a pull request.
1-3 hours
Activity statusHow available the issue appears right now: fresh, active, stale, blocked, or waiting on maintainer input.
fresh
ClarityHow clearly the issue explains the expected change, acceptance criteria, and next step.
clear
Prerequisites
Elixir basicslibcluster setupQuantum scheduling
Newbie friendlinessA 1-100 score estimating how approachable this issue is for first-time contributors.
40
Research direction
The issue points to quantum core/lib/quantum/run strategy/random.ex line 46, where `Node.list()` is used. The problem is that `Node.list` returns all nodes in the cluster, regardless of which application they run. The fix likely involves filtering nodes to only those running the same OTP application (e.g., using `:pg` groups or checking application name). Examine how Quantum registers its nodes or consider passing the node list from the scheduler's context. A simpler approach may be to compare the application name from the node's cookie or use a library like `libcluster` to scope nodes by application.