feat: per-RPC latency tracking integrated into the generated call path

Open
  • Is something wrong?
#322 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
12/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Quiet
Tech stack
go
Domain
distributed-systems, documentation

Research direction

Start with Node.Latency() and the gorums.Latency comparator, which hold the existing node-wide EWMA, and the generated quorum-call and unicast stubs where per-node response timing would be recorded. Then read the NodeResponse, ResponseSeq and Responses types to see where a per-RPC accessor could live. Done means per-RPC estimates update with no call-site changes, unmeasured RPCs return a negative sentinel, existing behaviour is unchanged, and the user guide's Measurement Limits section documents the new accessor. The issue leaves the design open, so settle the API with a maintainer before coding.

Written by the indexing model from the issue text.

Description

enhancement performance

Background

Node.Latency() maintains a single exponential moving average per node, shared across all RPC types. This is sufficient for coarse node ranking but, becomes misleading when different RPCs have substantially different server-side processing times — the estimate conflates network latency with compute time and provides no per-operation breakdown.

A user-space workaround (measuring around node.MyRPC() calls and feeding a custom SortBy closure) is impractical because the normal call site is cfg.MyRPC(), not a per-node call. There is no clean hook for users to instrument at the right level without bypassing the configuration abstraction.

Proposed change

Add integrated per-RPC latency tracking at the code-generation layer so that each generated RPC method records its own moving average alongside the existing node-wide average. Concretely:

  • Extend Node (or a new NodeStats struct) with a per-RPC latency map, keyed by RPC method name or a generated constant.
  • The generated quorum-call and unicast stubs measure elapsed time per node response and update the per-RPC map, using the same EWMA approach as the current node-wide measurement.
  • Expose the latency via the NodeResponse type, which could be accessed as part of the ResponseSeq or Responses types.
  • Another (less attractive) alternative is to expose a Node.LatencyFor(rpc string) time.Duration accessor (or a typed variant via generated code) so that SortBy closures can read per-RPC estimates.
  • SortBy(gorums.Latency) continues to use the node-wide estimate; per-RPC ordering requires a user-supplied closure.

Acceptance criteria

  • Per-RPC estimates are updated automatically with no changes required at call sites.
  • Unmeasured RPCs return a sentinel (negative duration) consistent with the existing node-wide convention, so SortBy closures can handle the cold-start case uniformly.
  • The existing Node.Latency() behavior and gorums.Latency comparator are unchanged.
  • The user guide "Measurement Limits" section is updated to note that per-RPC tracking is available and link to the relevant accessor.
Dominant language
Go
Stars
149
Forks
15
Avg merge
1d 7h
Merged PRs (30d)
46

Getting set up

This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from relab/gorums

All issues in relab/gorums

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.