Hacktoberfest 2026: the issues maintainers tagged for October, open and beginner-friendly. Browse Hacktoberfest issues

rivetkit 3.0: HTTP 400 overloaded on node pool upsert is treated as fatal and exits the process

Open Beginner friendly
  • Is something wrong?
#5,933 1 comment 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
78/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
rust
Domain
backend, distributed-systems

Research direction

Read rivetkit-rust/packages/rivetkit-core/src/registry/runner_config.rs, especially response_error, and trace how its retryable result reaches the existing retry loop. Treat response body code overloaded as retryable, then check the registry startup tests that reproduce overloaded node pool upserts; done when startup retries this response instead of exiting.

Written by the indexing model from the issue text.

Description

Summary

In rivetkit 3.0.0-preview.3, a registry that starts against a local engine exits the process when the engine answers the node pool upsert with HTTP 400 overloaded. The engine marks that answer as retryable ("retry after 100 ms"), but the client treats it as fatal.

Where

rivetkit-rust/packages/rivetkit-core/src/registry/runner_config.rs, response_error. It classifies only 404, 429, 5xx and body code not_found as retryable. The engine answers a warming or busy storage host with:

400 Bad Request {"code":"overloaded","group":"depot","message":"The storage host refused the command (routing_unavailable); retry after 100 ms."}
400 Bad Request {"code":"overloaded","group":"depot","message":"The storage host refused the command (command_capacity); retry after 100 ms."}

That error goes to ensure_local_normal_runner_config, then to serveRegistry, and rivetkit/dist/tsup/mod.js calls process.exit(1) on "runtime registry serve errored".

Reproduction

Single local engine, rivet-engine 3.0.0-preview.3 on macOS arm64, file-system object storage, storage.auto_provision_partitions=1, storage.balancing.enabled=false (the same settings as engine_env in rivetkit-rust/packages/engine-process).

Send PUT /pools/default?namespace=default with body {"drain_on_version_upgrade":true} every 100 ms:

  • From engine start until about 1.7 s, every call gets overloaded (routing_unavailable).
  • With 1 call at a time after that, every call succeeds.
  • With 8 parallel calls, each round has one success and the other calls get overloaded (command_capacity), for as long as the loop runs.

So a registry that starts in the first ~1.7 s after the engine, or at the same time as another registry against the same engine, ends its process. In our test suite this failed 6 of 23 tests in one file.

Expected

Treat body code overloaded as AttemptError::Retryable, so the existing 60 x 500 ms retry loop covers it.

Workaround

Before registry.startAndWait(), we send the same PUT ourselves, retry it on overloaded, and start one registry at a time in the process.

Dominant language
Rust
Stars
6.5k
Forks
291
Avg merge
14h 38m
Merged PRs (30d)
38

Getting set up

  • No Dockerfile or Docker Compose file
  • Has a pull request template
  • No contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from rivet-dev/rivet

All issues in rivet-dev/rivet

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.