[BYO-S3] local-dev: the S3 endpoint baked into warehouse storage profiles goes stale on every network change

Open
#8,569 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Active
Tech stack
docker-compose

Research direction

Start with the lakekeeper-init note in bin/local-dev/docker-compose.override.yml and the REST-catalog behavior noted in IcebergUtil.createRestCatalog. Reproduce a local-dev network change, then evaluate endpoint options for both the Lakekeeper container and host-native JVMs. Done means new and existing warehouse storage profiles remain usable after the host address changes without manual reconfiguration.

Written by the indexing model from the issue text.

Description

Problem

On local-dev, STORAGE_S3_ENDPOINT has two audiences at once: the Lakekeeper container dials it for S3 operations and bakes it into every warehouse's storage profile, and the host-native JVMs then receive that same string back at runtime — the REST catalog vends the S3 settings to clients (see the note in IcebergUtil.createRestCatalog).

The wrapper script satisfies both today by sniffing the host's LAN IP at startup (see the lakekeeper-init note in bin/local-dev/docker-compose.override.yml). That address is only valid until DHCP hands out a new one. After a network change:

  • creating a warehouse fails — the running services still carry the old address from boot, and Lakekeeper's storage validation cannot reach it;
  • existing warehouses keep the old address persisted in their storage profiles, so reads and writes fail even after a restart;
  • recovery is manual every time: re-edit the config, restart the services, and re-issue POST /management/v1/warehouse/{id}/storage for every existing warehouse.

This has repeatedly bitten local BYO-S3 testing (#6870).

Observations so far
  • localhost satisfies only the host JVMs; it is unreachable from the Lakekeeper container.
  • Container DNS (http://texera-rustfs:9000) satisfies only the compose network; the catalog vends it to host JVMs, which cannot resolve it — the caveat already documented in that override file.
  • host.docker.internal was tried at one point and did not work end to end. Two isolated observations from this round: a container on the compose network could open a TCP connection to host.docker.internal:9000, while macOS itself does not resolve the name (so anything vended to a host process would fail there). No end-to-end validation of any alternative has been done.

Part of #6870.

Dominant language
Scala
Stars
316
Forks
189
Avg merge
2d 20h
Merged PRs (30d)
198

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from apache/texera

All issues in apache/texera

Similar issues

More Scala issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.