Routing.Type=custom with any DHT router: sweep provider is a NoopProvider, `provide once` reports "queued" but announces nothing

Open
#11,482 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go
Domain
backend, cli

Research direction

Start with core/node/provider.go and core/node/libp2p/routing.go, then compare the provider selection and typed-nil DHT behavior described in the report with fixes #11112 and #10975. Reproduce with the supplied three custom routing configurations and fake HTTP router. Done means the default sweep configuration announces the CID through the configured provider, or provide once reports an error when no provider can announce it.

Written by the indexing model from the issue text.

Description

Checklist
  • This is a bug report, not a question.
  • I have searched the issue tracker.
  • I am running the latest kubo version (reproduced identically on 0.43.0 and on the 0.43.1 release binary; core/node/provider.go is unchanged on master since v0.43.1 apart from a kad-dht bump).
Description

#11112 fixed Routing.Type=custom with only HTTP routers by falling back to the LegacyProvider. The same failure is still there as soon as the custom config lists any DHT router, even one that Methods.provide never references:

  1. hasDHTRouting() sees the DHT router, so the default Provide.DHT.SweepEnabled=true picks the SweepingProvider.
  2. In custom mode the Composer is neither a *ddht.DHT nor exposes Routers(), so BaseRouting emits a typed-nil *ddht.DHT as dhtc (core/node/libp2p/routing.go).
  3. SweepingProviderOpt fails its inDht != nil check and returns NoopProvider (core/node/provider.go).
  4. ipfs provide once prints queued 1 CID(s) for immediate provide and exits 0, but nothing is announced anywhere: not to the HTTP routers behind Methods.provide, and not to the DHT. The reprovide cycle is a no-op too.

Setting Provide.DHT.SweepEnabled=false brings the announcements back, which is why this is easy to miss: the config that works today (HTTP-only) silently stops providing the moment someone adds a DHT router for find-peers.

Reproduction

A fake /routing/v1 HTTP router logs every PUT /routing/v1/providers/. Three custom configs, all with Methods.provide pointing at the HTTP router, then ipfs provide once <cid> and an 8s wait:

Case Routers Provide.DHT.SweepEnabled provide once output PUTs containing the CID
A HTTP only default queued 1 CID(s), exit 0 2
B HTTP + DHT (find-peers → DHT) default queued 1 CID(s), exit 0 0
C HTTP + DHT (find-peers → DHT) false queued 1 CID(s), exit 0 2

Routing for case B:

{
  "Type": "custom",
  "Routers": {
    "HttpRouter": { "Type": "http", "Parameters": { "Endpoint": "http://127.0.0.1:<port>" } },
    "DhtRouter": { "Type": "dht", "Parameters": { "Mode": "client", "PublicIPNetwork": true } }
  },
  "Methods": {
    "provide": { "RouterName": "HttpRouter" },
    "find-providers": { "RouterName": "HttpRouter" },
    "find-peers": { "RouterName": "DhtRouter" },
    "get-ipns": { "RouterName": "HttpRouter" },
    "put-ipns": { "RouterName": "HttpRouter" }
  }
}
Full repro script (bash + node, uses a throwaway repo per case)
#!/usr/bin/env bash
# Repro: Routing.Type=custom + a DHT router listed + default sweep provider => provide once is a silent no-op.
# Runs kubo against a local fake /routing/v1 HTTP router that logs every PUT it receives.
set -u
IPFS_BIN=${IPFS_BIN:?set IPFS_BIN}
WORK=$(cd "$(dirname "$0")" && pwd)/run
rm -rf "$WORK"; mkdir -p "$WORK"
ROUTER_PORT=$((40000 + RANDOM % 5000))

# Fake delegated router: logs PUT /routing/v1/providers bodies, answers empty results.
node -e '
const http = require("http"); const fs = require("fs");
http.createServer((req, res) => {
  let body = ""; req.on("data", c => body += c); req.on("end", () => {
    fs.appendFileSync(process.argv[2], `${req.method} ${req.url} ${body.slice(0, 4000)}\n`);
    res.setHeader("Content-Type", "application/json");
    res.end(req.method === "PUT" ? "{\"ProvideResults\":[]}" : "{\"Providers\":[]}");
  });
}).listen(Number(process.argv[1]), "127.0.0.1");
' "$ROUTER_PORT" "$WORK/router.log" &
ROUTER_PID=$!
trap 'kill $ROUTER_PID 2>/dev/null' EXIT
sleep 1

run_case() {
  local name=$1 routing=$2 sweep=$3
  export IPFS_PATH="$WORK/$name"
  local api=$((45000 + RANDOM % 5000)) swarm=$((50000 + RANDOM % 5000))
  "$IPFS_BIN" init --profile test >/dev/null 2>&1
  "$IPFS_BIN" config --json Bootstrap '[]'
  "$IPFS_BIN" config Addresses.API "/ip4/127.0.0.1/tcp/$api"
  "$IPFS_BIN" config Addresses.Gateway "/ip4/127.0.0.1/tcp/$((api + 1))"
  "$IPFS_BIN" config --json Addresses.Swarm "[\"/ip4/127.0.0.1/tcp/$swarm\"]"
  "$IPFS_BIN" config --json Routing "$routing"
  [ -n "$sweep" ] && "$IPFS_BIN" config --json Provide.DHT.SweepEnabled "$sweep"
  "$IPFS_BIN" daemon >"$WORK/$name.daemon.log" 2>&1 &
  local dpid=$!
  for _ in $(seq 60); do curl -s -X POST "http://127.0.0.1:$api/api/v0/id" >/dev/null && break; sleep 0.5; done
  local cid; cid=$(echo "repro-$name-$RANDOM" | "$IPFS_BIN" block put)
  : >"$WORK/router.log"
  echo "== case $name (Provide.DHT.SweepEnabled=${sweep:-default})"
  echo "-- ipfs provide once $cid:"
  "$IPFS_BIN" provide once "$cid" 2>&1 | head -5; echo "   exit=${PIPESTATUS[0]}"
  sleep 8
  echo "-- PUTs to the HTTP router after provide once:"
  grep -c '^PUT' "$WORK/router.log" | sed 's/^/   count=/'
  grep '^PUT' "$WORK/router.log" | grep -c "$cid" | sed 's/^/   containing cid=/'
  kill "$dpid"; wait "$dpid" 2>/dev/null
}

HTTP_ROUTER="\"HttpRouter\":{\"Type\":\"http\",\"Parameters\":{\"Endpoint\":\"http://127.0.0.1:$ROUTER_PORT\"}}"
DHT_ROUTER="\"DhtRouter\":{\"Type\":\"dht\",\"Parameters\":{\"Mode\":\"client\",\"PublicIPNetwork\":true}}"

# A: HTTP-only custom routing (fixed by kubo#11112) - expect a PUT containing the cid.
run_case A "{\"Type\":\"custom\",\"Routers\":{$HTTP_ROUTER},\"Methods\":{
  \"provide\":{\"RouterName\":\"HttpRouter\"},\"find-providers\":{\"RouterName\":\"HttpRouter\"},
  \"find-peers\":{\"RouterName\":\"HttpRouter\"},\"get-ipns\":{\"RouterName\":\"HttpRouter\"},\"put-ipns\":{\"RouterName\":\"HttpRouter\"}}}" ""

# B: same, but find-peers goes to a DHT router; provide still points only at the HTTP router.
ROUTING_B="{\"Type\":\"custom\",\"Routers\":{$HTTP_ROUTER,$DHT_ROUTER},\"Methods\":{
  \"provide\":{\"RouterName\":\"HttpRouter\"},\"find-providers\":{\"RouterName\":\"HttpRouter\"},
  \"find-peers\":{\"RouterName\":\"DhtRouter\"},\"get-ipns\":{\"RouterName\":\"HttpRouter\"},\"put-ipns\":{\"RouterName\":\"HttpRouter\"}}}"
run_case B "$ROUTING_B" ""

# C: B with the sweep provider disabled - expect the PUT to come back.
run_case C "$ROUTING_B" "false"
Expected

One of:

  • the provider selection treats Routing.Type=custom the way #11112 does for HTTP-only, and uses the LegacyProvider over the Composer whenever the sweep provider has no usable DHT client; or
  • the DHT router built by irouting.Parse is exposed as dhtc so the sweep provider actually runs (this changes where reprovides go, so it probably needs its own decision, see #11411); or, at minimum,
  • provide once returns an error instead of reporting queued when nd.Provider is a NoopProvider.
Related
  • #11089 / #11112: the HTTP-only half of this, fixed in 0.40.
  • #10975: made a missing DHT a silent NoopProvider instead of an fx error, which is where the silence comes from.
  • #11411 (comment 5910057787): the per-call router selector proposal hits the same wall, since a dht target resolving to nd.Provider is a no-op under custom.
Dominant language
Go
Stars
17.2k
Forks
3.2k
PR merge metrics
No merged PRs in 30d

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from ipfs/kubo

All issues in ipfs/kubo

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.