Flock 0.8.0 group-file race crashes a subscriber during concurrent joins (push_fanout / detector_narrowing benchmarks)
Nobody has claimed this yet.
Assessment
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Newbie friendliness
- 15/100
Research direction
CI is already building Flock from commit 7b484c5 via 02c28cf; start by checking that CI pin and the na+sm runs for push_fanout_benchmark and detector_narrowing_benchmark. Done means several green runs with both benchmarks passing, then CI can move to a Flock release newer than 0.8.0; if failures persist, symbolize the crash with debug info as the issue describes.
Written by the indexing model from the issue text.
Description
Symptom
push_fanout_benchmark and detector_narrowing_benchmark intermittently fail in the na+sm CI job. The writer reports subscriber N never signaled ready, with a different N each time:
| Run | Mercury | Case | Subscriber |
|---|---|---|---|
| 35791025169 | 2.3.1 | push_fanout, 4 subscribers | 2 |
| 35791025169 | 2.3.1 | detector_narrowing | 8 |
| 35974637262 | 2.4.1 | push_fanout, 8 subscribers | 4 |
Cause
That subscriber didn't time out. It crashed while joining. Both push_fanout logs show a SIGSEGV in an Argobots thread inside libflock-server.so, at the same offset (+0x471d) under both Mercury versions:
b_push_fanout:45640 terminated with signal 11 ...
/home/runner/mochi-install/lib/libflock-server.so.0(+0x471d)
/home/runner/mochi-install/lib/libflock-server.so.0(+0xe14f)
...
/home/runner/mochi-install/lib/libabt.so.1(+0x1a6bc)
The 2.4.1 run also shows the writer failing to update the group file several times while subscribers join:
[error] [flock] Could not rename file b_push_fanout_4.h5.vsgroup.swp into b_push_fanout_4.h5.vsgroup: No such file or directory
This is a Flock 0.8.0 bug. On every membership change, the writer's provider rewrites the group file (dispatch_member_update -> serialize_view_to_file, src/provider.c), and nothing serializes those rewrites:
file_serializer()(src/view-serialize.c) always writes to the same<file>.swpand then renames it. When two updates run at once, one rename moves the other's half-written file into place (a torn group file), and the other rename fails with ENOENT.flock_group_view_serialize()walksv->memberswithout the view lock, and the SWIM gossip handler can reallocate that array (flock_group_view_add_member) at the same moment.
The ENOENT rename errors prove that two updates really do run concurrently in one process. The crash matches unlocked access to the member array, but I haven't mapped +0x471d to a symbol, so that part is inferred, not confirmed.
vol-stream already works around the read side of the same problem (tr_mercury.c, vs_tr_reader_join_group(): a retry because flock_group_view_from_file() ignores fread()'s return value).
Fix
Upstream fixed all three points in mochi-flock 7b484c5 ("better access to group files", 2026-08-28): an mkstemp temp file, the view lock held while serializing, and a checked fread. It's the only commit after v0.8.0 and hasn't been released yet.
02c28cf builds Flock from that commit in CI.
To close this
- Several green na+sm runs with both benchmarks passing.
- Move CI back to a release tag once Flock ships one after 0.8.0, and consider raising vol-stream's
flock >= 0.8.0floor, because users building against the 0.8.0 release will still hit this. - If it still happens with the pin, symbolize the crash (build Flock with debug info and resolve the offset) before looking elsewhere.
My first version of this issue blamed a tight 10-second wait in the test and CPU contention on the runner. That was wrong: I had only read the writer's warnings, not the subscriber's crash.
- Dominant language
- C
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Getting set up
This project ships no dev container, Dockerfile or contributing guide, so setting up is up to you: start from its README, and see our first-contribution guide for the general steps.
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Similar issues
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 70/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
awslabs/aws-c-auth#315 ·
-
GCC 15.2: -Wcast-qual warning - cast discards const qualifier from pointer target type (H5trace.c)Open
Difficulty 2/5 1-3 hours Newbie friendliness 73/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
FujiNetWIFI/fujinet-firmware#2365 ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 79/100
ChupathingyCE/chupathingyce#1128 ·
Maintainers usually reply within 1 day