delta consolidate reports a governor refusal as ENOMEM without the session denial flag
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Newbie friendliness
- 20/100
Research direction
Read wl_columnar_eval_delta_consolidate in wirelog/columnar/eval_delta.c and the linked pull request #2165 to see how the work is proceeding. The issue identifies refusals in col_rel_append_all and col_rel_prepare_replacement; inspect those paths and the untimestamped col_op_consolidate route. Done means both budget refusals report ENOSPC with the denial flag set, while allocator faults remain ENOMEM with the flag clear; cover both cases in tests/test_session.c.
Written by the indexing model from the issue text.
Description
Summary
wl_columnar_eval_delta_consolidate (wirelog/columnar/eval_delta.c) can return ENOMEM for a governor refusal without the session's memory_budget_denied flag, so the public facades report WIRELOG_ERR_MEMORY instead of WIRELOG_ERR_MEMORY_BUDGET. Found while fixing #2160 (the nonrecursive multiworker merge), whose plan ends in CONSOLIDATE; the merge itself is handled there, this is the consolidate step that runs after it.
Where
wl_columnar_eval_delta_consolidate builds a $consol candidate and replaces the target with it:
col_rel_append_all(candidate, target, NULL)(eval_delta.c ~1834): a refused growth returnsENOMEMwithcandidate->memory_budget_denial_pendingset. The candidate is never insess->rels, andwl_columnar_eval_stack_cleanup_finish(&frame)(~1899) destroys it, so neither the rc norwl_columnar_session_budget_deniedreports a denial.col_rel_prepare_replacement(target, candidate, ...)(~1867): a refused reservation returnsENOMEMwithtarget->memory_budget_denial_pendingset. When the target is registered insess->rels,wl_columnar_session_budget_deniedsees the evidence, but the rc staysENOMEMandwl_facade_session_error_code_with_budget(wirelog/extension_registry.c) returnsWIRELOG_ERR_MEMORY_BUDGETonly forENOSPC. When the target is a temporary (eval.cfinal->rel,combined), the evidence is lost too.
Creating the candidate is already mapped (ENOSPC with the flag). The function has six callers (eval.c wl_columnar_eval_finalize_relation, wl_columnar_eval_nonrec_relation_parallel and tdd_global_read_exchange_deltas; eval_delta.c; eval_serial.c; eval_tdd_insert.c), so the mapping belongs inside it.
Reproduction (observable state only)
The #2160 test harness in tests/test_session.c (merge_denial_run), with the plan changed from VARIABLE(input) to VARIABLE(input), CONSOLIDATE and the budget left in place after phase boundary 4:
- 2 workers,
WIRELOG_NONREC_TDD_MIN_ROWS_PER_WORKER=64, 128 distinct timestamped INT64 input rows,outputpre-registered with 1 timestamped row; - at phase boundary 3, the governor is set ENFORCING with
usable_bytes = reserved + headroom; - headroom swept from 0 in 256-byte steps, with the #2160 fix applied.
| headroom (bytes) | phase 4 (merge done) | phase 5 (consolidate done) | step rc | denial flag | public code |
|---|---|---|---|---|---|
| 0-13056 | no | no | ENOSPC | set | WIRELOG_ERR_MEMORY_BUDGET |
| 13312 | yes | no | ENOSPC | set | WIRELOG_ERR_MEMORY_BUDGET |
| 13568-27904 | yes | no | ENOMEM | clear | WIRELOG_ERR_MEMORY |
| 28160 | yes | yes | 0 | clear | OK |
With fprintf instrumentation (not needed for the reproduction above), the ENOMEM band splits into the candidate append (about 13504-21695) and prepare_replacement (about 21696-27967).
Expected
ENOSPC with sess->memory_budget_denied set for both refusals, so every caller reports a budget denial.
Suggested fix and test
- Inside
wl_columnar_eval_delta_consolidate, map each result while its relation is alive (ENOMEMplus the relation'smemory_budget_denial_pendingbecomesENOSPC; the candidate for the append, the target forprepare_replacement) and setsess->memory_budget_deniedonENOSPC, before the cleanup frame destroys the candidate. - Audit the branch for candidates without timestamps (the
col_op_consolidateroute), which this reproduction does not exercise. - Test both refusals with a searched headroom (as the #2160 test does), asserting
ENOSPC, the flag andWIRELOG_ERR_MEMORY_BUDGET, and add an allocator-fault case that must stayENOMEMwith the flag clear.
- Dominant language
- C
- Stars
- 18
- Forks
- 2
- Avg merge
- 8h
- Merged PRs (30d)
- 287
Getting set up
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from semantic-reasoning/wirelog
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
semantic-reasoning/wirelog#2163 ·
Maintainers usually reply within 1 day
-
Doc: insert executor and incremental consolidation leave sorted runs, not globally sorted headsOpen
Difficulty 1/5 Under an hour Newbie friendliness 83/100
semantic-reasoning/wirelog#2133 ·
Maintainers usually reply within 1 day
-
Difficulty 1/5 Under an hour Newbie friendliness 88/100
semantic-reasoning/wirelog#2042 ·
Maintainers usually reply within 1 day
-
p2-medium
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
semantic-reasoning/wirelog#1914 ·
Maintainers usually reply within 1 day
-
enhancement
Difficulty 3/5 1-2 days Newbie friendliness 62/100
semantic-reasoning/wirelog#2142 ·
Maintainers usually reply within 1 day
All issues in semantic-reasoning/wirelog
Similar issues
-
Difficulty 2/5 Half a day Newbie friendliness 62/100
Maintainers usually reply within 1 day
-
bug good first issue
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
rpc : a tensor with oversized ne aborts ggml-rpc-server in deserialize_tensorPossibly taken @void-mckenzie claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 66/100
Maintainers usually reply within 1 day
-
enhancement
Difficulty 1/5 Under an hour Newbie friendliness 85/100
FujiNetWIFI/fujinet-lib#122 · 1 comment ·
Maintainers usually reply within 1 day