csplit: a `/REGEX/` pattern after a line-number pattern never tests the line the split stopped at
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 3/5
- Estimated time
- Half a day
- Newbie friendliness
- 25/100
Research direction
The bug is in the csplit implementation, in SplitWriter::do_to_line and SplitWriter::do_to_match. The line-number pattern leaves line N buffered, and do_to_match drains that buffer without testing it. Read the tests in tests/by-util/test_csplit.rs, especially test_line_num_range_with_up_to_match3, which has a NOTE about the wrong behaviour. Done means the three reproduction commands match GNU 9.12 and that NOTE is removed. Open PR #15444 already targets this, so check it before starting.
Written by the indexing model from the issue text.
Description
After a line-number pattern N, line N is the current line. A following /REGEX/ pattern should test it. uutils csplit starts searching at line N+1, so line N goes into the next file without being tested.
The GNU manual (csplit invocation) says:
n: Create an output file containing the input up to but not including line n (a positive integer).
/regexp/[offset]: Create an output file containing the current line up to (but not including) the next line of the input file that contains a match for regexp.
The only lines the manual excludes from later regexp searches are "lines within a negative offset of a regexp pattern". GNU coreutils 9.12 tests line N:
printf 'a\n1\n2\n' | csplit - 2 '/[0-9]/'
# GNU 9.12: 2 / 0 / 4
# uutils: 2 / 2 / 2 (line "1" matches /[0-9]/ but goes into xx01)
seq 4 | csplit - 3 /3/
# GNU 9.12: 4 / 0 / 4
# uutils: 4 / 4
# csplit: '/3/': match not found (exit 1)
seq 50 > 50.txt; csplit 50.txt 10 /10/ -k
# GNU 9.12: 18 / 0 / 123
# uutils: 18 / 123
# csplit: '/10/': match not found (exit 1)
So lines land in the wrong output file. When the only match is on line N, csplit fails with "match not found", and without -k it deletes the files it already wrote. No options are needed to hit it.
Upstream's tests already note the difference. In tests/by-util/test_csplit.rs, test_line_num_range_with_up_to_match3 (the third command above) has the comment // NOTE: output different than gnu's: the pattern /10/ is matched but should not. GNU does match line 10 there and prints 18 0 123.
Versions. uutils main 2e28cc3042006f5811e05040232b6154bb905465 and the 0.12.0 release binary, Linux x86_64, LC_ALL=C, compared with GNU coreutils 9.12 (and 9.4, which prints the same).
Where. SplitWriter::do_to_line stops at line N and leaves it buffered for the next pattern. SplitWriter::do_to_match then writes the buffered lines into the new split without matching them (// NOTE: drain the buffer of input_iter, no match should be done within.). That is right for lines an earlier regex already looked at, but not for a line held back by a line-number pattern.
Related, in the same function: #6126 (--suppress-matched with a line number that was already passed).
Found with an automated differential tester that compares uutils with GNU 9.12. I reproduced it by hand on main and on 0.12.0.
Reproduction steps
export LC_ALL=C
cd "$(mktemp -d)"
printf 'a\n1\n2\n' | csplit - 2 '/[0-9]/'; echo "exit=$?"
# GNU 9.12: 2 / 0 / 4, exit=0 (xx00 = "a", xx01 = empty, xx02 = "1\n2")
# uutils: 2 / 2 / 2, exit=0 (xx00 = "a", xx01 = "1", xx02 = "2")
rm -f xx*
seq 4 | csplit - 3 /3/; echo "exit=$?"
# GNU 9.12: 4 / 0 / 4, exit=0
# uutils: 4 / 4, "csplit: '/3/': match not found", exit=1
- Dominant language
- Rust
- Stars
- 24.3k
- Forks
- 2.1k
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 313
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- No pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from uutils/coreutils
-
minorbug(mv): invalid mode errors not quotedPossibly taken @henrimarconni claimed this 4 days ago. OpenU - mv
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
ptx: break file missing path in error messagePossibly taken @Franklin-Qi claimed this 4 days ago. OpenU - ptx
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
uutils/coreutils#15307 · 1 comment ·
Maintainers usually reply within 1 day
-
minorbug(stat): os errno not stripped when failing to flush stdoutPossibly taken @cestercian claimed this 11 days ago. OpenU - stat
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
uutils/coreutils#15200 · 3 comments ·
Maintainers usually reply within 1 day
-
stty: support B460800 and B921600 baud ratesPossibly taken @oech3 claimed this 7 days ago. Openawaiting upstream U - stty
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Maintainers usually reply within 1 day
-
B - high priority U - du
Difficulty 1/5 Under an hour Newbie friendliness 82/100
uutils/coreutils#14984 · 1 comment ·
Maintainers usually reply within 1 day
All issues in uutils/coreutils
Similar issues
-
engine-rs P3 review-deferral wasm
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
maniator/verticopolis#1154 ·
Maintainers usually reply within 1 day
-
FFMPEG_BIN/FFPROBE_BIN are always overridden, so a working system ffmpeg cannot be usedPossibly taken @Bilker1422 claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Stremio/stremio-service#116 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 83/100
Maintainers usually reply within 1 day