CSV rows for stdout sit in a 4 KB buffer until the run ends

Open Beginner friendly
  • Is something wrong?
#319 0 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 1 day

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
76/100
Issue type
Bug
Clarity
Clearly specified
Activity status
Active
Tech stack
go
Domain
cli

Research direction

Start in internal/output/writer.go: ensureCSV wraps stdout in a csv.NewWriter, and writeCSVRows writes each row without flushing, so the flush only happens in Finish. Reproduce with -simulate -format csv ... | cat versus -format jsonl. Done when each stdout CSV row is flushed as it is written and a test in the output package (using -simulate or the local DNS test server, not a live domain) reads a row from stdout before the run ends.

Written by the indexing model from the issue text.

Description

area: output bug good first issue severity: low

Summary

With -format csv, results reach stdout only when 4 KB has accumulated or the run finishes, unlike text, JSON and JSONL, which are written as each result is found; a force quit loses the buffered rows.

Where

  • internal/output/writer.go:328
  • internal/output/writer.go:354
  • internal/output/writer.go:390
  • internal/output/writer.go:30

Details

ensureCSV wraps stdout in csv.NewWriter(w.stdOut()) (line 328) and writeCSVRows only calls w.csvStdout.Write(row) (line 355). encoding/csv.Writer buffers through a bufio.Writer and is flushed in exactly one place, Finish (lines 390-392; git grep "Flush()" confirms). Text, JSONL and even the JSON array (w.stdOut().Write(data), line 279) are written to stdout as each result is found. The FormatCSV doc comment says it "streams" rows (line 30).

Impact

With -format csv, nothing reaches stdout until 4 KB of rows have accumulated or the run finishes: a consumer on a pipe sees no live results, on a terminal the rows appear after the "Scan complete" breakdown instead of as they are found, and a force quit (the second Ctrl+C, which main_session.go:519-521 deliberately restores) loses every buffered row. The other three formats do not behave this way.

Steps to reproduce

subenum -simulate -hit-rate 100 -rate 2 -format csv -w examples/sample_wordlist.txt example.com | cat prints nothing until the scan ends; the same command with -format jsonl prints a line per hit.

Suggested fix

Call w.csvStdout.Flush() at the end of writeCSVRows when w.csvStdout != nil (stdout only; the file writer already sits on the -o bufio.Writer and can stay buffered).

Acceptance criteria

  • Each CSV row written to stdout is flushed when it is written
  • A test reads a row from stdout before the run finishes

Getting started

  • Relevant files: internal/output/writer.go
  • How to run tests: make test (or go test ./...). Tests must stay hermetic: use -simulate, -simulate-zone or the local DNS test server, never a live domain.
  • Related docs: docs/DEVELOPER_GUIDE.md, docs/ARCHITECTURE.md

Filed by repo-audit/full-audit at d0c56f0. Severity: low. Effort: S. audit-key: bug|internal/output/writer.go|writeCSVRows

Dominant language
Go
Stars
1
Forks
1
Avg merge
6h 42m
Merged PRs (30d)
7

Getting set up

Open in Codespaces

Starts the project's dev container in your browser, under your own GitHub account.

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from TMHSDigital/subenum

All issues in TMHSDigital/subenum

Similar issues

More Go issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.