CSVDocumentCleaner silently discards data for negative ignore counts
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 78/100
Research direction
Start at the CSVDocumentCleaner entry point and run the supplied reproduction for negative ignore_rows and ignore_columns. Trace where these values reach pandas positional slicing, then add validation while preserving current zero and positive behavior. Done means negative counts raise ValueError and regression tests cover all three cases.
Written by the indexing model from the issue text.
Description
CSVDocumentCleaner accepts negative ignore_rows and ignore_columns. These values reach pandas positional slicing and can silently discard data.
Reproduction on a main snapshot downloaded on 2026-10-02:
from haystack import Document
from haystack.components.preprocessors import CSVDocumentCleaner
document = Document(content='A,B\n1,2\n3,4\n')
for options in ({'ignore_rows': -1}, {'ignore_columns': -1}):
result = CSVDocumentCleaner(**options).run(documents=[document])
print(options, repr(result['documents'][0].content))
Observed output:
{'ignore_rows': -1} '3,4\n'
{'ignore_columns': -1} 'B\n2\n4\n'
I expected negative counts to raise ValueError, while zero and positive counts retain their current behavior. I have a local patch for validation, regression tests, documentation, and a release note. Would this change be welcome? I would like to contribute the fix.
AI assistance was used to prepare this report and the local patch.
- Dominant language
- Python
- Stars
- 26.8k
- Forks
- 3.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 258
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from deepset-ai/haystack
-
CSVToDocument accepts an unsupported conversion_mode and silently converts in row modePossibly taken @Lesereingrape claimed this 16 days ago. OpenP3
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
deepset-ai/haystack#13070 ·
Maintainers usually reply within 1 day
-
PPTXToDocument drops soft line breaks when link_format is markdown or plainPossibly taken @bogdankostic claimed this 6 days ago. OpenP3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
deepset-ai/haystack#13043 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
InMemoryDocumentStore: count_unique_metadata_by_filter and get_metadata_field_unique_values disagree on int/float/bool valuesPossibly taken @julian-risch claimed this 16 days ago. OpenP3
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
deepset-ai/haystack#13040 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
AutoMergingRetriever raises ValueError when a matched document has no parent (for example the root document)Possibly taken @julian-risch claimed this 18 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
deepset-ai/haystack#13029 · 2 comments · 1 assignee ·
Maintainers usually reply within 1 day
-
CSVToDocument row mode fails on CR-separated CSV inputPossibly taken @julian-risch claimed this 17 days ago. OpenP3
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
deepset-ai/haystack#13022 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
All issues in deepset-ai/haystack
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 77/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 67/100
webrecorder/warcio#209 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
Jeffy123-zhu/logspecter#20 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 85/100
Maintainers usually reply within 1 day