A file scan with Limit 0 loses the file's columns, and the CSVOld scan throws
Maintainers usually reply within 1 day
Nobody has claimed this yet.
Assessment
This issue has not been assessed yet.
Description
What happened?
The Limit on a file-scan source is meant to bound how many rows the operator outputs. It also bounds the sample sourceSchema() reads to infer the columns, so a Limit of 0 leaves the inference nothing to look at and the operator reports a schema that does not describe the file.
The three readers fail differently on the same file. CSV File Scan and JSONL File Scan return a schema with no attributes at all. CSVOld File Scan throws ArrayIndexOutOfBoundsException, because it takes the column names from the header row and the types from the sample, then asks the empty type array for the first header's type.
Zero is a value the property editor accepts, and it is the natural way to ask for no rows. The columns of a file do not depend on how many of its rows the user wants, so a Limit of 0 should give an operator with the file's schema and an empty output, not one with no columns or one that throws.
How to reproduce?
Point a CSVOld File Scan at a CSV with a header and a couple of data rows, set Limit to 0, and run. The execution fails as the executor is constructed, on the operator's own schema inference, before a row is read. The same two steps on a CSV File Scan or a JSONL File Scan instead give downstream operators an operator with no columns.
Set Limit to 1 on any of the three and the schema is correct again, which is what tells the two concerns apart: CSVOldScanSourceOpDesc.sourceSchema reads its inference sample from slice(startOffset, startOffset + limit), and CSVScanSourceOpDesc and JSONLScanSourceOpDesc bound theirs the same way.
Version/Branch
1.4.0-incubating-SNAPSHOT (main)
Commit Hash (Optional)
e7d1676e1
What browsers are you seeing the problem on?
No response
Relevant log output
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.$anonfun$sourceSchema$8(CSVOldScanSourceOpDesc.scala:204)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.sourceSchema(CSVOldScanSourceOpDesc.scala:201)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpExec.<init>(CSVOldScanSourceOpExec.scala:38)
- Dominant language
- Scala
- Stars
- 316
- Forks
- 192
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 136
Getting set up
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/texera
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
apache/texera#8775 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/texera#8700 · 1 comment ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
apache/texera#8682 · 5 comments ·
Maintainers usually reply within 1 day
-
JSONL File Scan reads a JSON null as the text "null"Possibly taken @CaroFernando claimed this 4 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 86/100
apache/texera#8674 · 1 comment · 1 assignee ·
Maintainers usually reply within 1 day
-
Difficulty 2/5 1-3 hours Newbie friendliness 84/100
Maintainers usually reply within 1 day
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
lichess-org/lila#21930 ·
Maintainers usually reply within 1 day
-
scope:security type:aquasec
Difficulty 1/5 Under an hour Newbie friendliness 90/100
AbsaOSS/StatusBoard#69 ·
-
A-consensus C-question
Difficulty 1/5 Under an hour Newbie friendliness 74/100
ergoplatform/ergo#2624 ·
Maintainers usually reply within 3 days
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
Maintainers usually reply within 1 day
-
[Rust][Flaky Test] multiple_deadlines_fire_in_order asserts a wall-clock gap instead of firing orderOpenCI/CD ⚒️ Flaky-tests 🐦
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
valkey-io/valkey-glide#7255 ·
Maintainers usually reply within 2 days