A file scan with Limit 0 loses the file's columns, and the CSVOld scan throws
维护者通常 1 天内回复
评估
这个 Issue 还没有评估数据。
描述
What happened?
The Limit on a file-scan source is meant to bound how many rows the operator outputs. It also bounds the sample sourceSchema() reads to infer the columns, so a Limit of 0 leaves the inference nothing to look at and the operator reports a schema that does not describe the file.
The three readers fail differently on the same file. CSV File Scan and JSONL File Scan return a schema with no attributes at all. CSVOld File Scan throws ArrayIndexOutOfBoundsException, because it takes the column names from the header row and the types from the sample, then asks the empty type array for the first header's type.
Zero is a value the property editor accepts, and it is the natural way to ask for no rows. The columns of a file do not depend on how many of its rows the user wants, so a Limit of 0 should give an operator with the file's schema and an empty output, not one with no columns or one that throws.
How to reproduce?
Point a CSVOld File Scan at a CSV with a header and a couple of data rows, set Limit to 0, and run. The execution fails as the executor is constructed, on the operator's own schema inference, before a row is read. The same two steps on a CSV File Scan or a JSONL File Scan instead give downstream operators an operator with no columns.
Set Limit to 1 on any of the three and the schema is correct again, which is what tells the two concerns apart: CSVOldScanSourceOpDesc.sourceSchema reads its inference sample from slice(startOffset, startOffset + limit), and CSVScanSourceOpDesc and JSONLScanSourceOpDesc bound theirs the same way.
Version/Branch
1.4.0-incubating-SNAPSHOT (main)
Commit Hash (Optional)
e7d1676e1
What browsers are you seeing the problem on?
No response
Relevant log output
java.lang.ArrayIndexOutOfBoundsException: Index 0 out of bounds for length 0
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.$anonfun$sourceSchema$8(CSVOldScanSourceOpDesc.scala:204)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpDesc.sourceSchema(CSVOldScanSourceOpDesc.scala:201)
at org.apache.texera.amber.operator.source.scan.csvOld.CSVOldScanSourceOpExec.<init>(CSVOldScanSourceOpExec.scala:38)
- 主要语言
- Scala
- 星标
- 322
- 派生
- 195
- 平均合并
- 2 天 16 小时
- 30 天内合并 PR
- 127
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
apache/texera 的其他 Issue
-
Search bar autocomplete stops working after a model appears in the suggestions可能已有人在做 @aglinxinyuan 于 25 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 62/100
维护者通常 1 天内回复
-
JSONL File Scan reads a JSON null as the text "null"可能已有人在做 @CaroFernando 于 23 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 86/100
apache/texera#8674 · 1 条评论 · 已指派 1 人 ·
维护者通常 1 天内回复
-
LARGE_BINARY test leaves a real S3 object behind on every S3-reachable run可能已有人在做 @ShawnGarello 于 8 天前认领。 未关闭
难度 2/5 1-3 小时 新手友好度 84/100
维护者通常 1 天内回复
-
Missing workflow returns 500 from workflow type endpoint可能重新可做 @VuMartin 于 48 天前认领，目前没有进行中的 PR。 未关闭
难度 2/5 1-3 小时 新手友好度 82/100
维护者通常 1 天内回复
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 72/100
scylladb/scylla-migrator#417 ·
-
arch build fails未关闭
难度 2/5 1-3 小时 新手友好度 65/100
-
难度 2/5 1-3 小时 新手友好度 68/100
digital-asset/participant-query-store#177 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 84/100
containerbase/base#7891 ·
维护者通常 1 天内回复
-
难度 1/5 1-3 小时 新手友好度 91/100
com-vitthalmirji/toon4s#117 ·
维护者通常 1 天内回复