`lpad` and `rpad` with a literal length above `i32::MAX` error on a NULL string or fill
Maintainers usually reply within 1 day
Assessment
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Newbie friendliness
- 77/100
Research direction
Start in datafusion/functions/src/unicode/pad.rs, comparing the scalar length fast path with the array path’s NULL handling. Check the existing string_literal.slt cases and the related split_part fix mentioned in the issue. Done when lpad and rpad return NULL for oversized literal lengths on rows with a NULL string or fill, while still erroring for rows where the relevant arguments are non-NULL.
Written by the indexing model from the issue text.
Description
Describe the bug
lpad(s, n) on a row where s is NULL and n is above i32::MAX errors when n is a literal, and returns NULL when n comes from a column. A NULL fill behaves the same, and so does rpad. The fast path for a scalar length rejects a length above i32::MAX before it looks at any row. The array path skips rows where any argument is NULL before it checks the length, so it errors only on a row whose string and fill are both non-NULL.
To Reproduce
-- Literal length: the fast path
SELECT lpad(s, 2147483648) AS r FROM (VALUES (NULL::text)) AS t(s);
-- Execution error: lpad requested length 2147483648 too large, maximum allowed length is 2147483647
-- The same row with the length from a column: the array path
SELECT lpad(s, n) AS r FROM (VALUES (NULL::text, 2147483648)) AS t(s, n);
-- NULL
-- A NULL fill with a literal length
SELECT lpad(s, 2147483648, f) AS r FROM (VALUES ('a', NULL::text)) AS t(s, f);
-- Execution error: lpad requested length 2147483648 too large, maximum allowed length is 2147483647
-- The same row with the length from a column
SELECT lpad(s, n, f) AS r FROM (VALUES ('a', 2147483648, NULL::text)) AS t(s, n, f);
-- NULL
The same four queries with rpad give the same results.
Expected behavior
All four return NULL, as the array path does. string_literal.slt already expects NULL for a NULL string or fill with a literal length in range, and #27352 made split_part's fast path return NULL in the same situation (#27276).
Could the fast path skip a length above i32::MAX, as it already skips one above MAX_SCALAR_TARGET_LEN, instead of returning the error? The array path would then return the error for the first row whose string and fill are non-NULL, and NULL for the other rows.
Additional context
Property-based tests found this on main at e2dc8b048b: one row of a call with an array of lengths returned NULL, and the same row called with every argument as a scalar returned the error. The reproduction runs on main at e2dc8b048b. The check in the fast path came with #20657 and moved to pad.rs in #26367.
- Dominant language
- Rust
- Stars
- 9.6k
- Forks
- 2.6k
- Avg merge
- 3d 5h
- Merged PRs (30d)
- 352
Getting set up
Starts the project's dev container in your browser, under your own GitHub account.
- No Dockerfile or Docker Compose file
- Has a pull request template
- Read the contributing guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from apache/datafusion
-
`verify-release-candidate.sh` version check doesn't catch `-dev` versions and is skipped on macOSPossibly taken @vinycalves claimed this today. Open
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
apache/datafusion#27563 · 1 comment ·
Maintainers usually reply within 1 day
-
`MoveTowardsLeafNodes` placement implies speculative evaluation, but the docs don't say soPossibly taken @Inf1mum claimed this 1 day ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 75/100
apache/datafusion#27487 · 3 comments ·
Maintainers usually reply within 1 day
-
Window constructors panic when the expression list is emptyPossibly taken @abp002 claimed this 4 days ago. Open
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
apache/datafusion#27315 · 3 comments · 1 assignee ·
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/datafusion#27094 · 1 comment ·
Maintainers usually reply within 1 day
-
Substrait consumer doesn't recognise `shift_left` / `shift_right` from functions_arithmetic.yamlPossibly taken @jonasHanhan claimed this 15 days ago. Openbug
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
apache/datafusion#27059 ·
Maintainers usually reply within 1 day
All issues in apache/datafusion
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
Maintainers usually reply within 1 day
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 82/100
tinystream-dev/tinystream#132 ·
-
contract good first issue testing
Difficulty 2/5 1-3 hours Newbie friendliness 72/100
Emmyt24/nova-launch#2436 ·
Maintainers usually reply within 1 day