Implement Spark-compatible `try_aes_decrypt`
维护者通常 1 天内回复
还没有人认领这个 Issue。
评估
调研方向
Start by reading the existing try_* functions under crates/spark/src/function/misc/ (try_to_binary, try_url_decode, try_parse_url) to see how per-row errors are turned into nulls, and confirm the state of the aes_decrypt kernel this must build on, since it is a prerequisite. Then implement the wrapper, add crates/sqllogictest/test_files/misc/try_aes_decrypt.slt with both success and error-to-NULL cases, and annotate with #[user_doc]. Done when every listed aes_decrypt error yields NULL, good rows still decrypt, and the sqllogictest file passes.
由索引模型根据 Issue 内容生成。
描述
Part of #272 and apache/datafusion-comet#7300.
What the function does
try_aes_decrypt(expr, key[, mode[, padding[, aad]]]) behaves like aes_decrypt but returns NULL instead of
raising an error. Arguments, defaults (mode = 'GCM', padding = 'DEFAULT', aad = ''), implicit casts and the
supported (mode, padding) pairs are the same as aes_decrypt: ECB/CBC with PKCS/DEFAULT, GCM with
NONE/DEFAULT; 16/24/32-byte keys; CBC input iv(16) || ct, GCM input iv(12) || ct || tag(16).
Returns nullable BINARY. Since 3.5.0.
Spark implements it as TryEval(AesDecrypt(...)) (RuntimeReplaceable with InheritAnalysisRules). TryEval
catches every Exception thrown while evaluating the wrapped expression, so all runtime errors become NULL.
Analysis-time errors (wrong argument count or types) are still raised.
Examples
SELECT cast(try_aes_decrypt(unhex('6E7CA17BBB468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'), '0000111122223333', 'GCM') AS STRING);
-- Spark SQL
SELECT try_aes_decrypt(unhex('----------468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'), '0000111122223333', 'GCM');
-- NULL
SELECT cast(try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'), 'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'This is an AAD mixed into the input') AS STRING);
-- Spark
SELECT cast(try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAAAAAAPSd4mWyMZ5mhvjiAPQJnfg='), 'abcdefghijklmnop12345678ABCDEFGH', 'CBC') AS STRING);
-- Spark
SELECT cast(try_aes_decrypt(unhex('83F16B2AA704794132802D248E6BFD4E380078182D1544813898AC97E709B28A94'), '0000111122223333') AS STRING);
-- Spark
The first two come from the ExpressionDescription. In the second, unhex of invalid hex already returns NULL, so
it does not exercise the error path. The others are MiscFunctionsSuite inputs whose aes_decrypt results are in
the aes_decrypt examples. The NULL cases below follow from TryEval plus the aes_decrypt error tests (code
reading):
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='), '12345678901234567'); -- NULL (AES_KEY_LENGTH)
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='), binary('1234567812345678'), 'ECB'); -- NULL (bad padding, wrong key)
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='), 'abcdefghijklmnop', 'GCM', 'PKCS'); -- NULL (AES_MODE)
SELECT try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'),
'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'The ciphertext is valid, but the AAD is wrong'); -- NULL (tag mismatch)
Edge cases to test
- Every
aes_decrypterror becomes NULL:INVALID_PARAMETER_VALUE.AES_KEY_LENGTH,UNSUPPORTED_FEATURE.AES_MODE,
UNSUPPORTED_FEATURE.AES_MODE_AAD(AAD with CBC),INVALID_PARAMETER_VALUE.AES_CRYPTO_ERROR(wrong key, corrupted
or truncated ciphertext, empty GCM input, wrong AAD). - Successful decrypts match
aes_decryptbyte for byte, for all modes and key sizes. - NULL in any argument gives NULL.
TryEvalwraps the argument expressions too, so a runtime error inside an argument (for example an ANSI cast
failure in aCASTwith ANSI on, orraise_error(...)passed askey) is also swallowed. In
DataFusion, arguments are evaluated before the UDF runs, so this cannot be reproduced inside the UDF. Document the
difference or handle it at plan-conversion time.- AAD with ECB is ignored, not an error (same as
aes_decrypt). - Literal and column for each argument, with a mix of good and bad rows in one batch (bad rows NULL, good rows
decrypted). - Deterministic.
Differences across Spark versions
- Not available in 3.4. Added in 3.5.0 with CBC and AAD support already present.
- 4.0: collated
mode/paddingaccepted. No behavior change through 4.2.
Implementation notes
- Spark:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala(TryAesDecrypt),
expressions/TryEval.scala, and
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java(aesInternal). - DataFusion core has no AES functions.
- Implement on top of the
aes_decryptkernel (see theaes_decryptissue). Per-row errors become nulls instead
of failing the batch. Existingtry_*functions in datafusion-spark (try_to_binary,try_url_decode,
try_parse_url) show the pattern. - Module:
crates/spark/src/function/misc/. - Tests: add
crates/sqllogictest/test_files/misc/try_aes_decrypt.slt; document with#[user_doc].
This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.
- 主要语言
- Rust
- 星标
- 7
- 派生
- 15
- 平均合并
- 16 小时 43 分钟
- 30 天内合并 PR
- 26
环境准备
- 没有 Dockerfile 或 Docker Compose 文件
- 没有 Pull Request 模板
- 阅读贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
apache/datafusion-spark 的其他 Issue
-
enhancement
难度 2/5 1-3 小时 新手友好度 70/100
apache/datafusion-spark#287 ·
维护者通常 1 天内回复
-
enhancement good first issue
难度 2/5 1-3 小时 新手友好度 80/100
apache/datafusion-spark#266 ·
维护者通常 1 天内回复
-
[Bug] make_dt_interval returns NULL for more than about 1.49 million days, Spark returns the interval可能已有人在做 @Subodh-17 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
apache/datafusion-spark#109 ·
维护者通常 1 天内回复
-
enhancement
难度 5/5 一周以上 新手友好度 18/100
apache/datafusion-spark#299 ·
维护者通常 1 天内回复
-
enhancement
难度 4/5 3-5 天 新手友好度 35/100
apache/datafusion-spark#298 ·
维护者通常 1 天内回复
查看 apache/datafusion-spark 的全部 Issue
相似的 Issue
-
comp-rbac
难度 2/5 1-3 小时 新手友好度 68/100
ClickHouse/ClickHouse#131875 ·
维护者通常 1 天内回复
-
bug(python): read_transaction raises TypeError on a Rewrite with rewritten indices可能已有人在做 @Borg93 今天认领。 未关闭bug
难度 2/5 1-3 小时 新手友好度 75/100
lance-format/lance#11117 ·
维护者通常 2 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
storytold/photocraft#15232 · 1 条评论 ·
维护者通常 1 天内回复
-
forge: fork preflight reads still send `requireCanonical: false`, which Rootstock rejects (-32603)可能已有人在做 @Ynyesto 今天认领。 未关闭
难度 2/5 1-3 小时 新手友好度 72/100
foundry-rs/foundry#18485 ·
维护者通常 1 天内回复
-
难度 2/5 1-3 小时 新手友好度 70/100
bondybondbond/rushcut#238 · 1 条评论 ·