Implement Spark-compatible `try_aes_decrypt`

オープン
#297 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る

メンテナーはふだん 1 日以内に返信

まだ誰も着手していません。

評価

難易度
4/5
見積もり時間
3〜5日
初心者へのやさしさ
48/100
issue の種類
機能追加
明瞭さ
明確に書かれている
活発さ
活発
技術スタック
rust, spark
領域
backend

調査の方向性

Start by reading the existing try_* functions under crates/spark/src/function/misc/ (try_to_binary, try_url_decode, try_parse_url) to see how per-row errors are turned into nulls, and confirm the state of the aes_decrypt kernel this must build on, since it is a prerequisite. Then implement the wrapper, add crates/sqllogictest/test_files/misc/try_aes_decrypt.slt with both success and error-to-NULL cases, and annotate with #[user_doc]. Done when every listed aes_decrypt error yields NULL, good rows still decrypt, and the sqllogictest file passes.

索引モデルが issue の本文から書いたものです。

説明

enhancement

Part of #272 and apache/datafusion-comet#7300.

What the function does

try_aes_decrypt(expr, key[, mode[, padding[, aad]]]) behaves like aes_decrypt but returns NULL instead of
raising an error. Arguments, defaults (mode = 'GCM', padding = 'DEFAULT', aad = ''), implicit casts and the
supported (mode, padding) pairs are the same as aes_decrypt: ECB/CBC with PKCS/DEFAULT, GCM with
NONE/DEFAULT; 16/24/32-byte keys; CBC input iv(16) || ct, GCM input iv(12) || ct || tag(16).
Returns nullable BINARY. Since 3.5.0.

Spark implements it as TryEval(AesDecrypt(...)) (RuntimeReplaceable with InheritAnalysisRules). TryEval
catches every Exception thrown while evaluating the wrapped expression, so all runtime errors become NULL.
Analysis-time errors (wrong argument count or types) are still raised.

Examples
SELECT cast(try_aes_decrypt(unhex('6E7CA17BBB468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'), '0000111122223333', 'GCM') AS STRING);
-- Spark SQL
SELECT try_aes_decrypt(unhex('----------468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'), '0000111122223333', 'GCM');
-- NULL
SELECT cast(try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'), 'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'This is an AAD mixed into the input') AS STRING);
-- Spark
SELECT cast(try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAAAAAAPSd4mWyMZ5mhvjiAPQJnfg='), 'abcdefghijklmnop12345678ABCDEFGH', 'CBC') AS STRING);
-- Spark
SELECT cast(try_aes_decrypt(unhex('83F16B2AA704794132802D248E6BFD4E380078182D1544813898AC97E709B28A94'), '0000111122223333') AS STRING);
-- Spark

The first two come from the ExpressionDescription. In the second, unhex of invalid hex already returns NULL, so
it does not exercise the error path. The others are MiscFunctionsSuite inputs whose aes_decrypt results are in
the aes_decrypt examples. The NULL cases below follow from TryEval plus the aes_decrypt error tests (code
reading):

SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='), '12345678901234567');            -- NULL (AES_KEY_LENGTH)
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='), binary('1234567812345678'), 'ECB'); -- NULL (bad padding, wrong key)
SELECT try_aes_decrypt(unbase64('4Hv0UKCx6nfUeAoPZo1z+w=='), 'abcdefghijklmnop', 'GCM', 'PKCS'); -- NULL (AES_MODE)
SELECT try_aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'),
  'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'The ciphertext is valid, but the AAD is wrong'); -- NULL (tag mismatch)
Edge cases to test
  • Every aes_decrypt error becomes NULL: INVALID_PARAMETER_VALUE.AES_KEY_LENGTH, UNSUPPORTED_FEATURE.AES_MODE,
    UNSUPPORTED_FEATURE.AES_MODE_AAD (AAD with CBC), INVALID_PARAMETER_VALUE.AES_CRYPTO_ERROR (wrong key, corrupted
    or truncated ciphertext, empty GCM input, wrong AAD).
  • Successful decrypts match aes_decrypt byte for byte, for all modes and key sizes.
  • NULL in any argument gives NULL.
  • TryEval wraps the argument expressions too, so a runtime error inside an argument (for example an ANSI cast
    failure in a CAST with ANSI on, or raise_error(...) passed as key) is also swallowed. In
    DataFusion, arguments are evaluated before the UDF runs, so this cannot be reproduced inside the UDF. Document the
    difference or handle it at plan-conversion time.
  • AAD with ECB is ignored, not an error (same as aes_decrypt).
  • Literal and column for each argument, with a mix of good and bad rows in one batch (bad rows NULL, good rows
    decrypted).
  • Deterministic.
Differences across Spark versions
  • Not available in 3.4. Added in 3.5.0 with CBC and AAD support already present.
  • 4.0: collated mode / padding accepted. No behavior change through 4.2.
Implementation notes
  • Spark: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala (TryAesDecrypt),
    expressions/TryEval.scala, and
    sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java (aesInternal).
  • DataFusion core has no AES functions.
  • Implement on top of the aes_decrypt kernel (see the aes_decrypt issue). Per-row errors become nulls instead
    of failing the batch. Existing try_* functions in datafusion-spark (try_to_binary, try_url_decode,
    try_parse_url) show the pattern.
  • Module: crates/spark/src/function/misc/.
  • Tests: add crates/sqllogictest/test_files/misc/try_aes_decrypt.slt; document with #[user_doc].

This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.

主要言語
Rust
スター
7
フォーク
15
平均マージ
16時間 43分
マージ済み PR(30日)
26

環境構築

はじめの一歩

  1. issue を最後まで読み、次にプロジェクトのコントリビューションガイドを読みます。
  2. 着手することを issue にコメントします — 二人が同じ作業をするのを防げます。
  3. リポジトリをフォークし、ブランチを切って変更します。
  4. issue 番号を参照したプルリクエストを送ります。

apache/datafusion-spark のほかの issue

apache/datafusion-spark の issue をすべて見る

似ている issue

Rust の issue をもっと見る

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。