Implement Spark-compatible `aes_decrypt`

Abierto
  • ¿Algo no está bien?
#296 0 comentarios 0 reacciones 0 asignados Ver en GitHub

Los mantenedores suelen responder en 1 día

Nadie ha tomado este issue todavía.

Evaluación

Dificultad
4/5
Tiempo estimado
3-5 días
Aptitud para principiantes
45/100
Tipo de issue
Nueva funcionalidad
Claridad
Bien especificado
Estado de actividad
Activo
Stack tecnológico
rust, spark
Área
cryptography, databases

Línea de trabajo

The work belongs in crates/spark/src/function/misc/, beside the existing AES code; first read Spark's AesDecrypt in misc.scala and aesDecrypt in ExpressionImplUtils.java for the exact argument layout and error mapping. Reuse a helper shared with aes_encrypt/try_aes_decrypt, wiring RustCrypto aes, aes-gcm, cbc and PKCS7 padding. Add crates/sqllogictest/test_files/misc/aes_decrypt.slt covering the listed examples, NULLs, key lengths and mode/padding errors; done when those queries return Spark's bytes and sqlStates.

Escrito por el modelo de indexación a partir del texto del issue.

Descripción

enhancement

Part of #272 and apache/datafusion-comet#7300.

What the function does

aes_decrypt(expr, key[, mode[, padding[, aad]]]) decrypts a value produced by aes_encrypt and returns BINARY.
expr, key and aad are BINARY; mode and padding are STRING (collated strings accepted from 4.0).
ImplicitCastInputTypes casts string arguments for expr, key and aad to their UTF-8 bytes. Defaults:
mode = 'GCM', padding = 'DEFAULT', aad = ''. Since 3.3.0 (aad since 3.5.0). NULL in any argument gives NULL.
There is no iv argument: the IV is read from the input.

Supported (mode, padding) pairs (case-insensitive): ECB + PKCS/DEFAULT, CBC + PKCS/DEFAULT,
GCM + NONE/DEFAULT. The key must be 16, 24 or 32 bytes. Expected input layout: ECB is the ciphertext only;
CBC is iv (16) || ciphertext; GCM is iv (12) || ciphertext || tag (16), with the 128-bit tag verified.
The result is the raw plaintext bytes. Use CAST(... AS STRING) to get text (the ExpressionDescription shows the
bytes as text).

Examples

From the ExpressionDescription, DataFrameFunctionsSuite and MiscFunctionsSuite:

SELECT cast(aes_decrypt(unhex('83F16B2AA704794132802D248E6BFD4E380078182D1544813898AC97E709B28A94'), '0000111122223333') AS STRING);
-- Spark
SELECT cast(aes_decrypt(unhex('6E7CA17BBB468D3084B5744BCA729FB7B2B7BCB8E4472847D02670489D95FA97DBBA7D3210'), '0000111122223333', 'GCM') AS STRING);
-- Spark SQL
SELECT cast(aes_decrypt(unbase64('3lmwu+Mw0H3fi5NDvcu9lg=='), '1234567890abcdef', 'ECB', 'PKCS') AS STRING);
-- Spark SQL
SELECT cast(aes_decrypt(unbase64('2NYmDCjgXTbbxGA3/SnJEfFC/JQ7olk2VQWReIAAFKo='), '1234567890abcdef', 'CBC') AS STRING);
-- Apache Spark
SELECT cast(aes_decrypt(unbase64('AAAAAAAAAAAAAAAAAAAAAPSd4mWyMZ5mhvjiAPQJnfg='), 'abcdefghijklmnop12345678ABCDEFGH', 'CBC', 'DEFAULT') AS STRING);
-- Spark
SELECT cast(aes_decrypt(unbase64('AAAAAAAAAAAAAAAAQiYi+sTLm7KD9UcZ2nlRdYDe/PX4'), 'abcdefghijklmnop12345678ABCDEFGH', 'GCM', 'DEFAULT', 'This is an AAD mixed into the input') AS STRING);
-- Spark
SELECT cast(aes_decrypt(unbase64('jmTOhz8XTbskI/zYFFgOFQ=='), 'abcdefghijklmnop', 'ECB') AS STRING);
-- (empty string)
SELECT cast(aes_decrypt(unbase64('9J3iZbIxnmaG+OIA9Amd+A=='), binary('abcdefghijklmnop12345678ABCDEFGH'), 'ECB') AS STRING);
-- Spark
SELECT aes_decrypt(cast(null as binary), 'abcdefghijklmnop');
-- NULL
Edge cases to test
  • NULL in each argument gives NULL. String vs binary key give the same result.
  • Key length not 16/24/32 (0, 15, 17): INVALID_PARAMETER_VALUE.AES_KEY_LENGTH (sqlState 22023), parameters
    `key`, `aes_encrypt`/`aes_decrypt`, actualLength.
  • Unsupported pairs ('GSM' with the default padding reports padding = DEFAULT; 'GCM','PKCS'; 'ECB','None';
    'CBC','NoPadding'): UNSUPPORTED_FEATURE.AES_MODE (sqlState 0A000).
  • Wrong key for ECB: INVALID_PARAMETER_VALUE.AES_CRYPTO_ERROR, parameter `expr`, `key`, detailMessage
    "Given final block not properly padded. Such issues can arise if a bad key is used during decryption."
  • Corrupted GCM ciphertext or wrong AAD: AES_CRYPTO_ERROR with "Tag mismatch" (JDK may append !).
  • Truncated CBC input: AES_CRYPTO_ERROR with a message starting "Input length". Empty GCM input:
    AES_CRYPTO_ERROR with "Invalid buffer arguments" (input shorter than the IV). The JCE messages are not
    reproducible in Rust; match the error condition and parameters, not detailMessage.
  • AAD with CBC: UNSUPPORTED_FEATURE.AES_MODE_AAD, and the message names `aes_encrypt` even when decrypting.
    AAD with ECB is silently ignored on decrypt (the ECB branch never looks at aad, by code reading).
  • Round trips through aes_encrypt for all modes and key sizes, empty plaintext, plaintext longer than one block,
    and multi-byte UTF-8 (results are bytes, not validated as UTF-8).
  • Literal and column for each argument, including per-row mode / key.
  • Deterministic.
Differences across Spark versions
  • 3.4: only ECB and GCM, no aad. A too-short GCM input throws an unmapped IllegalArgumentException, because
    only GeneralSecurityException maps to AES_CRYPTO_ERROR.
  • 3.5: adds CBC, aad, UNSUPPORTED_FEATURE.AES_MODE_AAD, and maps IllegalArgumentException too.
  • 4.0: collated mode / padding accepted. No behavior change through 4.2.
Implementation notes
  • Spark: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala (AesDecrypt,
    RuntimeReplaceable over StaticInvoke) and
    sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/ExpressionImplUtils.java (aesDecrypt,
    aesInternal). Errors in sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala.
  • DataFusion core has no AES functions.
  • Suggested crates: RustCrypto aes, aes-gcm, cbc, ecb + block-padding (PKCS7).
  • Put it in crates/spark/src/function/misc/ with a helper module shared with aes_encrypt and try_aes_decrypt.
    The helper should return a typed error so try_aes_decrypt can turn every failure into NULL.
  • Tests: add crates/sqllogictest/test_files/misc/aes_decrypt.slt; document with #[user_doc].

This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.

Lenguaje dominante
Rust
Estrellas
7
Forks
15
Merge medio
16 h 43 min
PR fusionados (30 d)
26

Preparar el entorno

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Más de apache/datafusion-spark

Todos los issues de apache/datafusion-spark

Issues similares

Más issues de Rust

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.