Hacktoberfest 2026: le issue che i maintainer hanno segnato per ottobre, aperte e adatte ai principianti. Sfoglia le issue Hacktoberfest

Avoid per-row pattern allocation in regexp_is_match for repeated patterns

Aperta
  • Qualcosa non va?
#11,083 1 commento 0 reazioni 0 assegnatari Vedi su GitHub

I maintainer di solito rispondono entro 1 giorno

Nessuno ha ancora preso questa issue.

Valutazione

Difficoltà
3/5
Tempo stimato
1-2 giorni
Idoneità per principianti
72/100
Tipo di issue
Refactoring
Chiarezza
Specificata chiaramente
Stato di attività
Attiva
Stack tecnologico
rust
Ambito
data, performance

Direzione di ricerca

Start at the regexp_is_match array/array kernel and compare it with regexp_is_match_scalar, focusing on cache lookup and iterator specialization. Extend coverage for repeated patterns, flags, null and empty patterns across Utf8, LargeUtf8, and Utf8View, then run the existing regexp_kernels bench and add a repeated-pattern regexp_is_match benchmark. Done means the focused kernel avoids per-row key allocation while preserving current output.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Descrizione

Is your feature request related to a problem or challenge?

regexp_is_match already caches compiled Regex values in a HashMap<String, Regex>, but it builds an owned String for every row before consulting that cache:

  • no flags: pattern.to_string()
  • with flags: format!("(?{flag}){pattern}")

Cache misses also get and then entry, hashing the key twice. For a low-cardinality pattern column (one or a few patterns applied to many rows) Regex compilation is already avoided, but the per-row allocation and copy remain. On short strings that fixed cost dominates.

The no-flags path is also boxed as Box<dyn Iterator<Item = Option<String>>>, which is unnecessary.

regexp_is_match_scalar already compiles once and does not have this overhead. The array/array kernel should be able to get much closer to that for repeated patterns.

Describe the solution you'd like

A focused change to regexp_is_match only:

  • No-flags path: HashMap<&str, Regex> so a cache hit allocates nothing.
  • Flags path: borrowed key (&str, Option<&str>); build (?flags)pattern only on Vacant.
  • Use HashMap::entry for a single lookup.
  • Specialize the no-flags path instead of a boxed iterator.
  • Do not change regexp_match or the scalar kernel in the same PR (regexp_match is dominated by result construction; a borrowed-key prototype did not help there).
  • Tests for repeated patterns, repeated flags, null/empty patterns, and Utf8 / LargeUtf8 / Utf8View.

Independent remeasurement on arm64 macOS against 381eea177, 65,536 one-byte values, one repeated pattern. Prototype output matched the current kernel.

Case Current Borrowed-key prototype Speedup
repeated pattern, no flags 1.906 ms 1.033 ms 1.85×
repeated pattern + flags "i" 2.980 ms 1.556 ms 1.91×
regexp_is_match_scalar (same pattern) 250 µs — current array path is 7.6× slower than scalar
Describe alternatives you've considered
  • Interning owned String keys. That still allocates on the first insert of each distinct pattern, and hits still have to construct the lookup String.
  • A small linear cache instead of HashMap. Fine for tiny cardinality, but HashMap<&str, Regex> is already the natural fit and matches the current design.
  • Folding regexp_match into the same PR. Local checks showed no benefit; keep that kernel separate.
Additional context

#5235 already removed per-row Regex clones. #5246 added a scalar path for regexp_match. Neither addresses the per-row String allocated before the cache lookup in regexp_is_match.

The in-tree regexp_kernels bench only covers regexp_match, not regexp_is_match. A PR should add a repeated-pattern regexp_is_match bench.

Lingua principale
Rust
Stelle
3.6k
Fork
1.3k
Merge medio
3g 10h
PR unite (30g)
148

Preparare l'ambiente

Come iniziare

  1. Leggi tutta la issue e poi la guida ai contributi del progetto.
  2. Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
  3. Fai un fork del repository e lavora su un branch.
  4. Apri una pull request che faccia riferimento al numero della issue.

Altre issue di apache/arrow-rs

Tutte le issue di apache/arrow-rs

Issue simili

Altre issue su Rust

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.