Implement Spark-compatible `regexp_instr`

Start by reading how an existing string function is registered and documented in crates/spark/src/function/string/, then compare with DataFusion core's datafusion/functions/src/regex/regexpinstr.rs and Spark's RegExpInStr in regexpExpressions.scala. The open question is the shared Java-regex compatibility helper proposed for rlike, since the Rust regex crate cannot be wrapped as-is. Done means regexp_instr is registered with #[user_doc], converts the byte offset to a UTF-16 offset, and crates/sqllogictest/test_files/string/regexp_instr.slt covers the NULL, idx, empty-pattern and invalid-pattern cases above.

Part of #174 and apache/datafusion-comet#7300.

What the function does

regexp_instr(str, regexp[, idx]) returns the 1-based position where the first match of the Java regular expression regexp begins in str, or 0 if there is no match. Spark computes Matcher.find() then start() + 1. Available since 3.4.0.

str and regexp are implicitly cast to STRING and the optional third argument idx to INT (default 0). idx is accepted but ignored by the evaluation, although a NULL idx still makes the result NULL. Result is INT. NULL if any argument is NULL.

Examples

SELECT regexp_instr(r"\abc", r"^\\abc$");
-- 1
SELECT regexp_instr('[email protected]', '@[^.]*');
-- 5
SELECT regexp_instr('abc', 'b');
-- 2
SELECT regexp_instr('abc', 'x');
-- 0
SELECT regexp_instr('ABC', '(?-i)b');
-- 0
SELECT regexp_instr('1a 2b 14m', '\\d{2}(a|b|m)');
-- 7
SELECT regexp_instr('abc', null);
-- NULL
SELECT regexp_instr('abc', col0, 1) FROM VALUES(') ?') AS t(col0);
-- error INVALID_PARAMETER_VALUE.PATTERN

Edge cases to test

• NULL str, NULL regexp, NULL idx (literal and column): NULL.
• idx values do not change the result: regexp_instr('abc', 'b', 5) and regexp_instr('abc', 'b', -1) both return 2 (no group-index check is performed).
• Empty pattern: matches at position 0, so the result is 1 (also for str = '').
• The position is Matcher.start() + 1, a UTF-16 code unit index, not a code point index. regexp_instr('😀a', 'a') returns 3 in Spark (the emoji is a surrogate pair). DataFusion's regexp_instr counts characters and returns 2; a byte-offset port would return 5.
• Invalid pattern: INVALID_PARAMETER_VALUE.PATTERN, message The value of parameter(s) `regexp` in `regexp_instr` is invalid: ', ('. (RegexpExpressionsSuite), for literal and per-row patterns.
• Non-literal pattern columns that change per row.
• spark.sql.parser.escapedStringLiterals (default false): the first example uses raw literals; with the config on, '\abc' and '^\\abc$' reach the function unchanged. In .slt, DataFusion keeps backslashes literal.
• Java vs Rust divergences to decline or test: lookaround / backreferences / atomic / possessive (Java only); ASCII \d \w \s in Java vs Unicode in Rust (regexp_instr('\u06631', '\\d'), U+0663 ARABIC-INDIC DIGIT THREE, is 2 in Spark); . excludes \r, \u0085, \u2028, \u2029 in Java; $ matches before a final line terminator in Java (regexp_instr('ab\n', 'b$') is 2); (?i) ASCII-only in Java; inline flag letters (U, u, d) and the (?-i) example; Java-only \Q..\E, \h, \R, \Z, octal; Rust-only (?P<n>), \<, [[:alpha:]].
• UTF8_LCASE collation (4.0+) matches case-insensitively.

Differences across Spark versions

• 4.0: collation-aware input types. Otherwise none found (3.4.0 through 4.2.0).

Implementation notes

• Spark: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala, RegExpInStr (extends RegExpExtractBase; catches IllegalStateException and returns 0).
• DataFusion core regexp_instr(str, regexp[, start[, N[, flags[, subexpr]]]]) (datafusion/functions/src/regex/regexpinstr.rs) returns BIGINT, treats the third argument as a start position, counts code points, and uses Rust regex syntax. It cannot be wrapped as-is.
• Use the shared Java-regex compatibility helper proposed for rlike (verified subset, decline otherwise, decided at planning time for literal patterns). Convert the Rust byte offset to a UTF-16 offset (count code points, adding 1 extra for each code point above U+FFFF) before adding 1.
• Belongs in crates/spark/src/function/string/. Add crates/sqllogictest/test_files/string/regexp_instr.slt and document with #[user_doc].

This issue was generated by an LLM (Claude) from the Apache Spark source code and tests. Examples come from Spark's ExpressionDescription examples, test suites and .sql.out golden files, but please verify them against Spark before relying on them.