Is your feature request related to a problem or challenge?
While reviewing #23540 (optimizing regexp_instr), it was noted that the new RegexCache introduced there overlaps with the existing compile_and_cache_regex helper in datafusion/functions/src/regex/mod.rs.
The version added in #23540 is a superset: in addition to the HashMap keyed by (pattern, flags), it memoizes the previous row's pattern (last) so that the common case of a literal pattern skips the hash lookup entirely on every row after the first. This gave a meaningful part of the ~40% speedup in that PR.
Today only regexp_instr benefits from this fast path. The other regex functions (regexp_count, regexp_match, regexp_replace) still use the plain compile_and_cache_regex HashMap lookup.
Describe the solution you'd like
Extract the RegexCache pattern from regexp_instr into datafusion/functions/src/regex/mod.rs as a shared, reusable cache, and migrate the other regex functions onto it so they get the same last-pattern memoization:
regexp_count
regexp_match
regexp_replace
This would consolidate the two caching implementations into one and let all regex functions skip per-row hashing for literal patterns.
Describe alternatives you've considered
Leave the two implementations separate. This keeps mod.rs unchanged but duplicates the caching logic and leaves the other functions without the faster path.
Additional context
Follow-on from #23540. Suggested by @alamb in review:
Maybe we could (as a follow on PR) extract the common regexp caching functionality -- I personally prefer this Cache pattern -- for use in the other functions with regular expressions
Is your feature request related to a problem or challenge?
While reviewing #23540 (optimizing
regexp_instr), it was noted that the newRegexCacheintroduced there overlaps with the existingcompile_and_cache_regexhelper indatafusion/functions/src/regex/mod.rs.The version added in #23540 is a superset: in addition to the
HashMapkeyed by(pattern, flags), it memoizes the previous row's pattern (last) so that the common case of a literal pattern skips the hash lookup entirely on every row after the first. This gave a meaningful part of the ~40% speedup in that PR.Today only
regexp_instrbenefits from this fast path. The other regex functions (regexp_count,regexp_match,regexp_replace) still use the plaincompile_and_cache_regexHashMap lookup.Describe the solution you'd like
Extract the
RegexCachepattern fromregexp_instrintodatafusion/functions/src/regex/mod.rsas a shared, reusable cache, and migrate the other regex functions onto it so they get the same last-pattern memoization:regexp_countregexp_matchregexp_replaceThis would consolidate the two caching implementations into one and let all regex functions skip per-row hashing for literal patterns.
Describe alternatives you've considered
Leave the two implementations separate. This keeps
mod.rsunchanged but duplicates the caching logic and leaves the other functions without the faster path.Additional context
Follow-on from #23540. Suggested by @alamb in review: