REPL.3: eager extension init off the prompt thread + init observability - #166
Draft
fupelaqu wants to merge 1 commit into
Draft
REPL.3: eager extension init off the prompt thread + init observability#166fupelaqu wants to merge 1 commit into
fupelaqu wants to merge 1 commit into
Conversation
…vability (REPL.3) - ExtensionApi.initializeExtensions(): default method forcing extension discovery + initialization on a caller-supplied ExecutionContext instead of the first query (~442ms ServiceLoader scan over a 278-jar REPL classpath) - ElasticClientDelegator forwards initializeExtensions() to the delegate — the wrapper's inherited registry is a second, unused one; run() consults the delegate's registry, so eager init must warm THAT one - ExtensionRegistry: per-extension initialize() and total scan durations logged (grep-able 'Extension discovery + initialization completed in …ms') - Cli: fire-and-forget gateway.initializeExtensions() after gateway creation, before REPL/batch dispatch — prompt appears immediately, extension-side warm-ups (e.g. arrow JoinExtension's DuckDB native load) start at startup - New ExtensionEagerInitSpec covering the count and the delegator forwarding - Version 0.20.1 -> 0.20.2-SNAPSHOT Refs #163 (fixes 1+2: async warm-up trigger point + eager init; the arrow-side DuckDBWarmup lands in softclient4es-arrow) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
REPL.3 — Async DuckDB warm-up + init observability (elasticsql side)
Refs #163 (fixes 1+2 of the issue: the async warm-up trigger point + eager init; #163 stays open — REPL.5 closes it).
What
The first query of ANY kind used to pay the extension
ServiceLoaderscan + per-extensioninitialize()(~442ms measured over a 278-jar REPL classpath), and the first JOIN additionally paid the silent ~1.5s DuckDB native-library load. This PR moves the scan to REPL/CLI startup, off the prompt thread, and gives all three init windows duration logs. The DuckDB warm-up itself lands insoftclient4es-arrow(sibling PR).ExtensionApi.initializeExtensions()(implicit ec)— default method eagerly forcingextensionRegistry.extensionson the given EC. Additive: the lazy vals stay the single init path; a racing first query blocks on the lazy-val monitor (single scan, no double-init); a FAILED lazy init is not cached and retries on next access.ElasticClientDelegator.initializeExtensionsforwards to the delegate — the wrapper's own inherited registry is a second, unused one (rundelegates todelegate.run, which consults the delegate's registry). Without the forwarding,Cli's eager init would double-initialize brand-new extension instances into a registry no query ever reads.ExtensionRegistry— per-extensioninitialize()durations + totalExtension discovery + initialization completed in …ms (N extension(s) active)log. Behaviour otherwise identical (same discovery, sameLeft⇒ drop semantics, same priority sort).Cli— fire-and-forgetgateway.initializeExtensions()after gateway creation, before REPL/batch dispatch;🔌 N extension(s) ready in …mson completion (slf4j, never REPL stdout). Runs in ALL modes; in batch mode the scan overlaps the statement instead of preceding it.ExtensionEagerInitSpec(count via core's own two META-INF extensions + the delegator-forwarding trap).0.20.1→0.20.2-SNAPSHOT(project-lead kickoff decision, 0.20.x patch train).AC coverage
--no-extensions/ no arrow classes): no arrow/DuckDB reference in core; with no extension jars the scan just finds the two core extensions.Extension discovery + initialization completed in …ms,N extension(s) ready in …ms(+🦆 DuckDB warm-up …from the arrow side).Verification
sbt "+ sql/compile" "+ core/compile"(2.12 + 2.13) green;core/testOnly *ExtensionEagerInitSpec*2/2;headerCheck,scalafmtCheckgreen.sbt "+ publishLocal"for the arrow build (0.20.2-SNAPSHOT).Notes for reviewers
build.sbtversion line (0.20.2-SNAPSHOT) trivially conflicts with the sibling REPL.x PRs — each bumps the same line (per the epic kickoff decision).+ publishLocalgotcha hit during dev: the copyBridge/sources race published an EMPTY es7 bridge 2.12 jar to ivy local twice; recovery =+ compileonce, thensbt "+ clean" "+ publishLocal"(recorded in memory).🤖 Generated with Claude Code
Part of epic #169 (REPL Hardening).