Skip to content

REPL.3: eager extension init off the prompt thread + init observability - #166

Draft
fupelaqu wants to merge 1 commit into
mainfrom
feature/REPL.3
Draft

REPL.3: eager extension init off the prompt thread + init observability#166
fupelaqu wants to merge 1 commit into
mainfrom
feature/REPL.3

Conversation

@fupelaqu

@fupelaqu fupelaqu commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

REPL.3 — Async DuckDB warm-up + init observability (elasticsql side)

Refs #163 (fixes 1+2 of the issue: the async warm-up trigger point + eager init; #163 stays open — REPL.5 closes it).

What

The first query of ANY kind used to pay the extension ServiceLoader scan + per-extension initialize() (~442ms measured over a 278-jar REPL classpath), and the first JOIN additionally paid the silent ~1.5s DuckDB native-library load. This PR moves the scan to REPL/CLI startup, off the prompt thread, and gives all three init windows duration logs. The DuckDB warm-up itself lands in softclient4es-arrow (sibling PR).

  • ExtensionApi.initializeExtensions()(implicit ec) — default method eagerly forcing extensionRegistry.extensions on the given EC. Additive: the lazy vals stay the single init path; a racing first query blocks on the lazy-val monitor (single scan, no double-init); a FAILED lazy init is not cached and retries on next access.
  • ElasticClientDelegator.initializeExtensions forwards to the delegate — the wrapper's own inherited registry is a second, unused one (run delegates to delegate.run, which consults the delegate's registry). Without the forwarding, Cli's eager init would double-initialize brand-new extension instances into a registry no query ever reads.
  • ExtensionRegistry — per-extension initialize() durations + total Extension discovery + initialization completed in …ms (N extension(s) active) log. Behaviour otherwise identical (same discovery, same Left ⇒ drop semantics, same priority sort).
  • Cli — fire-and-forget gateway.initializeExtensions() after gateway creation, before REPL/batch dispatch; 🔌 N extension(s) ready in …ms on completion (slf4j, never REPL stdout). Runs in ALL modes; in batch mode the scan overlaps the statement instead of preceding it.
  • NEW ExtensionEagerInitSpec (count via core's own two META-INF extensions + the delegator-forwarding trap).
  • Version 0.20.10.20.2-SNAPSHOT (project-lead kickoff decision, 0.20.x patch train).

AC coverage

  • AC 3 (prompt immediate): fire-and-forget — no await anywhere on the prompt thread.
  • AC 4 (--no-extensions / no arrow classes): no arrow/DuckDB reference in core; with no extension jars the scan just finds the two core extensions.
  • AC 5 (logged windows): Extension discovery + initialization completed in …ms, N extension(s) ready in …ms (+ 🦆 DuckDB warm-up … from the arrow side).
  • AC 1/2/6: arrow-side warm-up + integration matrix — see sibling PR.

Verification

  • sbt "+ sql/compile" "+ core/compile" (2.12 + 2.13) green; core/testOnly *ExtensionEagerInitSpec* 2/2; headerCheck, scalafmtCheck green.
  • sbt "+ publishLocal" for the arrow build (0.20.2-SNAPSHOT).

Notes for reviewers

  • Expected conflict: the build.sbt version line (0.20.2-SNAPSHOT) trivially conflicts with the sibling REPL.x PRs — each bumps the same line (per the epic kickoff decision).
  • Fresh-worktree + publishLocal gotcha hit during dev: the copyBridge/sources race published an EMPTY es7 bridge 2.12 jar to ivy local twice; recovery = + compile once, then sbt "+ clean" "+ publishLocal" (recorded in memory).

🤖 Generated with Claude Code

Part of epic #169 (REPL Hardening).

…vability (REPL.3)

- ExtensionApi.initializeExtensions(): default method forcing extension
  discovery + initialization on a caller-supplied ExecutionContext instead
  of the first query (~442ms ServiceLoader scan over a 278-jar REPL classpath)
- ElasticClientDelegator forwards initializeExtensions() to the delegate —
  the wrapper's inherited registry is a second, unused one; run() consults
  the delegate's registry, so eager init must warm THAT one
- ExtensionRegistry: per-extension initialize() and total scan durations
  logged (grep-able 'Extension discovery + initialization completed in …ms')
- Cli: fire-and-forget gateway.initializeExtensions() after gateway creation,
  before REPL/batch dispatch — prompt appears immediately, extension-side
  warm-ups (e.g. arrow JoinExtension's DuckDB native load) start at startup
- New ExtensionEagerInitSpec covering the count and the delegator forwarding
- Version 0.20.1 -> 0.20.2-SNAPSHOT

Refs #163 (fixes 1+2: async warm-up trigger point + eager init; the arrow-side
DuckDBWarmup lands in softclient4es-arrow)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant