Skip to content

Add deterministic Electron E2E and harden local model attachments - #19

Merged
sambitcreate merged 64 commits into
mainfrom
feature/playwright-e2e-lmstudio-attachments
Aug 11, 2026
Merged

Add deterministic Electron E2E and harden local model attachments#19
sambitcreate merged 64 commits into
mainfrom
feature/playwright-e2e-lmstudio-attachments

Conversation

@sambitcreate

Copy link
Copy Markdown
Owner

Summary

  • add a hermetic Playwright Electron harness with six deterministic workflows and a separate opt-in live LM Studio vision acceptance
  • make LM Studio and Ollama onboarding discover and persist canonical local-provider models, including migration of released onboarding identities
  • support Finder drop and clipboard-image attachments through fixed-purpose preload channels
  • enforce descriptor-bound reads, process-owned ingestion admission, exact raster MIME/signature validation, and authoritative append parsing
  • gate both pull-request CI and release builds on the deterministic Electron suite

Why

The app had no repeatable application-level Electron acceptance gate. Local-provider onboarding could save stale identities or choose incomplete catalogs, and drag/drop or clipboard attachments lacked a safe end-to-end bridge. The new suite reproduces those workflows without requiring LM Studio on GitHub runners, while the production changes close the provider-migration, attachment-race, and payload-validation gaps found during three independent final audits.

CI and LM Studio

The default npm run test:e2e suite starts a test-owned LM Studio-compatible server on a random loopback port. GitHub CI does not need LM Studio installed, a downloaded model, or port 1234.

Real visual inference remains opt-in through npm run test:e2e:live:lmstudio and is not part of CI or release workflows.

Validation

  • npm test
  • npm run test:e2e — 6/6 deterministic Electron workflows
  • npm run test:e2e:live:lmstudio — live vision model read a per-run image token
  • focused attachment matrix — 76/76
  • focused provider/model suites — green
  • npm run type-check
  • npm run type-check:e2e
  • npm run lint
  • npm audit --audit-level=low — 0 vulnerabilities
  • git diff --check
  • workflow YAML parse
  • npm run package
  • npm run package:verify

Review notes

The branch contains 47 commits for 47 changed files, with exactly one file per commit. The pre-integration local safety stash remains untouched.

…fore spawn (macOS posix_spawnp fix)

node-pty 1.1.0's npm prebuilt tarball restores spawn-helper without its
execute bit (0644). posix_spawn of a non-executable file is exactly what
surfaces to users as 'posix_spawnp failed.' the first time they open the
terminal drawer.

TerminalService.ensureSpawnHelperExecutable now resolves every prebuilds/*
helper (including app.asar.unpacked), chmods only when the execute bit is
missing, and verifies afterward — throwing a path-bearing remediation error
instead of silently swallowing (the prior catch {}). Shell resolution now
verifies the candidate is executable with a fallback chain (/bin/zsh ->
/bin/zsh -> /bin/bash -> /bin/sh) so a stale SHELL can no longer break
terminal creation.
Packaged builds must never ship spawn-helper without its execute bit. The
afterPack hook now walks app.asar.unpacked/.../node-pty/prebuilds/*,
chmods each spawn-helper to 0755, and throws in CI if any is missing or
still non-executable — so a broken package fails at build time rather than
at the user's first terminal open.
…ation

Adds TerminalHistoryStore: a debounced, line-capped, per-workspace log of
terminal output rooted at <userData>/terminal-history. Output is sanitized
before persisting so a replayed snapshot cannot trigger fresh shell replies
— CSI cursor-position reports, device-attributes/status queries, DECRQM/PM,
XTVERSION, Kitty keyboard, DCS DECRQSS/XTGETTCAP, and OSC color queries are
stripped while benign SGR/cursor sequences survive. Partial sequences split
across chunks are carried via a pending prefix.

Ported from t3code's sanitizeTerminalHistoryChunk (Manager.ts:953).
Shell fallback: the spawn path now walks an executable candidate list
($SHELL -> /bin/zsh -> /bin/bash -> /bin/sh) and retries the next on a
retryable failure (posix_spawnp failed, ENOENT, not found). A broken $SHELL
self-heals instead of throwing. Non-retryable errors (EINVAL, out of fds)
surface immediately. The session result gains resolvedShell and
preferredShellSkipped so the renderer can tell the user which shell launched.

History wiring: TerminalService now accepts an optional historyStore. On open
the prior sanitized output seeds the buffer (the renderer re-hydrates xterm
from snapshot, so no renderer change is needed for the seed); each PTY data
event appends to the store; terminate/exit flush the final chunk.
…hell refactor

The phase3 contract test asserts the terminal.ts source orders revalidate →
abort-check → spawn → abort-check. The spawn call changed shape (single
spawn → trySpawnShell destructure) in the shell-fallback PR; update the
assertion to match while preserving the ordering invariant it protects.
@sambitcreate
sambitcreate marked this pull request as ready for review August 11, 2026 15:34
…history

feat(terminal): shell fallback retry + persisted sanitized history

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — 47 files adding deterministic E2E Electron tests, hardening attachment ingestion with process-owned admission and raster signature validation, migrating released onboarding provider identities, and wiring drag/drop + clipboard image attachment through a fixed-purpose preload bridge.

  • E2E harness — 6 deterministic Playwright Electron workflows against a test-owned mock LM Studio server, hermetic environment isolation validated at launch, CI + release gating.
  • Attachment admissionAttachmentIngestionAdmission enforces per-document serialization and global concurrency/byte budgets. Picker, drop, and clipboard paths all route through runOwnedAttachmentIngestion with isActive guards.
  • Raster validationimageBytesMatchMime verifies PNG/JPEG/GIF/WebP/BMP/HEIC/HEIF file signatures. New appends reject SVG and mismatched MIME/bytes; stored/legacy mode preserves backward compat.
  • Preload bridgecreateAttachmentPreloadBridge uses only webUtils.getPathForFile for drop paths; clipboard payloads are validated before IPC transit.
  • Onboarding — local providers are discovered and tested before saving; model cache and selection are persisted inline. Focus target class prevents stale button refs after editor close.
  • Provider migrationcustom:onboarding-lmstudio/custom:onboarding-ollama migrate to canonical IDs with collision-safe numeric siblings. Recognized by isLmStudioProviderId/isOllamaProviderId regex.
  • Model discovery — Ollama tags + detail requests share one AbortSignal with bounded timeout. LM Studio response surfaces recommendedModel from first loaded instance.
  • Composer — drag-over, drop, and paste handlers feed the fixed attachment bridge. attachmentOperationRef guard with finally cleanup prevents concurrent ingestion.
  • Contract tests — 14 new/updated source-reading and IPC-contract tests enforce channel allowlists, preload key shapes, exact-key parsing, and raster admission rules.
  • Unit + integration tests — attachment admission (per-doc, global count, representation budget), clipboard payloads (detached bytes, too-large, mismatch-signature), Ollama timeout shared signal, onboarding identity migration, icon resolution for numeric siblings, Google provider localStorage migration through aliases.

Pullfrog  | View workflow run | Using DeepSeek Pro𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — 8 new commits adding terminal shell retry/fallback, a per-workspace history store with control-sequence sanitization, build-time and runtime spawn-helper executable fixes, a renderer toast for fallback shells, and IPC contract additions.

  • Shell retry looptrySpawnShell tries ordered candidates ($SHELL, /bin/zsh, /bin/bash, /bin/sh); retryable failures (e.g. posix_spawnp failed) fall through to the next candidate; non-retryable errors (EINVAL) surface immediately.
  • Spawn-helper executable fixes — runtime ensureSpawnHelperExecutable now enumerates all prebuild dirs and .asar.unpacked mirrors; build-time makeSpawnHelpersExecutable in configureElectronFuses runs at afterPack. Both paths chmod + verify with descriptive errors.
  • Terminal history storeTerminalHistoryStore persists per-workspace sanitized output to disk with debounced writes; sanitizeTerminalHistoryChunk strips device-query/reply sequences (CSI/DCS/OSC) across chunk boundaries while preserving benign SGR colors.
  • IPC contractTerminalSession gains resolvedShell and preferredShellSkipped; TerminalHistoryStoreLike structural interface keeps the service testable.
  • Renderer toastterminal-drawer.tsx surfaces a one-time toast when a fallback shell launched the terminal.
  • Tests — 27 additional unit tests for shell retry, spawn-helper chmod/verify, history sanitization, round-trip, isolation, clear, and debounce coalescing. 5 additional build-script tests for makeSpawnHelpersExecutable. All pass.

ℹ️ Deferred wiring observation

The TerminalHistoryStore class and all its wiring hooks in TerminalService (onData, onExit, terminate) are complete and tested, but the store is not yet activated in production: the terminalService singleton is new TerminalService() with no historyStore option, and TerminalHistoryStore.create() is never called. The feature is fully implemented — the ?. guards ensure no crash — but has no runtime effect until explicitly wired.

Technical details
# Deferred wiring

## Affected sites
- `main/services/terminal-history.ts:284``static async create()` is never called in production
- `main/services/terminal.ts:519``new TerminalService()` produces a service with no `historyStore`
- `main/services/terminal.ts:317,330,403` — all three hooks use `?.` guards correctly

## Required outcome
- Call `TerminalHistoryStore.create()` at app startup and pass it to the `TerminalService` constructor, or confirm the infrastructure-first landing is intentional and wiring will follow in a separate PR.

## Open questions for the human
- Should the `pending` map in `TerminalHistoryStore` evict entries for workspaces that haven't seen a terminal in some time window? Currently `flush()` persists but doesn't remove the in-memory entry, and `clear()` is never called from production code.

Pullfrog  | View workflow run | Using DeepSeek Pro𝕏

@pullfrog pullfrog Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ No new issues found.

Reviewed changes — the incremental delta resolves the deferred wiring observation from the prior review and hardens the history store's internal state management.

  • Production wiringTerminalHistoryStore is created at app.whenReady() and installed into terminalService; a flushHistory() call is added to shutdownAndQuit. This directly addresses the prior review's observation that the store had no runtime effect.
  • Character-bound capcapHistory now enforces MAX_HISTORY_CHARS (200,000) as a hard upper bound alongside the 5,000-line bound. Histories without line breaks are no longer unbounded.
  • Revision trackingPendingWorkspaceState gains revision, persistedRevision, and writeInFlight fields. persist() serializes writes per workspace and reschedules a debounced write when new output arrives during an in-flight write.
  • flush() rewrite — uses a retry loop that cancels scheduled writes, calls persist(), and repeats until all revisions are caught up. flushAll() flushes all workspaces concurrently for shutdown.
  • read() state initialization — newly populates the pending map, so a read() + append() + flush() sequence works without an explicit first append().
  • Onboarding copy — the terminal feature bento description now mentions sanitized local history.
  • Tests — 5 new tests: character-bound cap, restart-append, flushAll settling, flushAll write-race, and a source-reading production-wiring assertion.

Pullfrog  | View workflow run | Using DeepSeek Pro𝕏

@sambitcreate
sambitcreate merged commit 403175b into main Aug 11, 2026
4 checks passed
@sambitcreate
sambitcreate deleted the feature/playwright-e2e-lmstudio-attachments branch August 12, 2026 04:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant