Skip to content

fix(store): escape interior double-quotes in sanitizeFTS to prevent FTS5 crash#585

Closed
blak0p wants to merge 1 commit into
Gentleman-Programming:mainfrom
blak0p:fix/fts5-interior-quote
Closed

fix(store): escape interior double-quotes in sanitizeFTS to prevent FTS5 crash#585
blak0p wants to merge 1 commit into
Gentleman-Programming:mainfrom
blak0p:fix/fts5-interior-quote

Conversation

@blak0p

@blak0p blak0p commented Jul 9, 2026

Copy link
Copy Markdown

🔗 Linked Issue

Closes #574


🏷️ PR Type

  • type:bug — Bug fix
  • type:feature — New feature
  • type:docs — Documentation only
  • type:refactor — Code refactoring (no behavior change)
  • type:chore — Maintenance, dependencies, tooling
  • type:breaking-change — Breaking change

📝 Summary

  • Fixes FTS5 search crash on queries containing an interior double-quote (e.g. hello"worldSQL logic error: unterminated string (1)).
  • Doubles interior " inside sanitizeFTS per the FTS5 string-literal escape rule (""), so hello"world"hello""world".
  • Adds TestSanitizeFTS with 8 cases (written first, failing per TDD; all pass after the fix).

🔧 Changes

File Change
internal/store/store.go Add strings.ReplaceAll(w, \"`, `""`)afterTriminsanitizeFTS` to escape interior double-quotes for FTS5.
internal/store/store_test.go Add TestSanitizeFTS covering plain words, multiple words, single/multiple interior quotes, already-quoted, just-quotes, mixed, and empty input.

🧪 Test Plan

  • New test TestSanitizeFTS fails on unpatched code (RED) and passes after the fix (GREEN):
    go test ./internal/store/ -run TestSanitizeFTS -v
  • Full store package passes (no regressions): go test ./internal/store/
  • Built binary no longer crashes: go build -o /tmp/engram-fix ./cmd/engram && /tmp/engram-fix search 'hello"world' → clean "No memories found" (exit 0).
  • Normal search still works: /tmp/engram-fix search 'sanitizeFTS' → returns results.

✅ Contributor Checklist

  • Linked an approved issue — note: issue [Bug] FTS5 search returns HTTP 500 on queries containing an interior double-quote (e.g. foo"bar) #574 does not currently carry status:approved; a maintainer label is required for the check-issue-approved CI job to pass.
  • Added exactly one type:* label (type:bug)
  • Ran shellcheck on modified scripts (no shell scripts modified)
  • Tests included with the fix
  • Docs updated if behavior changed (internal fix; no user-visible behavior change beyond crash removal)
  • Conventional commit format
  • No Co-Authored-By trailers

Summary by CodeRabbit

  • Bug Fixes
    • Improved search query handling so inputs with double quotes are processed safely, avoiding malformed searches.
  • Tests
    • Added coverage for quoted, unquoted, and empty search input cases to verify consistent behavior.

…TS5 crash

sanitizeFTS wrapped each whitespace-separated word in double-quotes for
FTS5 MATCH queries, but only stripped surrounding quotes via
strings.Trim(w, `"`). Interior quotes were left untouched, so an input
like `hello"world` produced `"hello"world"` — an unterminated FTS5 string
literal that crashed every search path (mem_search, candidate detection)
with "SQL logic error: unterminated string (1)". See issue Gentleman-Programming#574.

FTS5 escapes a literal double-quote inside a quoted phrase by doubling
it (`""`), so we apply strings.ReplaceAll(w, `"`, `""`) after the Trim.
Now `hello"world` correctly becomes `"hello""world"`.

Why this spot and not the query layer: sanitizeFTS is the single funnel
used by both search entry points (store.go:2171, store.go:2644), so the
fix covers all affected call sites without touching sanitizeFTSCandidates
(relations.go), which builds OR-based queries and has its own escaping.

Adds TestSanitizeFTS covering plain words, multiple words, single and
multiple interior quotes, already-quoted input, just-quotes, mixed input,
and the empty case. Written first (failing) per TDD; all 8 cases pass
after the one-line fix.
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b35895c7-be4c-4b0e-8058-4116994a99b8

📥 Commits

Reviewing files that changed from the base of the PR and between be4b613 and 9190069.

📒 Files selected for processing (2)
  • internal/store/store.go
  • internal/store/store_test.go

📝 Walkthrough

Walkthrough

The sanitizeFTS helper in the store package is updated to escape interior double quotes by doubling them before wrapping terms in quotes for FTS5 MATCH queries, preventing malformed SQL. A corresponding unit test validates the escaping across several input cases.

Changes

FTS Query Sanitization

Layer / File(s) Summary
Escape interior quotes and add test
internal/store/store.go, internal/store/store_test.go
sanitizeFTS now doubles interior double quotes in each term before quoting for FTS5 queries; a new test verifies expected output for empty, quoted, and interior-quote inputs.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related issues

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main fix: escaping interior double-quotes in sanitizeFTS to prevent FTS5 failures.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
⚔️ Resolve merge conflicts
  • Resolve merge conflict in branch fix/fts5-interior-quote

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Alan-TheGentleman

Copy link
Copy Markdown
Collaborator

Closing this because it was opened without a linked status:approved issue, and no PR merges without one. The fix itself is narrow and welcome. #574 is now approved, so reopen this PR with Closes #574 in the body and it will go into review. This is the normal process here, not a judgment on the code.

@blak0p

blak0p commented Jul 9, 2026

Copy link
Copy Markdown
Author

Reabrí el fix como PR #586 — mismo contenido, ahora con Closes #574 en el body. ¡Gracias!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] FTS5 search returns HTTP 500 on queries containing an interior double-quote (e.g. foo"bar)

2 participants