diff --git a/.changeset/eql-v3-rename-contains-to-matches.md b/.changeset/eql-v3-rename-contains-to-matches.md new file mode 100644 index 00000000..2e576b46 --- /dev/null +++ b/.changeset/eql-v3-rename-contains-to-matches.md @@ -0,0 +1,30 @@ +--- +'@cipherstash/stack-drizzle': minor +'@cipherstash/stack-supabase': minor +--- + +Rename the EQL v3 encrypted free-text operator `contains()` → `matches()` (#617). + +Encrypted free-text search is fuzzy bloom-filter token matching — order- and +multiplicity-insensitive and one-sided (a `true` may be a false positive) — not +containment. The name `contains()` promised substring/containment semantics it +never had. It is renamed to `matches()` on the encrypted surface; `contains()` is +kept for genuine, exact containment: + +- **Drizzle** (`@cipherstash/stack-drizzle/v3`): `matches()` = bloom free-text on + `text_match`/`text_search` columns; `contains()` = exact encrypted-JSON `@>` on + `types.Json` (ste_vec) columns. +- **Supabase** (`@cipherstash/stack-supabase`): `.matches()` = encrypted free-text; + `.contains()` = native jsonb/array `@>` on plaintext columns (and throws on an + encrypted column, pointing to `matches()`). + +Also on the Supabase v3 surface, `like()`/`ilike()` on an encrypted column are no +longer rejected — they are delegated to `matches()` as a best-effort compatibility +shim. This is APPROXIMATE (fuzzy, case-insensitive, one-sided; anchoring and +wildcards are not honored): surrounding `%` are stripped, an internal `%` or any +`_` is rejected, and a one-time warning is emitted. A plaintext column keeps real +SQL LIKE. + +Breaking: encrypted `contains()` callers must migrate to `matches()`. The +encrypted operator has not shipped in a stable release (it lands via the EQL v3 +work), so there is no deprecation alias. diff --git a/.changeset/stash-skills-contains-to-matches.md b/.changeset/stash-skills-contains-to-matches.md new file mode 100644 index 00000000..88c709c4 --- /dev/null +++ b/.changeset/stash-skills-contains-to-matches.md @@ -0,0 +1,10 @@ +--- +'stash': patch +--- + +Update the `stash-drizzle` and `stash-supabase` skills for the EQL v3 +`contains()` → `matches()` rename (#617): the encrypted free-text operator is now +`matches()` (fuzzy bloom token matching), `contains()` is reserved for exact +containment, and Supabase `like()`/`ilike()` on encrypted columns are documented +as an approximate compatibility shim delegating to `matches()`. Skills ship inside +the `stash` tarball, so they must track the adapter surface. diff --git a/docs/reference/supabase-sdk.md b/docs/reference/supabase-sdk.md index e84b3f6e..6561f255 100644 --- a/docs/reference/supabase-sdk.md +++ b/docs/reference/supabase-sdk.md @@ -55,8 +55,10 @@ The builder surface is shared across v2 and v3: transforms (`.order/.limit/.range/.single/.maybeSingle/.csv/.abortSignal/.throwOnError`), plus `.withLockContext(lockContext)` and `.audit(config)` — with one fork: free-text search. v2 exposes `.like/.ilike` (SQL wildcard matching); v3 -exposes `.contains()` (token containment) and rejects `like`/`ilike` on -encrypted columns (see "v3 encoding details" below). +exposes `.matches()` (fuzzy bloom token search) on encrypted columns, keeps +`.contains()` for native (exact) containment on plaintext columns, and treats +`like`/`ilike` on an encrypted column as an approximate shim that delegates to +`.matches()` (see "v3 encoding details" below). ### Typing (v3) @@ -84,7 +86,7 @@ const es = await encryptedSupabaseV3( shape (schema columns get their domain plaintext types — `types.IntegerOrd` → `number`, `types.TimestampOrd` → `Date`, …). Storage-only columns (e.g. `types.Boolean`) are excluded from every filter method — including `.match()` — -at the type level, `contains()` is narrowed to match-indexed columns, and +at the type level, `matches()` is narrowed to match-indexed columns, and `order()` to plaintext columns; filtering a storage-only column is always a clear runtime error. Undeclared tables behave exactly as with no `schemas` at all: `from(tableName)` returns the untyped v3 builder, with `Row` @@ -205,14 +207,23 @@ These are internal to the adapter but explain observable behaviour. Envelopes proxies, and Supabase request logs. The remaining gap is PostgREST operand casting; until the adapter can reach the query domains (CIP-3402), operands keep carrying ciphertext. -- **Free-text search is `contains()`; `like`/`ilike` are rejected** on - encrypted columns with an error naming `contains()`. The v3 domains define - no LIKE operator — free-text search is bloom-filter token containment - (`contains()` → PostgREST `cs` → `@>`), where match is tokenized + - downcased and `%` is tokenized like any other character, so a `like` - pattern is a category error. On plaintext columns `like`/`ilike` (and - native `contains`) pass through unchanged. -- **`contains()` matches substrings.** The search term blooms to its own +- **Free-text search is `matches()`; `contains()` on an encrypted column is + rejected** with an error naming `matches()`. The v3 domains define no LIKE + operator — encrypted free-text search is fuzzy bloom-filter token matching + (`matches()` → PostgREST `cs` → `@>`), one-sided (a match may be a false + positive, a non-match never is) and order-/multiplicity-insensitive, where + match is tokenized + downcased and `%` is tokenized like any other character. + `contains()` is reserved for native (exact) jsonb/array containment on + plaintext columns, which pass through unchanged. +- **`like`/`ilike` on an encrypted column are an approximate compatibility + shim** that delegates to `matches()`: leading/trailing `%` are stripped and + the residual term is fuzzy-matched (emitting the same `cs` wire, with a + one-time warning). Because the match is case-insensitive, one-sided, and + anchoring is not honored, the result is APPROXIMATE. A pattern with an + internal `%` or any `_` cannot be approximated and throws; call `matches()` + directly to make the fuzzy intent explicit. On plaintext columns `like`/`ilike` + pass through unchanged as real SQL LIKE. +- **`matches()` matches substrings.** The search term blooms to its own trigrams, and a row matches when the stored value's bloom contains all of them — so any substring of at least 3 characters (the tokenizer's `token_length`) matches. Shorter terms bloom to nothing and would match every diff --git a/packages/stack-drizzle/__tests__/v3/operators.test.ts b/packages/stack-drizzle/__tests__/v3/operators.test.ts index 95ef0051..95a2f805 100644 --- a/packages/stack-drizzle/__tests__/v3/operators.test.ts +++ b/packages/stack-drizzle/__tests__/v3/operators.test.ts @@ -358,9 +358,9 @@ describe('createEncryptionOperatorsV3 - comparison & range', () => { describe('createEncryptionOperatorsV3 - free-text match', () => { it.each( matchDomains, - )('%s contains emits latest eql_v3.contains with a query-term operand', async (eqlType, spec) => { + )('%s matches emits latest eql_v3.contains with a query-term operand', async (eqlType, spec) => { const { ops, encryptQuery, render } = setup() - const q = render(await ops.contains(matrixColumn(eqlType), needleFor(spec))) + const q = render(await ops.matches(matrixColumn(eqlType), needleFor(spec))) expect(q.sql).toContain( `eql_v3.contains("matrix_users"."${slug(eqlType)}", $1::${qcast(eqlType)})`, @@ -377,20 +377,20 @@ describe('createEncryptionOperatorsV3 - free-text match', () => { // throw rather than silently return the whole table. it.each( matchDomains, - )('%s contains rejects a needle shorter than token_length before encrypting', async (eqlType) => { + )('%s matches rejects a needle shorter than token_length before encrypting', async (eqlType) => { const { ops, encryptQuery } = setup() - await expect(ops.contains(matrixColumn(eqlType), 'ad')).rejects.toThrow( + await expect(ops.matches(matrixColumn(eqlType), 'ad')).rejects.toThrow( /at least 3 characters/, ) - await expect(ops.contains(matrixColumn(eqlType), '')).rejects.toThrow( + await expect(ops.matches(matrixColumn(eqlType), '')).rejects.toThrow( EncryptionOperatorError, ) expect(encryptQuery).not.toHaveBeenCalled() }) - it('contains accepts a needle exactly at token_length', async () => { + it('matches accepts a needle exactly at token_length', async () => { const { ops, render } = setup() - const q = render(await ops.contains(users.email, 'ada')) + const q = render(await ops.matches(users.email, 'ada')) expect(q.sql).toContain( 'eql_v3.contains("users"."email", $1::eql_v3.query_text_search)', ) @@ -398,7 +398,7 @@ describe('createEncryptionOperatorsV3 - free-text match', () => { it('negation is expressed through the passthrough Drizzle not operator', async () => { const { ops, render } = setup() - const q = render(ops.not(await ops.contains(users.email, 'example.com'))) + const q = render(ops.not(await ops.matches(users.email, 'example.com'))) expect(q.sql).toMatch(/^not /i) expect(q.sql).toContain( 'eql_v3.contains("users"."email", $1::eql_v3.query_text_search)', @@ -714,17 +714,24 @@ describe('createEncryptionOperatorsV3 - gating errors', () => { ) }) - it('contains on a column without match throws', async () => { + it('matches on a column without a match index throws', async () => { const { ops } = setup() - await expect(ops.contains(users.nickname, 'ada')).rejects.toBeInstanceOf( - EncryptionOperatorError, + await expect(ops.matches(users.nickname, 'ada')).rejects.toThrow( + /free-text search/, + ) + }) + + it('contains on a column without JSON containment throws', async () => { + const { ops } = setup() + await expect(ops.contains(users.nickname, 'ada')).rejects.toThrow( + /JSON containment/, ) }) it('null operands throw and point callers to null checks', async () => { const { ops } = setup() await expect(ops.eq(users.nickname, null)).rejects.toThrow(/isNull/) - await expect(ops.contains(users.email, undefined)).rejects.toThrow(/isNull/) + await expect(ops.matches(users.email, undefined)).rejects.toThrow(/isNull/) }) it('eq on a storage-only column throws', async () => { diff --git a/packages/stack-drizzle/integration/adapter.ts b/packages/stack-drizzle/integration/adapter.ts index 496c49ff..524ad33d 100644 --- a/packages/stack-drizzle/integration/adapter.ts +++ b/packages/stack-drizzle/integration/adapter.ts @@ -42,7 +42,7 @@ const SUPPORTED_OPS: ReadonlySet = new Set([ 'lte', 'between', 'notBetween', - 'contains', + 'matches', 'order', 'isNull', 'isNotNull', @@ -90,8 +90,8 @@ export function makeDrizzleAdapter(): IntegrationAdapter { return ops.between(col, op.lo as never, op.hi as never) case 'notBetween': return ops.notBetween(col, op.lo as never, op.hi as never) - case 'contains': - return ops.contains(col, op.needle as never) + case 'matches': + return ops.matches(col, op.needle as never) case 'isNull': return ops.isNull(col) case 'isNotNull': diff --git a/packages/stack-drizzle/integration/json-contains.integration.test.ts b/packages/stack-drizzle/integration/json-contains.integration.test.ts index c955f7d6..de3e7b39 100644 --- a/packages/stack-drizzle/integration/json-contains.integration.test.ts +++ b/packages/stack-drizzle/integration/json-contains.integration.test.ts @@ -1,11 +1,12 @@ /** * Live JSON containment for the v3 `types.Json()` column through the Drizzle * operators. Seeds encrypted JSONB documents and queries them with - * `ops.contains(col, subObject)` — the same operator text search uses, but on a - * `json` column it emits the `@>` operator over the encrypted JSONB document - * (json has no `eql_v3.contains` overload) with a `query_jsonb` needle from - * `encryptQuery` — asserting it returns exactly the rows whose document contains - * the sub-object (jsonb `@>` semantics), and excludes the rest. + * `ops.contains(col, subObject)` — EXACT containment (distinct from the fuzzy + * `ops.matches` used for text search). On a `json` column it emits the `@>` + * operator over the encrypted JSONB document (json has no `eql_v3.contains` + * overload) with a `query_jsonb` needle from `encryptQuery` — asserting it + * returns exactly the rows whose document contains the sub-object (jsonb `@>` + * semantics), and excludes the rest. */ import type { JsonDocument } from '@cipherstash/stack/eql/v3' diff --git a/packages/stack-drizzle/integration/relational.integration.test.ts b/packages/stack-drizzle/integration/relational.integration.test.ts index 933c9bc2..ee36fcf7 100644 --- a/packages/stack-drizzle/integration/relational.integration.test.ts +++ b/packages/stack-drizzle/integration/relational.integration.test.ts @@ -323,9 +323,9 @@ describe('v3 drizzle — relational, needle guards, pagination', () => { ['ad', '👍👍', ''].map((needle) => [eqlType, needle] as const), ), )( - '%s contains rejects the unanswerable needle %j before encrypting', + '%s matches rejects the unanswerable needle %j before encrypting', async (eqlType, needle) => { - const attempt = ops.contains(matrixColumn(eqlType), needle) + const attempt = ops.matches(matrixColumn(eqlType), needle) await expect(attempt).rejects.toBeInstanceOf(EncryptionOperatorError) // Assert the GUARD rejected it, not the encryption layer. // `operandFailure` also throws `EncryptionOperatorError`, so matching only @@ -362,10 +362,10 @@ describe('v3 drizzle — relational, needle guards, pagination', () => { ), ), )( - '%s contains answers the codepoint-sufficient needle %j with no match', + '%s matches answers the codepoint-sufficient needle %j with no match', async (eqlType, needle) => { const rows = await selectRowKeys( - await ops.contains(matrixColumn(eqlType), needle), + await ops.matches(matrixColumn(eqlType), needle), ) expect(rows).toEqual([]) @@ -374,7 +374,7 @@ describe('v3 drizzle — relational, needle guards, pagination', () => { // constant-false `contains` also returns [], and nothing above catches // it. Prove the same column still answers a needle that IS present. const present = await selectRowKeys( - await ops.contains(matrixColumn(eqlType), 'ada'), + await ops.matches(matrixColumn(eqlType), 'ada'), ) expect(present.length).toBeGreaterThan(0) }, diff --git a/packages/stack-drizzle/src/v3/operators.ts b/packages/stack-drizzle/src/v3/operators.ts index dbf11dab..e8105202 100644 --- a/packages/stack-drizzle/src/v3/operators.ts +++ b/packages/stack-drizzle/src/v3/operators.ts @@ -165,7 +165,7 @@ type ChainableOperation = { } /** - * Build v3-aware query operators (`eq`, `gte`, `contains`, `asc`, …) bound to an + * Build v3-aware query operators (`eq`, `gte`, `matches`, `contains`, `asc`, …) bound to an * encryption `client`. Each comparison/containment operator AUTO-ENCRYPTS its * plaintext operand into an EQL v3 query term before handing it to Drizzle, so * callers pass plaintext and the emitted SQL compares encrypted values. Every @@ -266,13 +266,16 @@ export function createEncryptionOperatorsV3( // order-capable column answers equality via its ordering term too. const EQUALITY_INDEXES = ['unique', 'ore', 'ope'] as const const ORDERING_INDEXES = ['ore', 'ope'] as const - // `contains` answers two shapes: bloom free-text (`match`, a `text_search`/ - // `text_match` column), which emits `eql_v3.contains(col, operand)`, and - // encrypted-JSONB containment (an `eql_v3_json` column, whose config carries - // the `ste_vec` index kind), which emits the `@>` operator instead — json has - // no `eql_v3.contains` overload. The branch in `contains()` picks the right SQL - // by index kind. - const CONTAINMENT_INDEXES = ['match', 'ste_vec'] as const + // Two DISTINCT operators, split by semantics (#617): + // - `matches` is bloom free-text (`match`, a `text_search`/`text_match` + // column): a one-sided, order- and multiplicity-insensitive token match that + // may false-positive. It emits `eql_v3.contains(col, operand)` (the SQL + // function keeps its bundle name) but is NOT containment. + // - `contains` is encrypted-JSONB containment (an `eql_v3_json` column, + // `ste_vec` index): exact jsonb `@>`, no false positives — genuine + // containment, so it keeps the `contains` name. + const MATCH_INDEXES = ['match'] as const + const JSON_CONTAINMENT_INDEXES = ['ste_vec'] as const function applyOperationOptions( op: ChainableOperation, @@ -473,37 +476,24 @@ export function createEncryptionOperatorsV3( return negate ? sql`NOT ${condition}` : condition } - async function contains( + /** + * Fuzzy free-text token match on a `text_search`/`text_match` column. NOT + * containment: it tests whether the needle's downcased 3-gram set is a subset + * of the haystack's, via a bloom filter — order- and multiplicity-insensitive + * and one-sided (a `true` may be a false positive, a `false` never is). Emits + * `eql_v3.contains(col, operand)` (the SQL function's bundle name). + */ + async function matches( left: SQLWrapper, right: unknown, operator: string, opts?: EncryptionOperatorCallOpts, ): Promise { const ctx = resolveContext(left, operator) - requireIndex( - ctx, - CONTAINMENT_INDEXES, - operator, - 'free-text search or JSON containment', - ) - - // JSON containment. `eql_v3_json` has no `eql_v3.contains` overload — its - // containment is the `@>` operator, whose `(eql_v3_json, eql_v3.query_jsonb)` - // form takes a NARROWED query term (searchableJson → no ciphertext). The - // needle is cast to `eql_v3.query_jsonb` (an irregular name, so it is handled - // here rather than by `queryCastForDomain`) and emitted with `@>`. - if (ctx.indexes.ste_vec) { - const needle = await encryptJsonContainmentTerm( - ctx, - right, - operator, - opts, - ) - return v3Dialect.containsJson(colSql(left), needle) - } - - // Bloom free-text (text_search / text_match): the answerable-needle rule - // applies, and the `query_` cast reaches the match overload. + requireIndex(ctx, MATCH_INDEXES, operator, 'free-text search') + // The answerable-needle rule applies (a sub-`token_length` needle blooms to + // nothing and would match every row); the `query_` cast reaches the + // match overload. requireAnswerableNeedle(ctx, right, operator) const enc = await encryptOperand( ctx, @@ -515,6 +505,25 @@ export function createEncryptionOperatorsV3( return v3Dialect.contains(colSql(left), enc) } + /** + * Exact encrypted-JSONB containment on an `eql_v3_json` (`ste_vec`) column: + * genuine jsonb `@>`, no false positives — hence it keeps the `contains` name. + * `eql_v3_json` has no `eql_v3.contains` overload; containment is the `@>` + * operator, whose `(eql_v3_json, eql_v3.query_jsonb)` form takes a NARROWED + * query term (searchableJson → no ciphertext), cast to `eql_v3.query_jsonb`. + */ + async function containsJsonOp( + left: SQLWrapper, + right: unknown, + operator: string, + opts?: EncryptionOperatorCallOpts, + ): Promise { + const ctx = resolveContext(left, operator) + requireIndex(ctx, JSON_CONTAINMENT_INDEXES, operator, 'JSON containment') + const needle = await encryptJsonContainmentTerm(ctx, right, operator, opts) + return v3Dialect.containsJson(colSql(left), needle) + } + /** * Build a `query_jsonb` containment needle for a `json` column — the JSON query * term carries no ciphertext and satisfies the `eql_v3.query_jsonb` CHECK the @@ -654,10 +663,17 @@ export function createEncryptionOperatorsV3( max: unknown, opts?: EncryptionOperatorCallOpts, ) => range(l, min, max, true, 'notBetween', opts), - /** Free-text containment: emits `eql_v3.contains` over the encrypted match - * term. Encrypts `r`. Requires a `match` (free-text search) index. */ + /** Fuzzy free-text token match — the needle's 3-gram set is (bloom-)tested + * as a subset of the column's. NOT containment: order/multiplicity- + * insensitive and one-sided (a `true` may be a false positive). Encrypts + * `r`. Requires a `match` (free-text search) index. */ + matches: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => + matches(l, r, 'matches', opts), + /** Exact encrypted-JSONB containment (`@>`): matches rows whose document + * contains the given sub-object. No false positives. Encrypts `r`. Requires + * a `ste_vec` index (a `types.Json` column). */ contains: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) => - contains(l, r, 'contains', opts), + containsJsonOp(l, r, 'contains', opts), /** Membership: ORs one encrypted `eq` term per value. The whole list is * encrypted in one `encryptQuery` batch crossing. Rejects an empty list; * requires a `unique` or `ore` index. */ diff --git a/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts b/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts index 684fa1f2..48391c69 100644 --- a/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts +++ b/packages/stack-supabase/__tests__/supabase-v3-builder.test.ts @@ -203,10 +203,10 @@ describe('encryptedSupabaseV3 wire encoding', () => { expect(JSON.parse(gte.args[1] as string).c).toBeDefined() }) - it('emits encrypted contains as PostgREST cs (bloom-filter containment)', async () => { + it('emits encrypted matches as PostgREST cs (bloom-filter containment)', async () => { const { es, supabase } = v3Instance() - await es.from('users', users).select('id, email').contains('email', 'a@b') + await es.from('users', users).select('id, email').matches('email', 'a@b') const filterCalls = supabase.callsFor('filter') expect(filterCalls).toHaveLength(1) @@ -219,16 +219,44 @@ describe('encryptedSupabaseV3 wire encoding', () => { expect(supabase.callsFor('contains')).toHaveLength(0) }) - it('refuses like/ilike on an encrypted column, naming contains', async () => { + it('refuses contains() on an encrypted column, naming matches', async () => { const { es } = v3Instance() - // The typed builder omits like/ilike, but an untyped JS caller reaches them. expect(() => - es.from('users', users).select('id').like('email', '%a@b%'), - ).toThrow(/token containment.*Use contains\(\)/s) + es.from('users', users).select('id').contains('email', 'a@b'), + ).toThrow(/Use matches\(\)/) + }) + + // like/ilike on an ENCRYPTED column are an approximate compatibility shim that + // DELEGATES to matches: surrounding `%` are stripped and the residual term is + // fuzzy-matched, emitting the same `cs` wire as matches() (#617). + it('delegates like/ilike on an encrypted column to matches (cs)', async () => { + for (const op of ['like', 'ilike'] as const) { + const { es, supabase } = v3Instance() + + await es.from('users', users).select('id, email')[op]('email', '%a@b%') + + const filterCalls = supabase.callsFor('filter') + expect(filterCalls).toHaveLength(1) + expect(filterCalls[0].args[0]).toBe('email') + expect(filterCalls[0].args[1]).toBe('cs') + expect(JSON.parse(filterCalls[0].args[2] as string).c).toBeDefined() + // The stripped `%a@b%` needle reaches the SAME cs wire as matches('a@b'). + expect(JSON.parse(filterCalls[0].args[2] as string).pt).toBe('a@b') + } + }) + + // An internal `%` or any `_` cannot be approximated by trigram matching, so the + // shim throws rather than silently dropping the wildcard. + it('rejects a like/ilike pattern with an internal % or _ on an encrypted column', async () => { + const { es } = v3Instance() + expect(() => - es.from('users', users).select('id').ilike('email', '%a@b%'), - ).toThrow(/token containment.*Use contains\(\)/s) + es.from('users', users).select('id').like('email', 'a%b'), + ).toThrow(/cannot honor/) + expect(() => + es.from('users', users).select('id').ilike('email', 'f_o'), + ).toThrow(/cannot honor/) }) it('passes contains through to native cs on a plaintext column', async () => { @@ -264,19 +292,29 @@ describe('encryptedSupabaseV3 wire encoding', () => { expect(supabase.callsFor('like')[0].args[0]).toBe('constructor') }) - it('maps not(contains) on encrypted columns to not(cs)', async () => { + it('maps not(matches) on encrypted columns to not(cs)', async () => { const { es, supabase } = v3Instance() await es .from('users', users) .select('id, email') - .not('email', 'contains', 'a@b') + .not('email', 'matches', 'a@b') const [not] = supabase.callsFor('not') expect(not.args[0]).toBe('email') expect(not.args[1]).toBe('cs') }) + // `not(col, 'contains', …)` on an encrypted column would negate a fuzzy bloom + // match under the `contains` name — the confusion #617 removes. It is rejected; + // use the honest `matches` spelling (above) or the raw `cs` operator. + it('rejects not(contains) on an encrypted column, steering to matches', () => { + const { es } = v3Instance() + expect(() => + es.from('users', users).select('id').not('email', 'contains', 'a@b'), + ).toThrow(/not apply to encrypted column .* Use not\(.*'matches'/) + }) + // An encrypted in-list is emitted through `filter()` as a pre-formatted // operand, NOT through postgrest-js's `in()`, which would leave the quotes // inside each envelope unescaped. `supabase-v3-wire.test.ts` pins the bytes. @@ -797,13 +835,13 @@ describe('encryptedSupabaseV3 wire encoding', () => { expect(plain).not.toContain('"pt":["ada","grace"]') }) - it('rewrites an encrypted contains in a structured or() to cs', async () => { + it('rewrites an encrypted matches in a structured or() to cs', async () => { const { es, supabase } = v3Instance() await es .from('users', users) .select('id') - .or([{ column: 'email', op: 'contains', value: 'ada' }]) + .or([{ column: 'email', op: 'matches', value: 'ada' }]) expect(supabase.callsFor('or')[0].args[0] as string).toMatch( /^email\.cs\."/, @@ -1025,7 +1063,7 @@ describe('encryptedSupabaseV3 wire encoding', () => { it('sends a full envelope as the not(cs) operand', async () => { const { es, supabase } = v3Instance() - await es.from('users', users).select('id').not('email', 'contains', 'a@b') + await es.from('users', users).select('id').not('email', 'matches', 'a@b') const [not] = supabase.callsFor('not') expect(not.args[0]).toBe('email') diff --git a/packages/stack-supabase/__tests__/supabase-v3-matrix.test.ts b/packages/stack-supabase/__tests__/supabase-v3-matrix.test.ts index cb03776d..df1e0f90 100644 --- a/packages/stack-supabase/__tests__/supabase-v3-matrix.test.ts +++ b/packages/stack-supabase/__tests__/supabase-v3-matrix.test.ts @@ -192,10 +192,10 @@ describe('supabase v3 wire encoding, every domain', () => { }) describe.each(matchDomains)('%s (freeTextSearch)', (eqlType, spec) => { - it('emits contains() as a cs containment filter', async () => { + it('emits matches() as a cs containment filter', async () => { const { q, supabase, name } = instanceFor(eqlType, spec) - await q.select(`id, ${name}`).contains(name, firstSample(spec)) + await q.select(`id, ${name}`).matches(name, firstSample(spec)) const [filter] = supabase.callsFor('filter') expect(filter.args[0]).toBe(name) @@ -205,13 +205,28 @@ describe('supabase v3 wire encoding, every domain', () => { expect(supabase.callsFor('like')).toHaveLength(0) }) - it('refuses like(), directing the caller to contains()', async () => { + it('refuses contains(), directing the caller to matches()', async () => { const { q, name } = instanceFor(eqlType, spec) - expect(() => q.like(name, firstSample(spec) as string)).toThrow( - /Use contains\(\)/, + expect(() => q.contains(name, firstSample(spec))).toThrow( + /Use matches\(\)/, ) }) + + // like/ilike on an encrypted free-text column are an approximate shim that + // DELEGATES to matches: a wildcard-free sample is fuzzy-matched, emitting the + // same `cs` wire as matches() (#617). (These matrix samples carry no `%`/`_`.) + it('delegates like() to matches, emitting the same cs filter', async () => { + const { q, supabase, name } = instanceFor(eqlType, spec) + + await q.select(`id, ${name}`).like(name, firstSample(spec) as string) + + const [filter] = supabase.callsFor('filter') + expect(filter.args[0]).toBe(name) + expect(filter.args[1]).toBe('cs') + expect(JSON.parse(filter.args[2] as string).c).toBeDefined() + expect(supabase.callsFor('like')).toHaveLength(0) + }) }) describe.each(storageOnlyDomains)('%s (storage only)', (eqlType, spec) => { diff --git a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts index 52560e9a..10ab39b2 100644 --- a/packages/stack-supabase/__tests__/supabase-v3.test-d.ts +++ b/packages/stack-supabase/__tests__/supabase-v3.test-d.ts @@ -154,71 +154,74 @@ describe('encryptedSupabaseV3 typed surface (with schemas)', () => { builder.order('note') }) - it('narrows contains() to freeTextSearch-capable columns', async () => { + it('narrows matches() to freeTextSearch-capable columns', async () => { const supabase = await encryptedSupabaseV3(supabaseClient, { schemas: { users }, }) const builder = supabase.from('users') // public.eql_v3_text_search — equality + orderAndRange + freeTextSearch - builder.contains('email', 'ada') + builder.matches('email', 'ada') // public.eql_v3_text_match — freeTextSearch only - builder.contains('bio', 'ada') + builder.matches('bio', 'ada') // @ts-expect-error — nickname is public.eql_v3_text_eq: no match index - builder.contains('nickname', 'ada') + builder.matches('nickname', 'ada') // @ts-expect-error — amount is public.eql_v3_integer_ord: no match index - builder.contains('amount', 'ada') + builder.matches('amount', 'ada') // @ts-expect-error — active is public.eql_v3_boolean (storage only) - builder.contains('active', 'ada') - }) - - // `V3FreeTextSearchableKeys` deliberately admits plaintext row keys so that - // `contains()` reaches PostgREST's NATIVE jsonb/array containment — which the - // runtime already does, forwarding a non-encrypted operand straight to - // `q.contains`. A blanket `value: string` made that unreachable from - // TypeScript: the operand type must follow the column. - it('accepts native containment operands on a plaintext key', () => { - mixedBuilder.contains('tags', ['vip']) - mixedBuilder.contains('meta', { plan: 'pro' }) - mixedBuilder.contains('tags', 'vip') + builder.matches('active', 'ada') }) - it('still pins an encrypted text-search operand to string', () => { - mixedBuilder.contains('email', 'ada') + // `matches()` is encrypted free-text ONLY: its operand is the string to + // tokenize, and a plaintext key is a compile error (use `contains()`). + it('pins the matches() operand to a string on encrypted columns', () => { + mixedBuilder.matches('email', 'ada') + mixedBuilder.matches('bio', 'ada') // @ts-expect-error — email is public.eql_v3_text_search: the match term is a string - mixedBuilder.contains('email', ['ada']) + mixedBuilder.matches('email', ['ada']) // @ts-expect-error — bio is public.eql_v3_text_match: the match term is a string - mixedBuilder.contains('bio', { a: 1 }) + mixedBuilder.matches('bio', { a: 1 }) }) - // A union column key is only as permissive as its STRICTEST member. If any - // member is a declared encrypted column the operand must be the string term: - // that member's runtime path hands the operand to `encrypt()`, which has no - // plaintext-type guard, so an array reaches protect-ffi as the plaintext for a - // `cast_as: text` column. - it('pins a mixed union key to the encrypted string operand', () => { - mixedBuilder.contains(mixedKey, 'ada') - // @ts-expect-error — the union includes email (public.eql_v3_text_search) - mixedBuilder.contains(mixedKey, ['vip']) - // @ts-expect-error — the union includes email (public.eql_v3_text_search) - mixedBuilder.contains(mixedKey, { plan: 'pro' }) + it('rejects matches() on plaintext keys — use contains()', () => { + // @ts-expect-error — tags is plaintext; matches() is encrypted free-text only + mixedBuilder.matches('tags', 'vip') + // @ts-expect-error — meta is plaintext; matches() is encrypted free-text only + mixedBuilder.matches('meta', 'vip') + // @ts-expect-error — a union with a plaintext member is not free-text-only + mixedBuilder.matches(plaintextKey, 'vip') + // @ts-expect-error — a mixed union (encrypted + plaintext) is not free-text-only + mixedBuilder.matches(mixedKey, 'ada') }) - it('leaves a union of plaintext keys on the native operand', () => { + // `contains()` is native (exact) containment on PLAINTEXT columns; the operand + // follows the column shape (`@>` is array/jsonb only). An encrypted key is a + // compile error (use `matches()`). + it('accepts native containment operand shapes via contains()', () => { + mixedBuilder.contains('tags', ['vip']) + mixedBuilder.contains('tags', 'vip') + mixedBuilder.contains('meta', { plan: 'pro' }) mixedBuilder.contains(plaintextKey, ['vip']) mixedBuilder.contains(plaintextKey, { plan: 'pro' }) }) - // `@>` is defined on arrays and jsonb, not on a scalar. Postgres answers a - // containment query against a plaintext `text` column with 42883 - // (operator does not exist), so the operand type must follow the column's own - // shape rather than admitting every native containment value on every - // plaintext key. - it('rejects containment on a plaintext scalar column', () => { + it('rejects contains() on encrypted keys — use matches()', () => { + // @ts-expect-error — email is encrypted; contains() is native (plaintext) only + mixedBuilder.contains('email', 'ada') + // @ts-expect-error — bio is encrypted; contains() is native (plaintext) only + mixedBuilder.contains('bio', 'ada') + // @ts-expect-error — a mixed union includes the encrypted email + mixedBuilder.contains(mixedKey, ['vip']) + }) + + // `@>` is defined on arrays and jsonb, not on a scalar, so the operand type + // must follow the column's own shape rather than admitting every native + // containment value on every plaintext key. + it('rejects a container operand on a plaintext scalar column', () => { // @ts-expect-error — note is plaintext text: `text @> text[]` does not exist mixedBuilder.contains('note', ['vip']) // @ts-expect-error — note is plaintext text: `text @> jsonb` does not exist mixedBuilder.contains('note', { a: 1 }) - // @ts-expect-error — a scalar column supports no containment operand at all + // @ts-expect-error — a scalar column supports no container operand at all mixedBuilder.contains('note', 'vip') }) @@ -227,15 +230,15 @@ describe('encryptedSupabaseV3 typed surface (with schemas)', () => { schemas: { users }, }) const builder = supabase.from('users') - // @ts-expect-error — v3 free-text search is token containment: use contains() + // @ts-expect-error — v3 free-text search is token containment: use matches() builder.like('email', '%ada%') - // @ts-expect-error — v3 free-text search is token containment: use contains() + // @ts-expect-error — v3 free-text search is token containment: use matches() builder.ilike('email', '%ada%') // The chain must not launder the removal back in via a widened return type. - // @ts-expect-error — use contains() + // @ts-expect-error — use matches() builder.select('id').eq('email', 'a@b.com').like('email', '%ada%') - // contains() survives the chain. - builder.select('id').eq('email', 'a@b.com').contains('email', 'ada') + // matches() survives the chain. + builder.select('id').eq('email', 'a@b.com').matches('email', 'ada') }) it('accepts plaintext model values on insert', async () => { @@ -301,16 +304,19 @@ describe('encryptedSupabaseV3 untyped surface (no schemas)', () => { builder.eq('missing', 1) }) - it('exposes contains and not like/ilike, exactly as the typed surface does', async () => { - // Without `schemas` there is no capability information, so `contains` cannot - // be narrowed — but the DIALECT is still v3, so `like`/`ilike` must be gone. - // Otherwise the untyped surface silently hands back the v2 builder type. + it('exposes matches and contains, but not like/ilike', async () => { + // Without `schemas` there is no capability information, so neither `matches` + // nor `contains` can be narrowed — but the DIALECT is still v3, so + // `like`/`ilike` must be gone. Otherwise the untyped surface silently hands + // back the v2 builder type. The untyped v3 builder exposes BOTH the encrypted + // free-text `matches` and the native `contains`. const supabase = await encryptedSupabaseV3(supabaseClient) const builder = supabase.from<{ id: number; email: string }>('users') + builder.matches('email', 'ada') builder.contains('email', 'ada') - // @ts-expect-error — v3 free-text search is token containment: use contains() + // @ts-expect-error — v3 free-text search is token containment: use matches() builder.like('email', '%ada%') - // @ts-expect-error — v3 free-text search is token containment: use contains() + // @ts-expect-error — v3 free-text search is token containment: use matches() builder.ilike('email', '%ada%') }) diff --git a/packages/stack-supabase/integration/adapter.ts b/packages/stack-supabase/integration/adapter.ts index 4df3689b..e8e1c68f 100644 --- a/packages/stack-supabase/integration/adapter.ts +++ b/packages/stack-supabase/integration/adapter.ts @@ -40,7 +40,7 @@ const SUPPORTED_OPS: ReadonlySet = new Set([ 'lte', 'between', 'notBetween', - 'contains', + 'matches', 'isNull', 'isNotNull', 'order', @@ -102,8 +102,8 @@ export function makeSupabaseAdapter(): IntegrationAdapter { { column: op.column, op: 'lt', value: op.lo }, { column: op.column, op: 'gt', value: op.hi }, ]) - case 'contains': - return q.contains(op.column, op.needle) + case 'matches': + return q.matches(op.column, op.needle) case 'isNull': return q.is(op.column, null) case 'isNotNull': diff --git a/packages/stack-supabase/integration/wire.integration.test.ts b/packages/stack-supabase/integration/wire.integration.test.ts index 56be1560..dd7b92be 100644 --- a/packages/stack-supabase/integration/wire.integration.test.ts +++ b/packages/stack-supabase/integration/wire.integration.test.ts @@ -341,10 +341,10 @@ describe('supabase v3 adapter over real PostgREST (wire + grants)', () => { // The four tests below discriminate: substrings that share no `c` with the // stored row match, and a trigram present in no row does not. Only bloom // containment explains both. - it('resolves contains() through cs containment for an exact value', async () => { + it('resolves matches() through cs containment for an exact value', async () => { const { data, error } = await from() .select('row_key') - .contains('email', 'ada@example.com') + .matches('email', 'ada@example.com') expect(error).toBeNull() expect(data.map((r: { row_key: string }) => r.row_key)).toEqual(['ada']) @@ -357,7 +357,7 @@ describe('supabase v3 adapter over real PostgREST (wire + grants)', () => { it('matches a longer substring through bloom containment', async () => { const { data, error } = await from() .select('row_key') - .contains('email', 'example') + .matches('email', 'example') expect(error).toBeNull() expect(data.map((r: { row_key: string }) => r.row_key)).toEqual(['ada']) @@ -366,7 +366,7 @@ describe('supabase v3 adapter over real PostgREST (wire + grants)', () => { it('matches a substring exactly one trigram long', async () => { const { data, error } = await from() .select('row_key') - .contains('email', 'ada') + .matches('email', 'ada') expect(error).toBeNull() expect(data.map((r: { row_key: string }) => r.row_key)).toEqual(['ada']) @@ -379,18 +379,34 @@ describe('supabase v3 adapter over real PostgREST (wire + grants)', () => { it('does not match a trigram absent from every stored value', async () => { const { data, error } = await from() .select('row_key') - .contains('email', 'zzz') + .matches('email', 'zzz') expect(error).toBeNull() expect(data).toEqual([]) }) - // The reason `like` is gone: `~~` is not defined on public.eql_v3_text_search, so - // had the adapter emitted it, PostgREST/Postgres would answer 42883. The - // client-side guard turns that into an actionable error before the round-trip. - it('refuses like() on an encrypted column rather than emitting an undefined operator', async () => { - expect(() => from().select('row_key').like('email', 'ada')).toThrow( - /Use contains\(\)/, + // `like`/`ilike` on an encrypted column are a compatibility shim: they delegate + // to `matches` (fuzzy bloom token search), NOT SQL `~~` (undefined on + // public.eql_v3_text_search — 42883). Surrounding `%` are stripped, so + // `like('email', '%ada%')` searches the term `ada` and returns the same row as + // `matches('email', 'ada')`. + it('delegates like() on an encrypted column to matches (fuzzy)', async () => { + const { data, error } = await from() + .select('row_key') + .like('email', '%ada%') + + expect(error).toBeNull() + expect(data.map((r: { row_key: string }) => r.row_key)).toEqual(['ada']) + }) + + // A pattern fuzzy matching cannot approximate (internal `%` / any `_`) is + // refused client-side before the round-trip. + it('rejects an unapproximable like() pattern on an encrypted column', () => { + expect(() => from().select('row_key').like('email', 'a%b')).toThrow( + /cannot honor/, + ) + expect(() => from().select('row_key').ilike('email', 'f_o')).toThrow( + /cannot honor/, ) }) diff --git a/packages/stack-supabase/src/helpers.ts b/packages/stack-supabase/src/helpers.ts index 9c6d69c6..d3469741 100644 --- a/packages/stack-supabase/src/helpers.ts +++ b/packages/stack-supabase/src/helpers.ts @@ -237,7 +237,11 @@ export function mapFilterOpToQueryType(op: FilterOp): QueryTypeName { return 'equality' case 'like': case 'ilike': + // `matches` is the encrypted free-text (bloom) operator. `contains` remains + // for completeness/v2 but is plaintext-only on the v3 surface, so it never + // reaches term encryption there (a plaintext operand is not encrypted). case 'contains': + case 'matches': return 'freeTextSearch' case 'gt': case 'gte': @@ -314,7 +318,7 @@ export function parseOrString(orString: string): PendingOrCondition[] { * `contains` is supabase-js's METHOD name for this operator; string-form `.or()` * callers write PostgREST's `cs` directly. Both reach here. */ -const CONTAINMENT_OPS = new Set(['contains', 'cs']) +const CONTAINMENT_OPS = new Set(['contains', 'matches', 'cs']) /** * The PostgREST operator token for a {@link FilterOp}. @@ -332,7 +336,7 @@ const CONTAINMENT_OPS = new Set(['contains', 'cs']) * containment broken while the encrypted path worked. */ function orOperatorToken(op: string): string { - return op === 'contains' ? 'cs' : op + return op === 'contains' || op === 'matches' ? 'cs' : op } /** diff --git a/packages/stack-supabase/src/query-builder-v3.ts b/packages/stack-supabase/src/query-builder-v3.ts index a84de692..608661ef 100644 --- a/packages/stack-supabase/src/query-builder-v3.ts +++ b/packages/stack-supabase/src/query-builder-v3.ts @@ -118,17 +118,19 @@ function assertNoPropertyDbNameCollision( * The full envelope satisfies the storage-domain CHECK by construction, and * the operators extract the term they need (`eq_term`/`ord_term`/ * `match_term`). - * - **`contains`, not `like`/`ilike`** — the v3 domains define no LIKE operator. - * Free-text search is TOKEN CONTAINMENT: the bundle declares `@>` on each - * match domain (`CREATE OPERATOR @> … FUNCTION = eql_v3.contains`), whose body - * is `match_term(a) @> match_term(b)` — `smallint[]` containment of the two - * bloom filters. PostgREST reaches it as `cs`. + * - **`matches`, not `like`/`ilike`/`contains`** — encrypted free-text search is + * FUZZY BLOOM TOKEN MATCHING, not containment: the bundle declares `@>` on each + * match domain (`CREATE OPERATOR @> … FUNCTION = eql_v3.contains`, the SQL + * function's name), whose body is `match_term(a) @> match_term(b)` — `smallint[]` + * containment of the two bloom filters. It is order- and multiplicity- + * insensitive and one-sided (a `true` may be a false positive). PostgREST + * reaches it as `cs`. The operator is named `matches` to signal that; `contains` + * is reserved for exact (native) containment on plaintext columns. * - * Match is tokenized + downcased, so `%` is NOT a wildcard — it is tokenized - * like any other character, and a `like` pattern is a category error. v3 - * Drizzle omits `like`/`ilike` for this reason and exposes `contains`; so do - * we. The typed builder has no `like`; the runtime methods throw on an - * encrypted column and pass through on a plaintext one. + * Match is tokenized + downcased, so `%` is NOT a wildcard. `like`/`ilike` on an + * encrypted column are delegated to `matches` as an APPROXIMATE compatibility + * shim (surrounding `%` stripped, internal `%`/`_` rejected, one warning) and + * pass through as real SQL LIKE on a plaintext column. * * Substrings DO match: the needle blooms to its own trigrams, and containment * holds whenever every one of them is present in the stored value's bloom — @@ -561,39 +563,113 @@ export class EncryptedQueryBuilderV3Impl< ) } + /** Warn once per (op, column) that a `like`/`ilike` was delegated to `matches`. */ + private static readonly warnedLikeDelegation = new Set() + + /** True when `column` is one of this table's encrypted v3 columns. */ + private isEncryptedV3Column(column: string): boolean { + return Boolean(this.v3Columns[column]) + } + /** - * `like`/`ilike` do not exist on the v3 surface (see the class doc). The - * typed builder omits them, but an untyped JS caller can still reach them — - * refuse loudly rather than emit a `~~` the domain has no operator for. - * - * A plaintext column is a genuine PostgREST text column, so `like` there is - * exactly what the caller means; let it through. + * `contains` on the v3 surface is EXACT containment only — native jsonb/array + * `@>` on a plaintext column. On an encrypted match/search column containment + * is not the operation (that is the fuzzy `matches`), so refuse loudly rather + * than silently emit a bloom match under a name that promises exactness. */ - private assertNotEncryptedPattern(column: string, op: string): void { - if (!this.v3Columns[column]) return - throw new Error( - `[supabase v3]: "${op}" is not supported on encrypted column "${column}" — EQL v3 free-text search is token containment, not SQL wildcard matching ("%" is tokenized like any other character). Use contains().`, - ) + override contains(column: string, value: unknown): this { + if (this.isEncryptedV3Column(column)) { + throw new Error( + `[supabase v3]: contains() is native (exact) containment and does not apply to encrypted column "${column}". Use matches() for encrypted free-text search.`, + ) + } + return super.contains(column, value) + } + + /** + * `matches` is the encrypted free-text operator: fuzzy bloom-filter token + * matching, one-sided (may false-positive), NOT containment. It requires an + * encrypted match/search column; on a plaintext column, `contains` (native + * `@>`) is what the caller means. + */ + override matches(column: string, value: unknown): this { + if (!this.isEncryptedV3Column(column)) { + throw new Error( + `[supabase v3]: matches() is encrypted free-text search and requires an encrypted column; "${column}" is not one. Use contains() for native containment.`, + ) + } + return super.matches(column, value) + } + + /** + * `not(col, 'contains', …)` on an encrypted column would negate a fuzzy bloom + * match under the `contains` name — the exact confusion #617 removes — because + * the base `not()` path rewrites the `contains` spelling to the `cs` wire + * operator. Reject it and steer to the `matches` spelling (or the raw `cs` + * operator, which is honest about the wire op). Plaintext columns keep native + * negated containment, and every other operator is delegated unchanged. + */ + override not(column: string, operator: string, value: unknown): this { + if (operator === 'contains' && this.isEncryptedV3Column(column)) { + throw new Error( + `[supabase v3]: not("${column}", 'contains', …) does not apply to encrypted column "${column}" — that is fuzzy free-text matching, not containment. Use not("${column}", 'matches', …) or the raw 'cs' operator.`, + ) + } + return super.not(column, operator, value) } + /** + * `like`/`ilike` on an ENCRYPTED column are a best-effort compatibility shim, + * delegated to `matches`. EQL v3 free-text search is fuzzy bloom token + * matching, not SQL pattern matching, so the result is APPROXIMATE — matching + * is case-insensitive and one-sided (may false-positive), and anchoring is + * lost. Leading/trailing `%` are stripped; an internal `%` or any `_` cannot be + * approximated by trigram matching and throws. A plaintext column keeps real + * SQL LIKE. + */ override like(column: string, pattern: string): this { - this.assertNotEncryptedPattern(column, 'like') - return super.like(column, pattern) + if (!this.isEncryptedV3Column(column)) return super.like(column, pattern) + return this.matches(column, this.likeNeedle(column, 'like', pattern)) } override ilike(column: string, pattern: string): this { - this.assertNotEncryptedPattern(column, 'ilike') - return super.ilike(column, pattern) + if (!this.isEncryptedV3Column(column)) return super.ilike(column, pattern) + return this.matches(column, this.likeNeedle(column, 'ilike', pattern)) + } + + /** + * Reduce a SQL LIKE pattern to a fuzzy-match needle, or throw when it cannot be + * approximated. Strips surrounding `%` (prefix/suffix wildcards, which fuzzy + * matching subsumes); an internal `%` or any `_` is unapproximable. Warns once + * per (op, column) that the delegation is approximate. + */ + private likeNeedle(column: string, op: string, pattern: string): string { + const needle = pattern.replace(/^%+/, '').replace(/%+$/, '') + if (needle.includes('%') || pattern.includes('_')) { + throw new Error( + `[supabase v3]: "${op}" pattern "${pattern}" on encrypted column "${column}" has wildcards fuzzy free-text matching cannot honor (an internal "%" or any "_"). Use matches("${column}", term) with a literal search term.`, + ) + } + const key = `${op}:${column}` + if (!EncryptedQueryBuilderV3Impl.warnedLikeDelegation.has(key)) { + EncryptedQueryBuilderV3Impl.warnedLikeDelegation.add(key) + logger.warn( + `[supabase v3]: "${op}" on encrypted column "${column}" is delegated to matches() (fuzzy bloom token search). Results are APPROXIMATE — case-insensitive, one-sided (may false-positive), and wildcards/anchoring are not honored. Call matches() directly to make this explicit.`, + ) + } + return needle } /** - * Encrypted `contains` goes through the bloom-filter `@>`, which the bundle + * Encrypted `matches` goes through the bloom-filter `@>`, which the bundle * declares on the domain as PostgREST's `cs`. The operand is the full storage - * envelope; `eql_v3.contains` extracts the `bf` array from both sides. + * envelope; `eql_v3.contains` (the SQL function) extracts the `bf` array from + * both sides. * * Emitted via `filter(col, 'cs', json)` rather than `q.contains(col, json)`: * postgrest-js's `contains` re-serializes a non-string operand, and our - * operand is already `JSON.stringify`d. + * operand is already `JSON.stringify`d. Plaintext `contains` (not encrypted) + * falls through to the base's native path. */ protected override applyContainsFilter( q: SupabaseQueryBuilder, @@ -612,10 +688,11 @@ export class EncryptedQueryBuilderV3Impl< * condition arrives as `cs` — not a {@link FilterOp}. Resolve it through the * same table the raw `.filter()` path uses, so `.or('amount.cs.5')` on an * `integer_ord` column is rejected by the capability guard rather than - * silently encrypted as an equality term. + * silently encrypted as an equality term. A structured `{ op: 'matches' }` + * condition maps to free-text directly. */ protected override queryTypeForOrOp(op: FilterOp): QueryTypeName { - if (op === 'contains') return 'freeTextSearch' + if (op === 'matches') return 'freeTextSearch' return this.queryTypeForRawOp(op) } diff --git a/packages/stack-supabase/src/query-builder.ts b/packages/stack-supabase/src/query-builder.ts index 57d0fc03..71587778 100644 --- a/packages/stack-supabase/src/query-builder.ts +++ b/packages/stack-supabase/src/query-builder.ts @@ -244,6 +244,17 @@ export class EncryptedQueryBuilderImpl< return this } + /** + * Encrypted free-text token match (v3 encrypted columns). Emits the same + * `cs`/`@>` wire operator as `contains`, but on a match-indexed encrypted + * column it is fuzzy bloom-filter token matching, not containment — see the v3 + * builder. The v3 dialect encrypts the operand as a free-text query term. + */ + matches(column: string, value: unknown): this { + this.filters.push({ op: 'matches', column, value }) + return this + } + is(column: string, value: null | boolean): this { this.filters.push({ op: 'is', column, value }) return this @@ -1077,7 +1088,11 @@ export class EncryptedQueryBuilderImpl< case 'ilike': q = this.applyPatternFilter(q, column, f.op, value, wasEncrypted) break + // `matches` (encrypted free-text) and `contains` (plaintext / encrypted + // JSON) share the `cs`/`@>` wire operator; the operand encoding is the + // same, so both emit through the one containment applier. case 'contains': + case 'matches': q = this.applyContainsFilter(q, column, value, wasEncrypted) break case 'is': @@ -1133,7 +1148,7 @@ export class EncryptedQueryBuilderImpl< // containment literal ourselves and emit the `cs` token, exactly as the // `.or()` path does. A scalar (including the encrypted envelope, already // serialized) yields `null` and is forwarded untouched. - if (nf.op === 'contains') { + if (nf.op === 'contains' || nf.op === 'matches') { const literal = formatContainmentOperand(value) q = q.not(nf.column, 'cs', literal ?? value) continue diff --git a/packages/stack-supabase/src/types.ts b/packages/stack-supabase/src/types.ts index 41cf4ca9..df5fe786 100644 --- a/packages/stack-supabase/src/types.ts +++ b/packages/stack-supabase/src/types.ts @@ -126,6 +126,25 @@ export type V3FreeTextSearchableKeys< Row extends Record, > = Exclude, NonFreeTextSearchV3Keys> +/** + * Row keys `matches()` accepts: ONLY the table's ENCRYPTED columns that carry a + * `freeTextSearch` capability (`public.eql_v3_text_match` / `text_search`). + * + * Unlike {@link V3FreeTextSearchableKeys} (which additionally lets plaintext keys + * through, because the old `contains` also served native containment), this + * excludes plaintext columns entirely — `matches()` is encrypted free-text only, + * so calling it on a plaintext column is a compile error, not a runtime throw. + * Derived from the encrypted-column keys minus the non-free-text ones. + */ +export type V3EncryptedFreeTextKeys< + Table extends AnyV3Table, + Row extends Record, +> = Exclude< + Extract, string>, + NonFreeTextSearchV3Keys
+> & + Extract + /** * The operand `contains()` accepts on a PLAINTEXT column, mirroring * postgrest-js's own untyped `contains` overload: a jsonb literal, an array, or @@ -257,10 +276,10 @@ export type V3PlaintextKeys< * filter methods narrowed to {@link V3FilterableKeys} and `order()` to * {@link V3OrderableKeys}. * - * `like`/`ilike` are absent by construction. EQL v3 free-text search is token - * containment over a bloom filter (`@>`), not SQL wildcard matching — `%` is + * `like`/`ilike` are absent by construction. EQL v3 free-text search is fuzzy + * bloom-filter token matching (`@>`), not SQL wildcard matching — `%` is * tokenized like any other character, so a `like` pattern is a category error. - * The v3 dialect of Drizzle omits them for the same reason. Use `contains`. + * The v3 dialect of Drizzle omits them for the same reason. Use `matches`. */ export interface EncryptedQueryBuilderV3< Table extends AnyV3Table, @@ -277,9 +296,19 @@ export interface EncryptedQueryBuilderV3< // still admits none. V3PlaintextKeys & StringKeyOf > { - contains & StringKeyOf>( + /** Encrypted free-text token match. Encrypted match/search columns only — + * plaintext columns are a compile error (use {@link contains}). The operand is + * the string to tokenize into a bloom-filter query term. */ + matches & StringKeyOf>( + column: K, + value: string, + ): EncryptedQueryBuilderV3 + /** Native (exact) jsonb/array containment (`@>`). Plaintext columns only — an + * encrypted column is a compile error (use {@link matches}). A scalar plaintext + * column resolves its operand to `never` (`@>` is array/jsonb only). */ + contains & StringKeyOf>( column: K, - value: V3ContainsValue, + value: PlaintextContainsValue, ): EncryptedQueryBuilderV3 } @@ -291,9 +320,9 @@ export interface EncryptedQueryBuilderV3< * {@link EncryptedQueryBuilder} would hand back the v2 surface. * * For the same reason nothing here can tell an encrypted match column from a - * plaintext jsonb one, so `contains` accepts the full native operand union - * (which subsumes the encrypted column's `string`); the runtime resolves the - * column and picks the encoding. + * plaintext jsonb one, so `matches`/`contains` accept the full native operand + * union (which subsumes the encrypted column's `string`); the runtime resolves + * the column and picks the encoding (and rejects the wrong-column-kind pairing). */ export interface EncryptedQueryBuilderV3Untyped< Row extends Record, @@ -302,6 +331,14 @@ export interface EncryptedQueryBuilderV3Untyped< StringKeyOf, EncryptedQueryBuilderV3Untyped > { + /** Fuzzy free-text token match on an encrypted match/search column. The + * operand is always the string term to tokenize (never an array/object), even + * on the untyped surface where the column kind is unknown. */ + matches>( + column: K, + value: string, + ): EncryptedQueryBuilderV3Untyped + /** Native jsonb/array containment on a plaintext column (PostgREST `cs`). */ contains>( column: K, value: NativeContainsValue, @@ -368,9 +405,12 @@ export type FilterOp = | 'neq' | 'like' | 'ilike' - /** Token containment. v3-only on encrypted columns; on a plaintext column it - * is PostgREST's native jsonb/array `cs`. */ + /** Native jsonb/array containment (PostgREST `cs` → `@>`). Plaintext columns + * on the v3 surface; also the encrypted-JSON path where applicable. */ | 'contains' + /** Encrypted free-text token match (bloom `@>`). v3 encrypted match/search + * columns only. Same `cs` wire operator as `contains`, different semantics. */ + | 'matches' | 'in' | 'gt' | 'gte' diff --git a/packages/test-kit/src/index.ts b/packages/test-kit/src/index.ts index ecdc2756..8b359f78 100644 --- a/packages/test-kit/src/index.ts +++ b/packages/test-kit/src/index.ts @@ -52,8 +52,8 @@ export { } from './ops.ts' export { comparePlain, - containsPlain, expectedKeysFor, + matchesPlain, plainValue, sortedKeysFor, } from './oracle.ts' diff --git a/packages/test-kit/src/integration/no-skips-reporter.ts b/packages/test-kit/src/integration/no-skips-reporter.ts index ee40f3e9..4759905b 100644 --- a/packages/test-kit/src/integration/no-skips-reporter.ts +++ b/packages/test-kit/src/integration/no-skips-reporter.ts @@ -4,7 +4,7 @@ import type { Reporter, TestModule } from 'vitest/node' * Fail the integration run if any test was skipped. * * A skipped test reads exactly like a passing one. Every silent hole this suite - * has found took that shape: `contains` never ran on `text_match` because its + * has found took that shape: `matches` never ran on `text_match` because its * needle was `''`; the non-ASCII ORE needle test had zero cases after the OPE * re-pin; the Supabase grants check quietly did not run for the Drizzle job * because `dbVariant()` mis-inferred the database. In each case a green run diff --git a/packages/test-kit/src/needle-for.ts b/packages/test-kit/src/needle-for.ts index 13f62b86..240ef30c 100644 --- a/packages/test-kit/src/needle-for.ts +++ b/packages/test-kit/src/needle-for.ts @@ -13,7 +13,7 @@ import type { V3_MATRIX } from './catalog' type MatrixSpec = (typeof V3_MATRIX)[keyof typeof V3_MATRIX] /** - * Pick a sample from a match domain that can actually be used as a `contains` + * Pick a sample from a match domain that can actually be used as a `matches` * needle. `sampleFor` is no good here — `TEXT_S[0]` is the empty string, which * tokenizes to nothing and is rejected as unanswerable. * diff --git a/packages/test-kit/src/ops.ts b/packages/test-kit/src/ops.ts index a6d86fb8..b95a1b35 100644 --- a/packages/test-kit/src/ops.ts +++ b/packages/test-kit/src/ops.ts @@ -12,7 +12,7 @@ export type QueryOpKind = | 'lte' | 'between' | 'notBetween' - | 'contains' + | 'matches' | 'order' | 'isNull' | 'isNotNull' @@ -38,7 +38,7 @@ export type QueryOp = asRawFilter?: boolean } | { kind: 'between' | 'notBetween'; column: string; lo: Plain; hi: Plain } - | { kind: 'contains'; column: string; needle: string } + | { kind: 'matches'; column: string; needle: string } | { kind: 'order'; column: string; direction: 'asc' | 'desc' } | { kind: 'isNull' | 'isNotNull'; column: string } @@ -56,12 +56,12 @@ export type QueryOp = const OPS_BY_CAPABILITY = { equality: ['eq', 'ne', 'in', 'notIn'], orderAndRange: ['gt', 'gte', 'lt', 'lte', 'between', 'notBetween', 'order'], - freeTextSearch: ['contains'], - // Encrypted-JSONB containment surfaces as `contains` too (doc @> subset), but + freeTextSearch: ['matches'], // json domains are `deferred` from this driver (ste_vec doesn't fit the scalar - // oracle) and covered by a dedicated suite, so this entry exists to satisfy - // the capability-keyed type rather than to gate a run. - searchableJson: ['contains'], + // oracle) and covered by a dedicated suite (`json-contains`, which calls the + // adapter's exact `contains()` directly), so this entry only satisfies the + // capability-keyed type — it gates no matrix run. `matches` is a placeholder. + searchableJson: ['matches'], } as const satisfies Record /** diff --git a/packages/test-kit/src/oracle.ts b/packages/test-kit/src/oracle.ts index 3006c702..bc05a529 100644 --- a/packages/test-kit/src/oracle.ts +++ b/packages/test-kit/src/oracle.ts @@ -94,12 +94,12 @@ export function sortedKeysFor( } /** - * The `contains` oracle. The match index downcases and tokenizes into 3-grams - * with `include_original: true`, so a needle matches iff it appears in the - * downcased value — with the caveat that a needle longer than the tokenizer + * The `matches` (free-text) oracle. The match index downcases and tokenizes into + * 3-grams with `include_original: true`, so a needle matches iff it appears in + * the downcased value — with the caveat that a needle longer than the tokenizer * window contributes a whole-value token no stored trigram supplies. Family * files choose needles accordingly; this models the plaintext side only. */ -export function containsPlain(value: Plain, needle: string): boolean { +export function matchesPlain(value: Plain, needle: string): boolean { return String(value).toLowerCase().includes(needle.toLowerCase()) } diff --git a/packages/test-kit/src/run-family-suite.ts b/packages/test-kit/src/run-family-suite.ts index 524df7ec..6cca1ca2 100644 --- a/packages/test-kit/src/run-family-suite.ts +++ b/packages/test-kit/src/run-family-suite.ts @@ -9,8 +9,8 @@ import { import { negativeOps, type Plain, positiveOps, type QueryOp } from './ops.ts' import { comparePlain, - containsPlain, expectedKeysFor, + matchesPlain, plainValue, sortedKeysFor, } from './oracle.ts' @@ -125,7 +125,7 @@ function sampleOpFor( case 'between': case 'notBetween': return { kind, column, lo: value, hi: value } - case 'contains': + case 'matches': return { kind, column, @@ -314,13 +314,13 @@ export function runFamilySuite( }) } - if (positive.has('contains')) { - it.each(needlesFrom(values))('contains: $label', async ({ + if (positive.has('matches')) { + it.each(needlesFrom(values))('matches: $label', async ({ needle, }) => { await expectRows( - { kind: 'contains', column: slug, needle }, - keysWhere((v) => containsPlain(v, needle)), + { kind: 'matches', column: slug, needle }, + keysWhere((v) => matchesPlain(v, needle)), ) }) } diff --git a/skills/stash-drizzle/SKILL.md b/skills/stash-drizzle/SKILL.md index 80ea215f..2b6dc44a 100644 --- a/skills/stash-drizzle/SKILL.md +++ b/skills/stash-drizzle/SKILL.md @@ -762,7 +762,7 @@ if (!enc.failure) await db.insert(users).values(enc.data) // Query — operators auto-encrypt their plaintext operands const rows = await db.select().from(users) .where(await ops.and( - ops.contains(users.email, "alice"), // free-text containment + ops.matches(users.email, "alice"), // fuzzy free-text token match ops.between(users.age, 18, 65), )) .orderBy(ops.asc(users.age)) @@ -777,17 +777,19 @@ const dec = await client.bulkDecryptModels(rows, usersSchema) |---|---| | `eq`, `ne`, `inArray`, `notInArray` | equality (`Eq`, `Ord`, `OrdOre`, `TextSearch`) | | `gt`, `gte`, `lt`, `lte`, `between`, `notBetween`, `asc`, `desc` | order/range (`Ord`, `OrdOre`, `TextSearch`) | -| `contains` | free-text (`TextMatch`, `TextSearch`) **or** encrypted-JSONB containment (`Json`) | +| `matches` | fuzzy free-text token match (`TextMatch`, `TextSearch`) | +| `contains` | exact encrypted-JSONB containment (`Json`) | | `and`, `or` | combinators — accept lazy (un-awaited) operators and `undefined`, resolve concurrently | | `isNull`, `isNotNull`, `not`, `exists`, `notExists` | Drizzle passthroughs, no encryption | Differences from the v2 operators to know about: -- **`like` / `ilike` do not exist — by design.** v3 free-text search is tokenised containment, not SQL pattern matching; `contains(col, needle)` is the free-text operator. Don't pass `%` wildcards. -- **`contains` on a `types.Json` column** answers encrypted-JSONB containment instead of free-text: `contains(col, { roles: ['admin'] })` matches every row whose document contains that sub-object (jsonb `@>` semantics; array containment is position-independent). It emits the `@>` operator with a `query_jsonb` needle — a `Json` column carries no `eq` / ordering, so those operators throw on it. +- **`like` / `ilike` do not exist — by design.** v3 free-text search is tokenised bloom matching, not SQL pattern matching; `matches(col, needle)` is the free-text operator. It is fuzzy (order- and multiplicity-insensitive) and one-sided (a match may be a false positive, a non-match never is). Don't pass `%` wildcards. +- **`matches` is fuzzy free-text, `contains` is exact JSON containment — two distinct operators (#617).** `matches(col, needle)` requires a `TextMatch` / `TextSearch` column and throws `EncryptionOperatorError` (`requires free-text search`) otherwise. `contains(col, subdoc)` requires a `types.Json` column and throws (`requires JSON containment`) otherwise. +- **`contains` on a `types.Json` column** answers exact encrypted-JSONB containment: `contains(col, { roles: ['admin'] })` matches every row whose document contains that sub-object (jsonb `@>` semantics, no false positives; array containment is position-independent). It emits the `@>` operator with a `query_jsonb` needle — a `Json` column carries no `eq` / ordering, so those operators throw on it. - **No plaintext-column fallback.** Every v3 operator requires an encrypted v3 column and throws `EncryptionOperatorError` otherwise. Use regular Drizzle operators for non-encrypted columns. - A `null` operand throws — use `isNull()` / `isNotNull()` for NULL checks. - `inArray` / `notInArray` reject an empty list, and encrypt the whole list in a single `encryptQuery` batch crossing. -- `contains` rejects a needle shorter than the match tokenizer's token length (it would otherwise silently match every row). +- `matches` rejects a needle shorter than the match tokenizer's token length, and `contains` rejects an empty-object needle (either would otherwise silently match every row). - Operators gate on the column's capabilities and throw `EncryptionOperatorError` (with `context.columnName` / `tableName` / `operator`) when the domain can't answer the operator. This `EncryptionOperatorError` is exported from `@cipherstash/stack-drizzle/v3` and is deliberately separate from the v2 class of the same name; there is no `EncryptionConfigError` on the v3 path. - Every operator takes an optional trailing `{ lockContext, audit }` argument; `createEncryptionOperatorsV3(client, { lockContext, audit })` sets defaults applied to every operand encryption. diff --git a/skills/stash-supabase/SKILL.md b/skills/stash-supabase/SKILL.md index 847a2108..d8046c9a 100644 --- a/skills/stash-supabase/SKILL.md +++ b/skills/stash-supabase/SKILL.md @@ -417,8 +417,10 @@ internally. Columns are stored in their native `public.eql_v3_*` domain (a `eql_v2_encrypted`. The query surface matches v2 — same filter methods, `withLockContext`, -`audit` — with one deliberate fork: free-text search is `contains()`, and -`like`/`ilike` are rejected on encrypted columns (see below). +`audit` — with one deliberate fork: encrypted free-text search is `matches()` +(fuzzy bloom token search); `contains()` stays native (exact) containment for +plaintext columns; and `like`/`ilike` on an encrypted column are an approximate +shim that delegates to `matches()` (see below). ### Setup @@ -464,7 +466,7 @@ const { data } = await es.from("users").select("id, email, joined").eq("email", A declared table gets a typed builder: rows infer each column's plaintext type (`types.IntegerOrd` → `number`, `types.TimestampOrd` → `Date`), -storage-only columns are excluded from every filter method, `contains()` is +storage-only columns are excluded from every filter method, `matches()` is narrowed to match-indexed columns, and `order()` to plaintext columns. Undeclared tables behave exactly as with no `schemas` at all. Every v3 column is fully described by its `types.*` factory — there are no capability or @@ -520,13 +522,22 @@ All envelopes (stored payloads and filter operands) are versioned `v: 3`. - **`select('*')` (and bare `select()`) works on v3** — it expands to the introspected column list. (v2 has no column list to expand, so it still requires explicit columns.) -- **Free-text search is `contains()`, not `like`/`ilike`.** The v3 domains - define no LIKE operator — free-text search is bloom-filter token - containment (PostgREST `cs` / SQL `@>`), where `%` is tokenized like any - other character, so a `like` pattern is a category error. Calling `like` or - `ilike` on an encrypted column throws an error pointing at `contains()`; - on plaintext columns both pass through unchanged. -- **`contains()` matches substrings.** The search term blooms to its own +- **Encrypted free-text search is `matches()`, not `contains()`/`like`/`ilike`.** + The v3 domains define no LIKE operator — encrypted free-text search is fuzzy + bloom-filter token matching (PostgREST `cs` / SQL `@>`): one-sided (a match + may be a false positive, a non-match never is) and order-/multiplicity- + insensitive, where `%` is tokenized like any other character. `matches(col, + needle)` is the operator; `contains()` on an encrypted column throws an error + pointing at `matches()`. `contains()` is reserved for native (exact) + jsonb/array containment on plaintext columns, which pass through unchanged. +- **`like`/`ilike` on an encrypted column are an approximate compatibility + shim** delegating to `matches()`: leading/trailing `%` are stripped and the + residual term is fuzzy-matched (same `cs` wire, plus a one-time warning). + Results are APPROXIMATE — case-insensitive, one-sided, and anchoring is not + honored. A pattern with an internal `%` or any `_` cannot be approximated and + throws; call `matches()` directly to make the fuzzy intent explicit. On + plaintext columns `like`/`ilike` stay real SQL LIKE. +- **`matches()` matches substrings.** The search term blooms to its own trigrams, and a row matches when the stored value's bloom contains all of them — so any substring of at least 3 characters (the tokenizer's `token_length`) matches. Shorter terms bloom to nothing and would match every