Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions .changeset/eql-v3-rename-contains-to-matches.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
---
'@cipherstash/stack-drizzle': minor
'@cipherstash/stack-supabase': minor
---

Rename the EQL v3 encrypted free-text operator `contains()` → `matches()` (#617).

Encrypted free-text search is fuzzy bloom-filter token matching — order- and
multiplicity-insensitive and one-sided (a `true` may be a false positive) — not
containment. The name `contains()` promised substring/containment semantics it
never had. It is renamed to `matches()` on the encrypted surface; `contains()` is
kept for genuine, exact containment:

- **Drizzle** (`@cipherstash/stack-drizzle/v3`): `matches()` = bloom free-text on
`text_match`/`text_search` columns; `contains()` = exact encrypted-JSON `@>` on
`types.Json` (ste_vec) columns.
- **Supabase** (`@cipherstash/stack-supabase`): `.matches()` = encrypted free-text;
`.contains()` = native jsonb/array `@>` on plaintext columns (and throws on an
encrypted column, pointing to `matches()`).

Also on the Supabase v3 surface, `like()`/`ilike()` on an encrypted column are no
longer rejected — they are delegated to `matches()` as a best-effort compatibility
shim. This is APPROXIMATE (fuzzy, case-insensitive, one-sided; anchoring and
wildcards are not honored): surrounding `%` are stripped, an internal `%` or any
`_` is rejected, and a one-time warning is emitted. A plaintext column keeps real
SQL LIKE.

Breaking: encrypted `contains()` callers must migrate to `matches()`. The
encrypted operator has not shipped in a stable release (it lands via the EQL v3
work), so there is no deprecation alias.
10 changes: 10 additions & 0 deletions .changeset/stash-skills-contains-to-matches.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
'stash': patch
---

Update the `stash-drizzle` and `stash-supabase` skills for the EQL v3
`contains()` → `matches()` rename (#617): the encrypted free-text operator is now
`matches()` (fuzzy bloom token matching), `contains()` is reserved for exact
containment, and Supabase `like()`/`ilike()` on encrypted columns are documented
as an approximate compatibility shim delegating to `matches()`. Skills ship inside
the `stash` tarball, so they must track the adapter surface.
33 changes: 22 additions & 11 deletions docs/reference/supabase-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,10 @@ The builder surface is shared across v2 and v3:
transforms (`.order/.limit/.range/.single/.maybeSingle/.csv/.abortSignal/.throwOnError`),
plus `.withLockContext(lockContext)` and `.audit(config)` — with one fork:
free-text search. v2 exposes `.like/.ilike` (SQL wildcard matching); v3
exposes `.contains()` (token containment) and rejects `like`/`ilike` on
encrypted columns (see "v3 encoding details" below).
exposes `.matches()` (fuzzy bloom token search) on encrypted columns, keeps
`.contains()` for native (exact) containment on plaintext columns, and treats
`like`/`ilike` on an encrypted column as an approximate shim that delegates to
`.matches()` (see "v3 encoding details" below).

### Typing (v3)

Expand Down Expand Up @@ -84,7 +86,7 @@ const es = await encryptedSupabaseV3(
shape (schema columns get their domain plaintext types — `types.IntegerOrd` →
`number`, `types.TimestampOrd` → `Date`, …). Storage-only columns (e.g.
`types.Boolean`) are excluded from every filter method — including `.match()` —
at the type level, `contains()` is narrowed to match-indexed columns, and
at the type level, `matches()` is narrowed to match-indexed columns, and
`order()` to plaintext columns; filtering a storage-only column is always a
clear runtime error. Undeclared tables behave exactly as with no `schemas` at
all: `from<Row>(tableName)` returns the untyped v3 builder, with `Row`
Expand Down Expand Up @@ -205,14 +207,23 @@ These are internal to the adapter but explain observable behaviour. Envelopes
proxies, and Supabase request logs. The remaining gap is PostgREST operand
casting; until the adapter can reach the query domains (CIP-3402), operands
keep carrying ciphertext.
- **Free-text search is `contains()`; `like`/`ilike` are rejected** on
encrypted columns with an error naming `contains()`. The v3 domains define
no LIKE operator — free-text search is bloom-filter token containment
(`contains()` → PostgREST `cs` → `@>`), where match is tokenized +
downcased and `%` is tokenized like any other character, so a `like`
pattern is a category error. On plaintext columns `like`/`ilike` (and
native `contains`) pass through unchanged.
- **`contains()` matches substrings.** The search term blooms to its own
- **Free-text search is `matches()`; `contains()` on an encrypted column is
rejected** with an error naming `matches()`. The v3 domains define no LIKE
operator — encrypted free-text search is fuzzy bloom-filter token matching
(`matches()` → PostgREST `cs` → `@>`), one-sided (a match may be a false
positive, a non-match never is) and order-/multiplicity-insensitive, where
match is tokenized + downcased and `%` is tokenized like any other character.
`contains()` is reserved for native (exact) jsonb/array containment on
plaintext columns, which pass through unchanged.
- **`like`/`ilike` on an encrypted column are an approximate compatibility
shim** that delegates to `matches()`: leading/trailing `%` are stripped and
the residual term is fuzzy-matched (emitting the same `cs` wire, with a
one-time warning). Because the match is case-insensitive, one-sided, and
anchoring is not honored, the result is APPROXIMATE. A pattern with an
internal `%` or any `_` cannot be approximated and throws; call `matches()`
directly to make the fuzzy intent explicit. On plaintext columns `like`/`ilike`
pass through unchanged as real SQL LIKE.
- **`matches()` matches substrings.** The search term blooms to its own
trigrams, and a row matches when the stored value's bloom contains all of
them — so any substring of at least 3 characters (the tokenizer's
`token_length`) matches. Shorter terms bloom to nothing and would match every
Expand Down
31 changes: 19 additions & 12 deletions packages/stack-drizzle/__tests__/v3/operators.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -358,9 +358,9 @@ describe('createEncryptionOperatorsV3 - comparison & range', () => {
describe('createEncryptionOperatorsV3 - free-text match', () => {
it.each(
matchDomains,
)('%s contains emits latest eql_v3.contains with a query-term operand', async (eqlType, spec) => {
)('%s matches emits latest eql_v3.contains with a query-term operand', async (eqlType, spec) => {
const { ops, encryptQuery, render } = setup()
const q = render(await ops.contains(matrixColumn(eqlType), needleFor(spec)))
const q = render(await ops.matches(matrixColumn(eqlType), needleFor(spec)))

expect(q.sql).toContain(
`eql_v3.contains("matrix_users"."${slug(eqlType)}", $1::${qcast(eqlType)})`,
Expand All @@ -377,28 +377,28 @@ describe('createEncryptionOperatorsV3 - free-text match', () => {
// throw rather than silently return the whole table.
it.each(
matchDomains,
)('%s contains rejects a needle shorter than token_length before encrypting', async (eqlType) => {
)('%s matches rejects a needle shorter than token_length before encrypting', async (eqlType) => {
const { ops, encryptQuery } = setup()
await expect(ops.contains(matrixColumn(eqlType), 'ad')).rejects.toThrow(
await expect(ops.matches(matrixColumn(eqlType), 'ad')).rejects.toThrow(
/at least 3 characters/,
)
await expect(ops.contains(matrixColumn(eqlType), '')).rejects.toThrow(
await expect(ops.matches(matrixColumn(eqlType), '')).rejects.toThrow(
EncryptionOperatorError,
)
expect(encryptQuery).not.toHaveBeenCalled()
})

it('contains accepts a needle exactly at token_length', async () => {
it('matches accepts a needle exactly at token_length', async () => {
const { ops, render } = setup()
const q = render(await ops.contains(users.email, 'ada'))
const q = render(await ops.matches(users.email, 'ada'))
expect(q.sql).toContain(
'eql_v3.contains("users"."email", $1::eql_v3.query_text_search)',
)
})

it('negation is expressed through the passthrough Drizzle not operator', async () => {
const { ops, render } = setup()
const q = render(ops.not(await ops.contains(users.email, 'example.com')))
const q = render(ops.not(await ops.matches(users.email, 'example.com')))
expect(q.sql).toMatch(/^not /i)
expect(q.sql).toContain(
'eql_v3.contains("users"."email", $1::eql_v3.query_text_search)',
Expand Down Expand Up @@ -714,17 +714,24 @@ describe('createEncryptionOperatorsV3 - gating errors', () => {
)
})

it('contains on a column without match throws', async () => {
it('matches on a column without a match index throws', async () => {
const { ops } = setup()
await expect(ops.contains(users.nickname, 'ada')).rejects.toBeInstanceOf(
EncryptionOperatorError,
await expect(ops.matches(users.nickname, 'ada')).rejects.toThrow(
/free-text search/,
)
})

it('contains on a column without JSON containment throws', async () => {
const { ops } = setup()
await expect(ops.contains(users.nickname, 'ada')).rejects.toThrow(
/JSON containment/,
)
})

it('null operands throw and point callers to null checks', async () => {
const { ops } = setup()
await expect(ops.eq(users.nickname, null)).rejects.toThrow(/isNull/)
await expect(ops.contains(users.email, undefined)).rejects.toThrow(/isNull/)
await expect(ops.matches(users.email, undefined)).rejects.toThrow(/isNull/)
})

it('eq on a storage-only column throws', async () => {
Expand Down
6 changes: 3 additions & 3 deletions packages/stack-drizzle/integration/adapter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ const SUPPORTED_OPS: ReadonlySet<QueryOpKind> = new Set([
'lte',
'between',
'notBetween',
'contains',
'matches',
'order',
'isNull',
'isNotNull',
Expand Down Expand Up @@ -90,8 +90,8 @@ export function makeDrizzleAdapter(): IntegrationAdapter {
return ops.between(col, op.lo as never, op.hi as never)
case 'notBetween':
return ops.notBetween(col, op.lo as never, op.hi as never)
case 'contains':
return ops.contains(col, op.needle as never)
case 'matches':
return ops.matches(col, op.needle as never)
case 'isNull':
return ops.isNull(col)
case 'isNotNull':
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
/**
* Live JSON containment for the v3 `types.Json()` column through the Drizzle
* operators. Seeds encrypted JSONB documents and queries them with
* `ops.contains(col, subObject)` — the same operator text search uses, but on a
* `json` column it emits the `@>` operator over the encrypted JSONB document
* (json has no `eql_v3.contains` overload) with a `query_jsonb` needle from
* `encryptQuery` — asserting it returns exactly the rows whose document contains
* the sub-object (jsonb `@>` semantics), and excludes the rest.
* `ops.contains(col, subObject)` — EXACT containment (distinct from the fuzzy
* `ops.matches` used for text search). On a `json` column it emits the `@>`
* operator over the encrypted JSONB document (json has no `eql_v3.contains`
* overload) with a `query_jsonb` needle from `encryptQuery` — asserting it
* returns exactly the rows whose document contains the sub-object (jsonb `@>`
* semantics), and excludes the rest.
*/

import type { JsonDocument } from '@cipherstash/stack/eql/v3'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -323,9 +323,9 @@ describe('v3 drizzle — relational, needle guards, pagination', () => {
['ad', '👍👍', ''].map((needle) => [eqlType, needle] as const),
),
)(
'%s contains rejects the unanswerable needle %j before encrypting',
'%s matches rejects the unanswerable needle %j before encrypting',
async (eqlType, needle) => {
const attempt = ops.contains(matrixColumn(eqlType), needle)
const attempt = ops.matches(matrixColumn(eqlType), needle)
await expect(attempt).rejects.toBeInstanceOf(EncryptionOperatorError)
// Assert the GUARD rejected it, not the encryption layer.
// `operandFailure` also throws `EncryptionOperatorError`, so matching only
Expand Down Expand Up @@ -362,10 +362,10 @@ describe('v3 drizzle — relational, needle guards, pagination', () => {
),
),
)(
'%s contains answers the codepoint-sufficient needle %j with no match',
'%s matches answers the codepoint-sufficient needle %j with no match',
async (eqlType, needle) => {
const rows = await selectRowKeys(
await ops.contains(matrixColumn(eqlType), needle),
await ops.matches(matrixColumn(eqlType), needle),
)
expect(rows).toEqual([])

Expand All @@ -374,7 +374,7 @@ describe('v3 drizzle — relational, needle guards, pagination', () => {
// constant-false `contains` also returns [], and nothing above catches
// it. Prove the same column still answers a needle that IS present.
const present = await selectRowKeys(
await ops.contains(matrixColumn(eqlType), 'ada'),
await ops.matches(matrixColumn(eqlType), 'ada'),
)
expect(present.length).toBeGreaterThan(0)
},
Expand Down
88 changes: 52 additions & 36 deletions packages/stack-drizzle/src/v3/operators.ts
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ type ChainableOperation<T> = {
}

/**
* Build v3-aware query operators (`eq`, `gte`, `contains`, `asc`, …) bound to an
* Build v3-aware query operators (`eq`, `gte`, `matches`, `contains`, `asc`, …) bound to an
* encryption `client`. Each comparison/containment operator AUTO-ENCRYPTS its
* plaintext operand into an EQL v3 query term before handing it to Drizzle, so
* callers pass plaintext and the emitted SQL compares encrypted values. Every
Expand Down Expand Up @@ -266,13 +266,16 @@ export function createEncryptionOperatorsV3(
// order-capable column answers equality via its ordering term too.
const EQUALITY_INDEXES = ['unique', 'ore', 'ope'] as const
const ORDERING_INDEXES = ['ore', 'ope'] as const
// `contains` answers two shapes: bloom free-text (`match`, a `text_search`/
// `text_match` column), which emits `eql_v3.contains(col, operand)`, and
// encrypted-JSONB containment (an `eql_v3_json` column, whose config carries
// the `ste_vec` index kind), which emits the `@>` operator instead — json has
// no `eql_v3.contains` overload. The branch in `contains()` picks the right SQL
// by index kind.
const CONTAINMENT_INDEXES = ['match', 'ste_vec'] as const
// Two DISTINCT operators, split by semantics (#617):
// - `matches` is bloom free-text (`match`, a `text_search`/`text_match`
// column): a one-sided, order- and multiplicity-insensitive token match that
// may false-positive. It emits `eql_v3.contains(col, operand)` (the SQL
// function keeps its bundle name) but is NOT containment.
// - `contains` is encrypted-JSONB containment (an `eql_v3_json` column,
// `ste_vec` index): exact jsonb `@>`, no false positives — genuine
// containment, so it keeps the `contains` name.
const MATCH_INDEXES = ['match'] as const
const JSON_CONTAINMENT_INDEXES = ['ste_vec'] as const

function applyOperationOptions<T>(
op: ChainableOperation<T>,
Expand Down Expand Up @@ -473,37 +476,24 @@ export function createEncryptionOperatorsV3(
return negate ? sql`NOT ${condition}` : condition
}

async function contains(
/**
* Fuzzy free-text token match on a `text_search`/`text_match` column. NOT
* containment: it tests whether the needle's downcased 3-gram set is a subset
* of the haystack's, via a bloom filter — order- and multiplicity-insensitive
* and one-sided (a `true` may be a false positive, a `false` never is). Emits
* `eql_v3.contains(col, operand)` (the SQL function's bundle name).
*/
async function matches(
left: SQLWrapper,
right: unknown,
operator: string,
opts?: EncryptionOperatorCallOpts,
): Promise<SQL> {
const ctx = resolveContext(left, operator)
requireIndex(
ctx,
CONTAINMENT_INDEXES,
operator,
'free-text search or JSON containment',
)

// JSON containment. `eql_v3_json` has no `eql_v3.contains` overload — its
// containment is the `@>` operator, whose `(eql_v3_json, eql_v3.query_jsonb)`
// form takes a NARROWED query term (searchableJson → no ciphertext). The
// needle is cast to `eql_v3.query_jsonb` (an irregular name, so it is handled
// here rather than by `queryCastForDomain`) and emitted with `@>`.
if (ctx.indexes.ste_vec) {
const needle = await encryptJsonContainmentTerm(
ctx,
right,
operator,
opts,
)
return v3Dialect.containsJson(colSql(left), needle)
}

// Bloom free-text (text_search / text_match): the answerable-needle rule
// applies, and the `query_<domain>` cast reaches the match overload.
requireIndex(ctx, MATCH_INDEXES, operator, 'free-text search')
// The answerable-needle rule applies (a sub-`token_length` needle blooms to
// nothing and would match every row); the `query_<domain>` cast reaches the
// match overload.
requireAnswerableNeedle(ctx, right, operator)
const enc = await encryptOperand(
ctx,
Expand All @@ -515,6 +505,25 @@ export function createEncryptionOperatorsV3(
return v3Dialect.contains(colSql(left), enc)
}

/**
* Exact encrypted-JSONB containment on an `eql_v3_json` (`ste_vec`) column:
* genuine jsonb `@>`, no false positives — hence it keeps the `contains` name.
* `eql_v3_json` has no `eql_v3.contains` overload; containment is the `@>`
* operator, whose `(eql_v3_json, eql_v3.query_jsonb)` form takes a NARROWED
* query term (searchableJson → no ciphertext), cast to `eql_v3.query_jsonb`.
*/
async function containsJsonOp(
left: SQLWrapper,
right: unknown,
operator: string,
opts?: EncryptionOperatorCallOpts,
): Promise<SQL> {
const ctx = resolveContext(left, operator)
requireIndex(ctx, JSON_CONTAINMENT_INDEXES, operator, 'JSON containment')
const needle = await encryptJsonContainmentTerm(ctx, right, operator, opts)
return v3Dialect.containsJson(colSql(left), needle)
}

/**
* Build a `query_jsonb` containment needle for a `json` column — the JSON query
* term carries no ciphertext and satisfies the `eql_v3.query_jsonb` CHECK the
Expand Down Expand Up @@ -654,10 +663,17 @@ export function createEncryptionOperatorsV3(
max: unknown,
opts?: EncryptionOperatorCallOpts,
) => range(l, min, max, true, 'notBetween', opts),
/** Free-text containment: emits `eql_v3.contains` over the encrypted match
* term. Encrypts `r`. Requires a `match` (free-text search) index. */
/** Fuzzy free-text token match — the needle's 3-gram set is (bloom-)tested
* as a subset of the column's. NOT containment: order/multiplicity-
* insensitive and one-sided (a `true` may be a false positive). Encrypts
* `r`. Requires a `match` (free-text search) index. */
matches: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) =>
matches(l, r, 'matches', opts),
/** Exact encrypted-JSONB containment (`@>`): matches rows whose document
* contains the given sub-object. No false positives. Encrypts `r`. Requires
* a `ste_vec` index (a `types.Json` column). */
contains: (l: SQLWrapper, r: unknown, opts?: EncryptionOperatorCallOpts) =>
contains(l, r, 'contains', opts),
containsJsonOp(l, r, 'contains', opts),
/** Membership: ORs one encrypted `eq` term per value. The whole list is
* encrypted in one `encryptQuery` batch crossing. Rejects an empty list;
* requires a `unique` or `ore` index. */
Expand Down
Loading
Loading