fix: oceanbase identity collation collapses case-variant scope_id and source_id

[thunguo]

Which issue or RFC does this PR close?

Closes #1276.

Rationale for this change

scope_id and source_id are opaque, byte-exact identities. Shared schema columns were declared as dialect-neutral String(...) with no collation, so MySQL/OceanBase inherited the server default utf8mb4_general_ci. SQLite compares the same columns with BINARY semantics, which is why the SQLite test matrix never caught this.

On OceanBase that collation collapses case-variant and accent-variant keys:

• scope_id="Alpha" and "alpha" are one Scope, so listing the second returns the first Scope's Memory.
• source_id="Turn-1" and "turn-1" are one Source, so a legitimate second capture is rejected as 409 source_conflict and that turn never enters the Source journal.

What changes are included in this PR?

• Add identity_string(length) in tables.py. SQLite keeps String; MySQL/OceanBase emit VARCHAR(..., charset="utf8mb4", collation="utf8mb4_bin").
• Use that type for identity columns in the shared persistence schema, Memory search indexes, and Handoff Report catalog / activity / workspace tables.
• Assert compiled MySQL DDL includes CHARACTER SET utf8mb4 COLLATE utf8mb4_bin for scope_id, source_id, and source_type.
• Add a SQLite and OceanBase parametrized e2e test that writes Memory under a case-variant and an accent-variant scope_id, asserts the other variant lists no entries, and captures both Turn-1 and turn-1.

create_all(checkfirst=True) does not rewrite existing column collations, and OceanBase rejects ALTER COLUMN ... COLLATE when foreign keys exist. Existing MySQL/OceanBase schemas must be recreated on a new database (or by dropping and recreating the tables). Startup does not run an in-place ALTER.

Are there any user-facing changes?

Public HTTP and SDK APIs are unchanged. On a fresh MySQL/OceanBase schema, identity comparison now matches SQLite and the Runtime: case-variant and accent-variant scope_id / source_id values are distinct.

This is a persisted-schema break for existing OceanBase/MySQL deployments. Databases created before this change keep utf8mb4_general_ci on identity columns and will still leak or reject those variants until the schema is recreated. SQLite files are unaffected.

How was this change tested?

• uv run pytest -q tests/builtin/persistence/test_mysql_schema.py tests/builtin/persistence/test_memory.py tests/test_handoff_report_catalog.py tests/test_handoff_report_workspace.py tests/test_handoff_report_repository.py
• POWERCONTEXT_TEST_OCEANBASE_URL=... uv run pytest -q tests/e2e/test_runtime_server.py::test_server_databases_keep_case_and_accent_variant_identities_distinct (sqlite + oceanbase)
• Live OceanBase SHOW CREATE TABLE pc_sources on a newly created database: scope_id, source_id, and source_type are varchar(...) COLLATE utf8mb4_bin
• Re-ran the bug: OceanBase identity collation collapses case-variant scope_id and source_id #1276 reproduction against that new schema: listing alpha after writing Alpha returned 0 entries; capturing turn-1 after Turn-1 was accepted

The OceanBase checks used a dedicated empty database. An older schema on the same instance still showed utf8mb4_general_ci on source_id and would still reproduce #1276.

AI usage statement

[Copilot]

Pull request overview

This PR fixes a MySQL/OceanBase collation mismatch where identity-like keys (e.g., scope_id, source_id) were being compared case-/accent-insensitively due to inheriting utf8mb4_general_ci, causing cross-scope leaks and false 409 source_conflict collisions. It introduces an explicit binary collation for identity string columns so MySQL/OceanBase semantics match SQLite’s byte-exact behavior.

Changes:

• Added identity_string(length) to emit VARCHAR(... CHARACTER SET utf8mb4 COLLATE utf8mb4_bin) on MySQL/OceanBase while keeping String(...) on SQLite.
• Switched shared persistence tables, Memory search-index tables, and Handoff Report tables to use identity_string for identity-like columns.
• Added MySQL DDL compilation assertions and an e2e regression test covering case-variant and accent-variant identity handling across SQLite and OceanBase.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/powercontext/builtin/persistence/tables.py	Introduces identity_string() and applies it to shared schema identity columns to enforce binary collation on MySQL/OceanBase.
src/powercontext/builtin/persistence/sqlite/memory_index.py	Updates SQLite Memory vector index schema to use identity_string for key columns.
src/powercontext/builtin/persistence/oceanbase/memory_index.py	Updates OceanBase vector index table schema to use identity_string for key columns.
src/powercontext/builtin/handoff_report/workspace_store.py	Switches Workspace Binding table string columns to identity_string to ensure byte-exact identity comparisons on MySQL/OceanBase.
src/powercontext/builtin/handoff_report/sqlite.py	Switches Handoff Report activity tables to identity_string for identity-like string columns.
src/powercontext/builtin/handoff_report/catalog_store.py	Switches Handoff Report catalog tables to identity_string for identity-like string columns.
tests/builtin/persistence/test_mysql_schema.py	Adds DDL assertions ensuring identity columns compile with utf8mb4_bin collation.
tests/builtin/persistence/test_memory.py	Adds a DDL assertion for scope_id collation in the Memory schema.
tests/e2e/test_runtime_server.py	Adds an end-to-end regression test validating case/accent variant identity isolation on SQLite and OceanBase.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.