fix: improve schema check. Add turso db checks

.changeset/quiet-tables-load.md

@@ -0,0 +1,8 @@
+---
+"@will-be-done/hyperdb": patch
+---
+
+Speed up repeated SQLite `loadTables()` calls with persisted schema signatures
+and make SQLite schema reconciliation handle identifiers case-insensitively.
+Ensure rejected async driver commands run generator cleanup, including
+transaction rollback and lock release.

README.md

@@ -173,6 +173,9 @@ export async function createAppDB() {
 }
 ```
 
+`AsyncSqlDriver` is also exercised against Turso Database's browser WASM
+engine in the shared driver conformance suite.
+
 If your whole app state can be loaded into memory at startup, you may not need
 `HybridDB`. A plain `new SubscribableDB(new DB(new BptreeInmemDriver()))` keeps
 reads and writes synchronous, so you can use `useSyncSelector`, `useSyncDispatch`,

packages/hyperdb-doc/src/content/docs/runtime/drivers.md

@@ -13,12 +13,12 @@ the sync or async runtime helpers, which depends on the storage path.
 
 ## Choosing a driver
 
-| Driver              | Import                 | Mode  | Environment | Use for                                                        |
-| ------------------- | ---------------------- | ----- | ----------- | -------------------------------------------------------------- |
-| `BptreeInmemDriver` | `.../drivers/inmemory` | sync  | both        | Tests, fully loaded app state, and the fast `HybridDB` cache   |
-| `IdbDriver`         | `.../drivers/idb`      | async | browser     | Browser persistence, usually as a `HybridDB` primary store     |
-| `SqlDriver`         | `.../drivers/sqlite`   | sync  | both        | Any synchronous SQLite binding (native server SQLite, sql.js)  |
-| `AsyncSqlDriver`    | `.../drivers/sqlite`   | async | both        | Async SQLite, including browser SQLite as a `HybridDB` primary |
+| Driver              | Import                 | Mode  | Environment | Use for                                                       |
+| ------------------- | ---------------------- | ----- | ----------- | ------------------------------------------------------------- |
+| `BptreeInmemDriver` | `.../drivers/inmemory` | sync  | both        | Tests, fully loaded app state, and the fast `HybridDB` cache  |
+| `IdbDriver`         | `.../drivers/idb`      | async | browser     | Browser persistence, usually as a `HybridDB` primary store    |
+| `SqlDriver`         | `.../drivers/sqlite`   | sync  | both        | Any synchronous SQLite binding (native server SQLite, sql.js) |
+| `AsyncSqlDriver`    | `.../drivers/sqlite`   | async | both        | Async SQLite, including Turso WASM as a `HybridDB` primary    |
 
 Sync drivers work with `execSync` / `syncDispatch` / `selectSync`. Async drivers
 require `execAsync` / `asyncDispatch` / `selectAsync`. `HybridDB` also uses the
@@ -107,12 +107,25 @@ not prepare or emit SQL diagnostic events. Use
 `formatAsyncSqlDriverDebugEvent(event)` when you want the old one-line message
 but need to send it to a custom logger.
 
+HyperDB's shared driver conformance suite runs `AsyncSqlDriver` against the
+browser build of Turso Database (`@tursodatabase/database-wasm`) in Chromium,
+covering the same runtime behavior as the SQL.js, IndexedDB, and in-memory
+drivers.
+
 The SQLite storage codec encodes `bigint`, `ArrayBuffer`, and
 typed-array/data-view values around JSON storage so they round-trip exactly.
 `uniqhash` indexes are created as SQLite `UNIQUE` indexes. Upserts delete the
 same primary keys first and then insert the new rows, so a secondary unique
 conflict throws instead of replacing a different row.
 
+The SQLite drivers persist each table's physical-layout signature in the
+reserved `_hyperdb_schema_metadata` table. After the first successful
+`loadTables()` reconciliation, loading unchanged definitions takes one metadata
+read instead of inspecting and rebuilding every table and index. The stored
+signature is tied to SQLite's `schema_version`, so external schema changes make
+the next load run the full transactional reconciliation and refresh the
+metadata only after it succeeds.
+
 ## SQLite Recipes
 
 ### SQL.js sync

packages/hyperdb-doc/src/content/docs/start/llm-cheat-sheet.md

@@ -42,7 +42,8 @@ and views that should re-run only when the exact index ranges they read change.
   trace metadata helpers.
 - `@will-be-done/hyperdb/drivers/inmemory`: `BptreeInmemDriver`.
 - `@will-be-done/hyperdb/drivers/sqlite`: `SqlDriver`, `AsyncSqlDriver`, and
-  SQLite adapter types.
+  SQLite adapter types. `AsyncSqlDriver` is compatible with Turso Database's
+  browser WASM build.
 - `@will-be-done/hyperdb/drivers/idb`: `openIndexedDBDriver`, `IdbDriver`, and
   IndexedDB driver options.
 - `@will-be-done/hyperdb-devtool/react`: separate package with
@@ -285,6 +286,11 @@ preparation. Use `logIdbDriverDebugEvent` from
 console style, and `logAsyncSqlDriverDebugEvent` from
 `@will-be-done/hyperdb/drivers/sqlite` for async SQLite.
 
+SQLite `loadTables()` stores physical-layout signatures in the reserved
+`_hyperdb_schema_metadata` table. Unchanged definitions use a one-query fast
+path; a changed definition or SQLite `schema_version` triggers full
+transactional schema reconciliation.
+
 Use a B-tree full-scan index such as `byIds` for `preloadTables`; the built-in
 `byId` index is a `uniqhash` index for exact id lookups. `SubscribableDB` adds
 revisions, subscriptions, selector invalidation, and lifecycle hooks. Pure

packages/hyperdb/package.json

@@ -66,6 +66,7 @@
   },
   "devDependencies": {
     "@eslint/js": "^9.39.1",
+    "@tursodatabase/database-wasm": "0.7.1",
     "@types/react": "19.2.17",
     "@types/react-dom": "19.2.3",
     "@types/sql.js": "^1.4.9",

packages/hyperdb/src/hyperdb/core/executor.test.ts

@@ -38,6 +38,36 @@ describe("executor", () => {
     await expect(execAsync(command())).rejects.toThrow("boom");
   });
 
+  it("throws rejected async commands back into generators", async () => {
+    let cleanedUp = false;
+
+    function* command(): Generator<DBCmd, void, unknown> {
+      try {
+        yield* unwrap(Promise.reject(new Error("boom")));
+      } finally {
+        cleanedUp = true;
+      }
+    }
+
+    await expect(execAsync(command())).rejects.toThrow("boom");
+    expect(cleanedUp).toBe(true);
+  });
+
+  it("throws rejected execMaybeAsync commands back into generators", async () => {
+    let cleanedUp = false;
+
+    function* command(): Generator<DBCmd, void, unknown> {
+      try {
+        yield* unwrap(Promise.reject(new Error("boom")));
+      } finally {
+        cleanedUp = true;
+      }
+    }
+
+    await expect(execMaybeAsync(command())).rejects.toThrow("boom");
+    expect(cleanedUp).toBe(true);
+  });
+
   it("continues through noop commands in execSync", () => {
     function* command(): Generator<DBCmd, string> {
       yield* noop();

packages/hyperdb/src/hyperdb/core/executor.ts

@@ -35,11 +35,11 @@ async function execAsyncFrom<T>(
   cmd: Generator<DBCmd, T>,
   value: unknown,
 ): Promise<T> {
-  let result = cmd.next(await value);
+  let result = await resumeAfterAsyncCommand(cmd, value);
 
   while (!result.done) {
     if (isUnwrapCmd(result.value)) {
-      result = cmd.next(await result.value.data);
+      result = await resumeAfterAsyncCommand(cmd, result.value.data);
     } else if (isNoopCmd(result.value)) {
       result = cmd.next();
     } else {
@@ -50,12 +50,26 @@ async function execAsyncFrom<T>(
   return result.value as T;
 }
 
+async function resumeAfterAsyncCommand<T>(
+  cmd: Generator<DBCmd, T>,
+  data: unknown,
+): Promise<IteratorResult<DBCmd, T>> {
+  let value: unknown;
+  try {
+    value = await data;
+  } catch (error) {
+    return cmd.throw(error);
+  }
+
+  return cmd.next(value);
+}
+
 export async function execAsync<T>(cmd: Generator<DBCmd, T>): Promise<T> {
   let result = cmd.next();
 
   while (!result.done) {
     if (isUnwrapCmd(result.value)) {
-      result = cmd.next(await result.value.data);
+      result = await resumeAfterAsyncCommand(cmd, result.value.data);
     } else if (isNoopCmd(result.value)) {
       result = cmd.next();
     } else {

packages/hyperdb/src/hyperdb/drivers/sqlite/async-sql-driver.test.ts

@@ -3,11 +3,16 @@ import { defineTable } from "../../schema/table";
 import { execAsync } from "../../core/executor";
 import { DB } from "../../runtime/db";
 import { v } from "../../schema/values";
-import { createSqlJsAsyncDriver } from "../../test-utils/sql-js-driver";
+import {
+  createInspectableSqlAsyncDriver,
+  createSqlJsAsyncDriver,
+} from "../../test-utils/sql-js-driver";
+import { createTursoWasmDriver } from "../../test-utils/turso-wasm-driver";
 import {
   AsyncSqlDriver,
   formatAsyncSqlDriverDebugEvent,
 } from "./async-sql-driver";
+import { SQLITE_SCHEMA_METADATA_TABLE } from "./sqlite-common";
 
 type Task = {
   type: "task";
@@ -89,7 +94,94 @@ describe("db", async () => {
     ]);
   });
 
-  for (const driver of [createSqlJsAsyncDriver]) {
+  it("uses one metadata read when async table schemas are unchanged", async () => {
+    const { driver, execLog } = await createInspectableSqlAsyncDriver();
+    const db = new DB(driver);
+
+    await execAsync(db.loadTables([tasksTable]));
+    execLog.length = 0;
+    await execAsync(db.loadTables([tasksTable]));
+
+    expect(execLog).toHaveLength(1);
+    expect(execLog[0]).toContain(`LEFT JOIN ${SQLITE_SCHEMA_METADATA_TABLE}`);
+    expect(execLog[0]).not.toContain("BEGIN TRANSACTION");
+  });
+
+  it("initializes async schema metadata when the adapter hides missing-table details", async () => {
+    let obscureMetadataReadError = true;
+    const { driver, sqldb, execLog } = await createInspectableSqlAsyncDriver(
+      {},
+      {
+        beforePrepare(sql) {
+          if (
+            obscureMetadataReadError &&
+            sql.includes(`LEFT JOIN ${SQLITE_SCHEMA_METADATA_TABLE}`)
+          ) {
+            throw new Error("statement preparation failed");
+          }
+        },
+      },
+    );
+    const db = new DB(driver);
+
+    await execAsync(db.loadTables([tasksTable]));
+    expect(
+      sqldb.exec(
+        `SELECT name FROM sqlite_schema WHERE name = '${SQLITE_SCHEMA_METADATA_TABLE}'`,
+      )[0]?.values,
+    ).toEqual([[SQLITE_SCHEMA_METADATA_TABLE]]);
+
+    await expect(execAsync(db.loadTables([tasksTable]))).rejects.toThrow(
+      "statement preparation failed",
+    );
+
+    obscureMetadataReadError = false;
+    execLog.length = 0;
+    await execAsync(db.loadTables([tasksTable]));
+    expect(execLog).toHaveLength(1);
+    expect(execLog[0]).toContain(`LEFT JOIN ${SQLITE_SCHEMA_METADATA_TABLE}`);
+  });
+
+  it("treats async SQLite index identifiers as case-insensitive", async () => {
+    const { driver, sqldb, execLog } = await createInspectableSqlAsyncDriver();
+    const db = new DB(driver);
+
+    await execAsync(db.loadTables([tasksTable]));
+    sqldb.exec("DROP INDEX idx_tasks_byTitle_sort_key_v2");
+    sqldb.exec(
+      "CREATE INDEX idx_tasks_bytitle_sort_key_v2 ON tasks(idx_byTitle_sort_key_v2, id) WHERE idx_byTitle_sort_key_v2 IS NOT NULL",
+    );
+    execLog.length = 0;
+
+    await execAsync(db.loadTables([tasksTable]));
+
+    expect(execLog.some((sql) => sql.startsWith("DROP INDEX"))).toBe(false);
+    expect(
+      (sqldb.exec("PRAGMA index_list(tasks)")[0]?.values ?? []).some(
+        (row) => String(row[1]) === "idx_tasks_bytitle_sort_key_v2",
+      ),
+    ).toBe(true);
+  });
+
+  it("uses the schema metadata fast path with Turso WASM", async () => {
+    const debug = vi.fn();
+    const db = new DB(await createTursoWasmDriver({ debug }));
+
+    await execAsync(db.loadTables([tasksTable]));
+    debug.mockClear();
+    await execAsync(db.loadTables([tasksTable]));
+
+    expect(debug).toHaveBeenCalledTimes(1);
+    expect(debug.mock.calls[0]?.[0]).toMatchObject({
+      operation: "scan",
+      status: "success",
+    });
+    expect(debug.mock.calls[0]?.[0].sql).toContain(
+      `LEFT JOIN ${SQLITE_SCHEMA_METADATA_TABLE}`,
+    );
+  });
+
+  for (const driver of [createSqlJsAsyncDriver, createTursoWasmDriver]) {
     it("preserves physical indexes whose logical names end in _sort_key", async () => {
       const debug = vi.fn();
       const db = new DB(await driver({ debug }));