fix(sync): deliver Mac connection changes to iPhone and iPad (#643)

CHANGELOG.md

@@ -21,6 +21,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Fixed
 
+- Editing a connection on one device no longer overwrites a different field of the same connection edited on another. Each device sent the whole connection on every sync, so the last one to sync won outright. Sync now sends only the fields that actually changed and merges the rest. (#643)
+- A per-connection query timeout set on iPhone or iPad no longer stops that connection from syncing at all. (#643)
+- A device catching up on a lot of iCloud changes at once now downloads all of them. Only the first page of changes was read and the rest were skipped, and the sync position was still moved past them, so those changes never arrived. (#643)
+- iCloud Sync reports a failed sync instead of always reporting success. A rejected change was logged and then dropped, the status went back to idle, and the Last Synced time was stamped as though everything had worked, so there was no way to tell sync was broken. (#643)
+- One rejected change no longer stops every other change in the same batch from syncing. Each change is now confirmed on its own, and only the ones iCloud accepted are marked as sent, so the rest are retried instead of being silently discarded. (#643)
+- Connection changes made on the Mac reach the iPhone and iPad app again. The Mac was sending fields that its iCloud database does not hold, so iCloud rejected every connection it sent while the app still reported the sync as successful. Changes made on iPhone or iPad always reached the Mac, which is why sync looked like it worked in one direction only. (#643)
+- A connection synced from iPhone or iPad keeps its safe mode setting on the Mac. Read-only and confirm-writes connections arrived with no protection at all, because the Mac did not recognise the values the mobile app writes and fell back to Silent. An unrecognised value now asks for confirmation before a write instead of allowing it. (#643)
+- A connection's colour tag set on iPhone or iPad no longer overwrites the colour label on the Mac. Both were stored in the same field despite meaning different things. (#643)
 - **Primary Key**, **Nullable**, **Auto Inc**, and the index **Unique** flag now change when you pick a value. The cell read YES or NO but the menu offered true and false, so picking true did nothing on PostgreSQL, DuckDB, BigQuery, Snowflake, Cassandra, Trino, and SurrealDB. The menu now offers YES and NO to match the cell, in both the new table editor and the structure editor, and no longer offers Set NULL for a flag that has no NULL state. Setting Primary Key to YES now also sets Nullable to NO, since a primary key column cannot hold NULL. (#1991)
 - A message sent in the AI panel right after launch is no longer replaced by the last saved conversation. Restoring that conversation reads from disk in the background, and it used to overwrite whatever you had already typed and sent.
 - A PostgreSQL partitioned table is listed once instead of once per partition. Expand it in the sidebar to see its partitions, and expand one again if it is subpartitioned. A table split into hundreds of partitions no longer buries the rest of the schema, and those partitions no longer fill up query autocomplete, Cmd+K, and the export picker. (#1984)

CLAUDE.md

@@ -151,6 +151,8 @@ When adding a new method to the driver protocol: add to `PluginDatabaseDriver` (
 
 These have caused real bugs when violated:
 
+**A synced CKRecord field must be deployed to Production before anything writes it**: both apps pin `com.apple.developer.icloud-container-environment` to `Production`, and CloudKit only auto-creates fields in the Development environment. So no build, not even a local Debug one, can create a field on the server. Saving a record that carries a field the Production schema does not declare makes CloudKit reject **that whole record**, and with `isAtomic = false` the rest of the batch still saves, so the symptom is one record type silently never syncing. `ConnectionSyncField` (`Packages/TableProCore/Sources/TableProSyncTransport/ConnectionSyncSchema.swift`) is the single declaration of every `Connection` wire key, and its gated `CKRecord` subscript refuses to write a field that is not `.verified`. A new case defaults to `.unverified`, so a field added without the deploy is inert rather than destructive. To ship one: add the field in CloudKit Console, deploy Development to Production, run `scripts/export-cloudkit-schema.sh`, commit the refreshed `CloudKit/production-schema.ckdb`, then mark the field verified. `ProductionSchemaParityTests` fails if the registry and the snapshot disagree in either direction. This shipped as `isFavorite` (#1452, unconditional on every connection) killing every Mac connection push for two months while the UI reported success (#643).
+
 **Sync delete ordering**: In `ConnectionStorage` (and all storage classes), `SyncChangeTracker.markDeleted()` must be called AFTER `saveConnections()`. The `markDeleted` call fires `postChangeNotification` which can trigger a sync. If the file on disk still contains the deleted item when sync runs, it may re-upload the deleted record. Persist first, then notify.
 
 **WelcomeViewModel tree rebuild**: The welcome screen renders `treeItems` (grouped/filtered), not `connections` directly. Every mutation to `connections` must call `rebuildTree()` afterward, or the UI won't update.

CloudKit/production-schema.ckdb

@@ -0,0 +1,152 @@
+DEFINE SCHEMA
+
+    RECORD TYPE AppSettings (
+        "___createTime" TIMESTAMP,
+        "___createdBy"  REFERENCE,
+        "___etag"       STRING,
+        "___modTime"    TIMESTAMP,
+        "___modifiedBy" REFERENCE,
+        "___recordID"   REFERENCE,
+        category        STRING QUERYABLE SEARCHABLE SORTABLE,
+        modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE,
+        schemaVersion   INT64 QUERYABLE SORTABLE,
+        settingsJson    BYTES QUERYABLE SORTABLE,
+        GRANT WRITE TO "_creator",
+        GRANT CREATE TO "_icloud",
+        GRANT READ TO "_world"
+    );
+
+    RECORD TYPE Connection (
+        "___createTime"      TIMESTAMP,
+        "___createdBy"       REFERENCE,
+        "___etag"            STRING,
+        "___modTime"         TIMESTAMP,
+        "___modifiedBy"      REFERENCE,
+        "___recordID"        REFERENCE QUERYABLE,
+        additionalFieldsJson BYTES QUERYABLE SORTABLE,
+        aiPolicy             STRING QUERYABLE SEARCHABLE SORTABLE,
+        color                STRING QUERYABLE SEARCHABLE SORTABLE,
+        colorTag             STRING QUERYABLE SEARCHABLE SORTABLE,
+        connectionId         STRING QUERYABLE SEARCHABLE SORTABLE,
+        database             STRING QUERYABLE SEARCHABLE SORTABLE,
+        groupId              STRING QUERYABLE SEARCHABLE SORTABLE,
+        host                 STRING QUERYABLE SEARCHABLE SORTABLE,
+        isReadOnly           INT64 QUERYABLE SORTABLE,
+        modifiedAtLocal      TIMESTAMP QUERYABLE SORTABLE,
+        name                 STRING QUERYABLE SEARCHABLE SORTABLE,
+        port                 INT64 QUERYABLE SORTABLE,
+        redisDatabase        INT64 QUERYABLE SORTABLE,
+        safeModeLevel        STRING QUERYABLE SEARCHABLE SORTABLE,
+        schemaVersion        INT64 QUERYABLE SORTABLE,
+        sortOrder            INT64 QUERYABLE SORTABLE,
+        sshConfigJson        BYTES QUERYABLE SORTABLE,
+        sshEnabled           INT64 QUERYABLE SORTABLE,
+        sshProfileId         STRING QUERYABLE SEARCHABLE SORTABLE,
+        sslConfigJson        BYTES QUERYABLE SORTABLE,
+        sslEnabled           INT64 QUERYABLE SORTABLE,
+        startupCommands      STRING QUERYABLE SEARCHABLE SORTABLE,
+        tagId                STRING QUERYABLE SEARCHABLE SORTABLE,
+        type                 STRING QUERYABLE SEARCHABLE SORTABLE,
+        username             STRING QUERYABLE SEARCHABLE SORTABLE,
+        GRANT WRITE TO "_creator",
+        GRANT CREATE TO "_icloud",
+        GRANT READ TO "_world"
+    );
+
+    RECORD TYPE ConnectionGroup (
+        "___createTime" TIMESTAMP,
+        "___createdBy"  REFERENCE,
+        "___etag"       STRING,
+        "___modTime"    TIMESTAMP,
+        "___modifiedBy" REFERENCE,
+        "___recordID"   REFERENCE,
+        color           STRING QUERYABLE SEARCHABLE SORTABLE,
+        groupId         STRING QUERYABLE SEARCHABLE SORTABLE,
+        modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE,
+        name            STRING QUERYABLE SEARCHABLE SORTABLE,
+        parentId        STRING QUERYABLE SEARCHABLE SORTABLE,
+        schemaVersion   INT64 QUERYABLE SORTABLE,
+        sortOrder       INT64 QUERYABLE SORTABLE,
+        GRANT WRITE TO "_creator",
+        GRANT CREATE TO "_icloud",
+        GRANT READ TO "_world"
+    );
+
+    RECORD TYPE ConnectionTag (
+        "___createTime" TIMESTAMP,
+        "___createdBy"  REFERENCE,
+        "___etag"       STRING,
+        "___modTime"    TIMESTAMP,
+        "___modifiedBy" REFERENCE,
+        "___recordID"   REFERENCE,
+        color           STRING QUERYABLE SEARCHABLE SORTABLE,
+        isPreset        INT64 QUERYABLE SORTABLE,
+        modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE,
+        name            STRING QUERYABLE SEARCHABLE SORTABLE,
+        schemaVersion   INT64 QUERYABLE SORTABLE,
+        tagId           STRING QUERYABLE SEARCHABLE SORTABLE,
+        GRANT WRITE TO "_creator",
+        GRANT CREATE TO "_icloud",
+        GRANT READ TO "_world"
+    );
+
+    RECORD TYPE QueryHistory (
+        "___createTime" TIMESTAMP,
+        "___createdBy"  REFERENCE,
+        "___etag"       STRING,
+        "___modTime"    TIMESTAMP,
+        "___modifiedBy" REFERENCE,
+        "___recordID"   REFERENCE,
+        connectionId    STRING QUERYABLE SEARCHABLE SORTABLE,
+        databaseName    STRING QUERYABLE SEARCHABLE SORTABLE,
+        entryId         STRING QUERYABLE SEARCHABLE SORTABLE,
+        errorMessage    STRING QUERYABLE SEARCHABLE SORTABLE,
+        executedAt      TIMESTAMP QUERYABLE SORTABLE,
+        executionTime   DOUBLE QUERYABLE SORTABLE,
+        query           STRING QUERYABLE SEARCHABLE SORTABLE,
+        rowCount        INT64 QUERYABLE SORTABLE,
+        schemaVersion   INT64 QUERYABLE SORTABLE,
+        wasSuccessful   INT64 QUERYABLE SORTABLE,
+        GRANT WRITE TO "_creator",
+        GRANT CREATE TO "_icloud",
+        GRANT READ TO "_world"
+    );
+
+    RECORD TYPE SSHProfile (
+        "___createTime" TIMESTAMP,
+        "___createdBy"  REFERENCE,
+        "___etag"       STRING,
+        "___modTime"    TIMESTAMP,
+        "___modifiedBy" REFERENCE,
+        "___recordID"   REFERENCE,
+        agentSocketPath STRING QUERYABLE SEARCHABLE SORTABLE,
+        authMethod      STRING QUERYABLE SEARCHABLE SORTABLE,
+        host            STRING QUERYABLE SEARCHABLE SORTABLE,
+        modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE,
+        name            STRING QUERYABLE SEARCHABLE SORTABLE,
+        port            INT64 QUERYABLE SORTABLE,
+        privateKeyPath  STRING QUERYABLE SEARCHABLE SORTABLE,
+        profileId       STRING QUERYABLE SEARCHABLE SORTABLE,
+        schemaVersion   INT64 QUERYABLE SORTABLE,
+        totpAlgorithm   STRING QUERYABLE SEARCHABLE SORTABLE,
+        totpDigits      INT64 QUERYABLE SORTABLE,
+        totpMode        STRING QUERYABLE SEARCHABLE SORTABLE,
+        totpPeriod      INT64 QUERYABLE SORTABLE,
+        useSSHConfig    INT64 QUERYABLE SORTABLE,
+        username        STRING QUERYABLE SEARCHABLE SORTABLE,
+        GRANT WRITE TO "_creator",
+        GRANT CREATE TO "_icloud",
+        GRANT READ TO "_world"
+    );
+
+    RECORD TYPE Users (
+        "___createTime" TIMESTAMP,
+        "___createdBy"  REFERENCE,
+        "___etag"       STRING,
+        "___modTime"    TIMESTAMP,
+        "___modifiedBy" REFERENCE,
+        "___recordID"   REFERENCE,
+        roles           LIST<INT64>,
+        GRANT WRITE TO "_creator",
+        GRANT READ TO "_world"
+    );

Packages/TableProCore/Package.swift

@@ -15,6 +15,7 @@ let package = Package(
         .library(name: "TableProImport", targets: ["TableProImport"]),
         .library(name: "TableProDatabase", targets: ["TableProDatabase"]),
         .library(name: "TableProQuery", targets: ["TableProQuery"]),
+        .library(name: "TableProSyncTransport", targets: ["TableProSyncTransport"]),
         .library(name: "TableProSync", targets: ["TableProSync"]),
         .library(name: "TableProAnalytics", targets: ["TableProAnalytics"]),
         .library(name: "TableProMSSQLCore", targets: ["TableProMSSQLCore"]),
@@ -53,9 +54,14 @@ let package = Package(
             dependencies: ["TableProModels", "TableProPluginKit", "TableProCoreTypes"],
             path: "Sources/TableProQuery"
         ),
+        .target(
+            name: "TableProSyncTransport",
+            dependencies: [],
+            path: "Sources/TableProSyncTransport"
+        ),
         .target(
             name: "TableProSync",
-            dependencies: ["TableProModels", "TableProCoreTypes"],
+            dependencies: ["TableProSyncTransport", "TableProModels", "TableProCoreTypes"],
             path: "Sources/TableProSync"
         ),
         .target(
@@ -120,7 +126,7 @@ let package = Package(
         ),
         .testTarget(
             name: "TableProSyncTests",
-            dependencies: ["TableProSync", "TableProModels"],
+            dependencies: ["TableProSync", "TableProSyncTransport", "TableProModels"],
             path: "Tests/TableProSyncTests"
         ),
         .testTarget(