diff --git a/CHANGELOG.md b/CHANGELOG.md index f5dfb833d..a7a4b15ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Fixed +- Editing a connection on one device no longer overwrites a different field of the same connection edited on another. Each device sent the whole connection on every sync, so the last one to sync won outright. Sync now sends only the fields that actually changed and merges the rest. (#643) +- A per-connection query timeout set on iPhone or iPad no longer stops that connection from syncing at all. (#643) +- A device catching up on a lot of iCloud changes at once now downloads all of them. Only the first page of changes was read and the rest were skipped, and the sync position was still moved past them, so those changes never arrived. (#643) +- iCloud Sync reports a failed sync instead of always reporting success. A rejected change was logged and then dropped, the status went back to idle, and the Last Synced time was stamped as though everything had worked, so there was no way to tell sync was broken. (#643) +- One rejected change no longer stops every other change in the same batch from syncing. Each change is now confirmed on its own, and only the ones iCloud accepted are marked as sent, so the rest are retried instead of being silently discarded. (#643) +- Connection changes made on the Mac reach the iPhone and iPad app again. The Mac was sending fields that its iCloud database does not hold, so iCloud rejected every connection it sent while the app still reported the sync as successful. Changes made on iPhone or iPad always reached the Mac, which is why sync looked like it worked in one direction only. (#643) +- A connection synced from iPhone or iPad keeps its safe mode setting on the Mac. Read-only and confirm-writes connections arrived with no protection at all, because the Mac did not recognise the values the mobile app writes and fell back to Silent. An unrecognised value now asks for confirmation before a write instead of allowing it. (#643) +- A connection's colour tag set on iPhone or iPad no longer overwrites the colour label on the Mac. Both were stored in the same field despite meaning different things. (#643) - **Primary Key**, **Nullable**, **Auto Inc**, and the index **Unique** flag now change when you pick a value. The cell read YES or NO but the menu offered true and false, so picking true did nothing on PostgreSQL, DuckDB, BigQuery, Snowflake, Cassandra, Trino, and SurrealDB. The menu now offers YES and NO to match the cell, in both the new table editor and the structure editor, and no longer offers Set NULL for a flag that has no NULL state. Setting Primary Key to YES now also sets Nullable to NO, since a primary key column cannot hold NULL. (#1991) - A message sent in the AI panel right after launch is no longer replaced by the last saved conversation. Restoring that conversation reads from disk in the background, and it used to overwrite whatever you had already typed and sent. - A PostgreSQL partitioned table is listed once instead of once per partition. Expand it in the sidebar to see its partitions, and expand one again if it is subpartitioned. A table split into hundreds of partitions no longer buries the rest of the schema, and those partitions no longer fill up query autocomplete, Cmd+K, and the export picker. (#1984) diff --git a/CLAUDE.md b/CLAUDE.md index 07412b461..23957042d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -151,6 +151,8 @@ When adding a new method to the driver protocol: add to `PluginDatabaseDriver` ( These have caused real bugs when violated: +**A synced CKRecord field must be deployed to Production before anything writes it**: both apps pin `com.apple.developer.icloud-container-environment` to `Production`, and CloudKit only auto-creates fields in the Development environment. So no build, not even a local Debug one, can create a field on the server. Saving a record that carries a field the Production schema does not declare makes CloudKit reject **that whole record**, and with `isAtomic = false` the rest of the batch still saves, so the symptom is one record type silently never syncing. `ConnectionSyncField` (`Packages/TableProCore/Sources/TableProSyncTransport/ConnectionSyncSchema.swift`) is the single declaration of every `Connection` wire key, and its gated `CKRecord` subscript refuses to write a field that is not `.verified`. A new case defaults to `.unverified`, so a field added without the deploy is inert rather than destructive. To ship one: add the field in CloudKit Console, deploy Development to Production, run `scripts/export-cloudkit-schema.sh`, commit the refreshed `CloudKit/production-schema.ckdb`, then mark the field verified. `ProductionSchemaParityTests` fails if the registry and the snapshot disagree in either direction. This shipped as `isFavorite` (#1452, unconditional on every connection) killing every Mac connection push for two months while the UI reported success (#643). + **Sync delete ordering**: In `ConnectionStorage` (and all storage classes), `SyncChangeTracker.markDeleted()` must be called AFTER `saveConnections()`. The `markDeleted` call fires `postChangeNotification` which can trigger a sync. If the file on disk still contains the deleted item when sync runs, it may re-upload the deleted record. Persist first, then notify. **WelcomeViewModel tree rebuild**: The welcome screen renders `treeItems` (grouped/filtered), not `connections` directly. Every mutation to `connections` must call `rebuildTree()` afterward, or the UI won't update. diff --git a/CloudKit/production-schema.ckdb b/CloudKit/production-schema.ckdb new file mode 100644 index 000000000..dcc9db045 --- /dev/null +++ b/CloudKit/production-schema.ckdb @@ -0,0 +1,152 @@ +DEFINE SCHEMA + + RECORD TYPE AppSettings ( + "___createTime" TIMESTAMP, + "___createdBy" REFERENCE, + "___etag" STRING, + "___modTime" TIMESTAMP, + "___modifiedBy" REFERENCE, + "___recordID" REFERENCE, + category STRING QUERYABLE SEARCHABLE SORTABLE, + modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE, + schemaVersion INT64 QUERYABLE SORTABLE, + settingsJson BYTES QUERYABLE SORTABLE, + GRANT WRITE TO "_creator", + GRANT CREATE TO "_icloud", + GRANT READ TO "_world" + ); + + RECORD TYPE Connection ( + "___createTime" TIMESTAMP, + "___createdBy" REFERENCE, + "___etag" STRING, + "___modTime" TIMESTAMP, + "___modifiedBy" REFERENCE, + "___recordID" REFERENCE QUERYABLE, + additionalFieldsJson BYTES QUERYABLE SORTABLE, + aiPolicy STRING QUERYABLE SEARCHABLE SORTABLE, + color STRING QUERYABLE SEARCHABLE SORTABLE, + colorTag STRING QUERYABLE SEARCHABLE SORTABLE, + connectionId STRING QUERYABLE SEARCHABLE SORTABLE, + database STRING QUERYABLE SEARCHABLE SORTABLE, + groupId STRING QUERYABLE SEARCHABLE SORTABLE, + host STRING QUERYABLE SEARCHABLE SORTABLE, + isReadOnly INT64 QUERYABLE SORTABLE, + modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE, + name STRING QUERYABLE SEARCHABLE SORTABLE, + port INT64 QUERYABLE SORTABLE, + redisDatabase INT64 QUERYABLE SORTABLE, + safeModeLevel STRING QUERYABLE SEARCHABLE SORTABLE, + schemaVersion INT64 QUERYABLE SORTABLE, + sortOrder INT64 QUERYABLE SORTABLE, + sshConfigJson BYTES QUERYABLE SORTABLE, + sshEnabled INT64 QUERYABLE SORTABLE, + sshProfileId STRING QUERYABLE SEARCHABLE SORTABLE, + sslConfigJson BYTES QUERYABLE SORTABLE, + sslEnabled INT64 QUERYABLE SORTABLE, + startupCommands STRING QUERYABLE SEARCHABLE SORTABLE, + tagId STRING QUERYABLE SEARCHABLE SORTABLE, + type STRING QUERYABLE SEARCHABLE SORTABLE, + username STRING QUERYABLE SEARCHABLE SORTABLE, + GRANT WRITE TO "_creator", + GRANT CREATE TO "_icloud", + GRANT READ TO "_world" + ); + + RECORD TYPE ConnectionGroup ( + "___createTime" TIMESTAMP, + "___createdBy" REFERENCE, + "___etag" STRING, + "___modTime" TIMESTAMP, + "___modifiedBy" REFERENCE, + "___recordID" REFERENCE, + color STRING QUERYABLE SEARCHABLE SORTABLE, + groupId STRING QUERYABLE SEARCHABLE SORTABLE, + modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE, + name STRING QUERYABLE SEARCHABLE SORTABLE, + parentId STRING QUERYABLE SEARCHABLE SORTABLE, + schemaVersion INT64 QUERYABLE SORTABLE, + sortOrder INT64 QUERYABLE SORTABLE, + GRANT WRITE TO "_creator", + GRANT CREATE TO "_icloud", + GRANT READ TO "_world" + ); + + RECORD TYPE ConnectionTag ( + "___createTime" TIMESTAMP, + "___createdBy" REFERENCE, + "___etag" STRING, + "___modTime" TIMESTAMP, + "___modifiedBy" REFERENCE, + "___recordID" REFERENCE, + color STRING QUERYABLE SEARCHABLE SORTABLE, + isPreset INT64 QUERYABLE SORTABLE, + modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE, + name STRING QUERYABLE SEARCHABLE SORTABLE, + schemaVersion INT64 QUERYABLE SORTABLE, + tagId STRING QUERYABLE SEARCHABLE SORTABLE, + GRANT WRITE TO "_creator", + GRANT CREATE TO "_icloud", + GRANT READ TO "_world" + ); + + RECORD TYPE QueryHistory ( + "___createTime" TIMESTAMP, + "___createdBy" REFERENCE, + "___etag" STRING, + "___modTime" TIMESTAMP, + "___modifiedBy" REFERENCE, + "___recordID" REFERENCE, + connectionId STRING QUERYABLE SEARCHABLE SORTABLE, + databaseName STRING QUERYABLE SEARCHABLE SORTABLE, + entryId STRING QUERYABLE SEARCHABLE SORTABLE, + errorMessage STRING QUERYABLE SEARCHABLE SORTABLE, + executedAt TIMESTAMP QUERYABLE SORTABLE, + executionTime DOUBLE QUERYABLE SORTABLE, + query STRING QUERYABLE SEARCHABLE SORTABLE, + rowCount INT64 QUERYABLE SORTABLE, + schemaVersion INT64 QUERYABLE SORTABLE, + wasSuccessful INT64 QUERYABLE SORTABLE, + GRANT WRITE TO "_creator", + GRANT CREATE TO "_icloud", + GRANT READ TO "_world" + ); + + RECORD TYPE SSHProfile ( + "___createTime" TIMESTAMP, + "___createdBy" REFERENCE, + "___etag" STRING, + "___modTime" TIMESTAMP, + "___modifiedBy" REFERENCE, + "___recordID" REFERENCE, + agentSocketPath STRING QUERYABLE SEARCHABLE SORTABLE, + authMethod STRING QUERYABLE SEARCHABLE SORTABLE, + host STRING QUERYABLE SEARCHABLE SORTABLE, + modifiedAtLocal TIMESTAMP QUERYABLE SORTABLE, + name STRING QUERYABLE SEARCHABLE SORTABLE, + port INT64 QUERYABLE SORTABLE, + privateKeyPath STRING QUERYABLE SEARCHABLE SORTABLE, + profileId STRING QUERYABLE SEARCHABLE SORTABLE, + schemaVersion INT64 QUERYABLE SORTABLE, + totpAlgorithm STRING QUERYABLE SEARCHABLE SORTABLE, + totpDigits INT64 QUERYABLE SORTABLE, + totpMode STRING QUERYABLE SEARCHABLE SORTABLE, + totpPeriod INT64 QUERYABLE SORTABLE, + useSSHConfig INT64 QUERYABLE SORTABLE, + username STRING QUERYABLE SEARCHABLE SORTABLE, + GRANT WRITE TO "_creator", + GRANT CREATE TO "_icloud", + GRANT READ TO "_world" + ); + + RECORD TYPE Users ( + "___createTime" TIMESTAMP, + "___createdBy" REFERENCE, + "___etag" STRING, + "___modTime" TIMESTAMP, + "___modifiedBy" REFERENCE, + "___recordID" REFERENCE, + roles LIST, + GRANT WRITE TO "_creator", + GRANT READ TO "_world" + ); diff --git a/Packages/TableProCore/Package.swift b/Packages/TableProCore/Package.swift index 4017c1b79..9325ef5b8 100644 --- a/Packages/TableProCore/Package.swift +++ b/Packages/TableProCore/Package.swift @@ -15,6 +15,7 @@ let package = Package( .library(name: "TableProImport", targets: ["TableProImport"]), .library(name: "TableProDatabase", targets: ["TableProDatabase"]), .library(name: "TableProQuery", targets: ["TableProQuery"]), + .library(name: "TableProSyncTransport", targets: ["TableProSyncTransport"]), .library(name: "TableProSync", targets: ["TableProSync"]), .library(name: "TableProAnalytics", targets: ["TableProAnalytics"]), .library(name: "TableProMSSQLCore", targets: ["TableProMSSQLCore"]), @@ -53,9 +54,14 @@ let package = Package( dependencies: ["TableProModels", "TableProPluginKit", "TableProCoreTypes"], path: "Sources/TableProQuery" ), + .target( + name: "TableProSyncTransport", + dependencies: [], + path: "Sources/TableProSyncTransport" + ), .target( name: "TableProSync", - dependencies: ["TableProModels", "TableProCoreTypes"], + dependencies: ["TableProSyncTransport", "TableProModels", "TableProCoreTypes"], path: "Sources/TableProSync" ), .target( @@ -120,7 +126,7 @@ let package = Package( ), .testTarget( name: "TableProSyncTests", - dependencies: ["TableProSync", "TableProModels"], + dependencies: ["TableProSync", "TableProSyncTransport", "TableProModels"], path: "Tests/TableProSyncTests" ), .testTarget( diff --git a/Packages/TableProCore/Sources/TableProSync/SyncConflict.swift b/Packages/TableProCore/Sources/TableProSync/SyncConflict.swift deleted file mode 100644 index 786537c26..000000000 --- a/Packages/TableProCore/Sources/TableProSync/SyncConflict.swift +++ /dev/null @@ -1,32 +0,0 @@ -import CloudKit -import Foundation - -public struct SyncConflict: Identifiable, Sendable { - public let id: UUID - public let recordType: SyncRecordType - public let entityName: String - public let localModifiedAt: Date - public let serverModifiedAt: Date - public let serverRecord: CKRecord - - public init( - recordType: SyncRecordType, - entityName: String, - localModifiedAt: Date, - serverModifiedAt: Date, - serverRecord: CKRecord - ) { - self.id = UUID() - self.recordType = recordType - self.entityName = entityName - self.localModifiedAt = localModifiedAt - self.serverModifiedAt = serverModifiedAt - self.serverRecord = serverRecord - } -} - -public enum SyncStatus: Equatable, Sendable { - case idle - case syncing - case error(String) -} diff --git a/Packages/TableProCore/Sources/TableProSync/SyncRecordMapper.swift b/Packages/TableProCore/Sources/TableProSync/SyncRecordMapper.swift index eb77ba0b4..044abacba 100644 --- a/Packages/TableProCore/Sources/TableProSync/SyncRecordMapper.swift +++ b/Packages/TableProCore/Sources/TableProSync/SyncRecordMapper.swift @@ -3,6 +3,7 @@ import Foundation import os import TableProModels +import TableProSyncTransport public enum SyncRecordMapper { private static let logger = Logger(subsystem: "com.TablePro", category: "SyncRecordMapper") @@ -23,39 +24,50 @@ public enum SyncRecordMapper { return CKRecord.ID(recordName: recordName, zoneID: zone) } + public static func parse(recordName: String) -> (type: SyncRecordType, id: String)? { + let prefixes: [(SyncRecordType, String)] = [ + (.connection, "Connection_"), + (.group, "Group_"), + (.tag, "Tag_") + ] + for (type, prefix) in prefixes where recordName.hasPrefix(prefix) { + return (type, String(recordName.dropFirst(prefix.count))) + } + return nil + } + // MARK: - Connection -> CKRecord public static func toRecord(_ connection: DatabaseConnection, zoneID: CKRecordZone.ID) -> CKRecord { let id = recordID(type: .connection, id: connection.id.uuidString, in: zoneID) let record = CKRecord(recordType: SyncRecordType.connection.rawValue, recordID: id) - record["connectionId"] = connection.id.uuidString as CKRecordValue - record["name"] = connection.name as CKRecordValue - record["host"] = connection.host as CKRecordValue - record["port"] = Int64(connection.port) as CKRecordValue - record["database"] = connection.database as CKRecordValue - record["username"] = connection.username as CKRecordValue - record["type"] = connection.type.rawValue as CKRecordValue - record["sortOrder"] = Int64(connection.sortOrder) as CKRecordValue - record["isReadOnly"] = Int64(connection.isReadOnly ? 1 : 0) as CKRecordValue - record["safeModeLevel"] = connection.safeModeLevel.rawValue as CKRecordValue - record["sshEnabled"] = Int64(connection.sshEnabled ? 1 : 0) as CKRecordValue - record["sslEnabled"] = Int64(connection.sslEnabled ? 1 : 0) as CKRecordValue + record[.connectionId] = connection.id.uuidString as CKRecordValue + record[.name] = connection.name as CKRecordValue + record[.host] = connection.host as CKRecordValue + record[.port] = Int64(connection.port) as CKRecordValue + record[.database] = connection.database as CKRecordValue + record[.username] = connection.username as CKRecordValue + record[.type] = connection.type.rawValue as CKRecordValue + record[.sortOrder] = Int64(connection.sortOrder) as CKRecordValue + record[.isReadOnly] = Int64(connection.isReadOnly ? 1 : 0) as CKRecordValue + record[.safeModeLevel] = connection.safeModeLevel.rawValue as CKRecordValue + record[.sshEnabled] = Int64(connection.sshEnabled ? 1 : 0) as CKRecordValue + record[.sslEnabled] = Int64(connection.sslEnabled ? 1 : 0) as CKRecordValue if let colorTag = connection.colorTag { - record["color"] = colorTag as CKRecordValue - record["colorTag"] = colorTag as CKRecordValue + record[.colorTag] = colorTag as CKRecordValue } if let groupId = connection.groupId { - record["groupId"] = groupId.uuidString as CKRecordValue + record[.groupId] = groupId.uuidString as CKRecordValue } if !connection.tagIds.isEmpty { let tagIdStrings = connection.tagIds.map { $0.uuidString } - record["tagIds"] = tagIdStrings as CKRecordValue - record["tagId"] = tagIdStrings[0] as CKRecordValue + record[.tagIds] = tagIdStrings as CKRecordValue + record[.tagId] = tagIdStrings[0] as CKRecordValue } if let queryTimeout = connection.queryTimeoutSeconds { - record["queryTimeoutSeconds"] = Int64(queryTimeout) as CKRecordValue + record[.queryTimeoutSeconds] = Int64(queryTimeout) as CKRecordValue } if let sshConfig = connection.sshConfiguration { @@ -63,7 +75,7 @@ public enum SyncRecordMapper { var syncSafe = sshConfig syncSafe.privateKeyData = nil let data = try encoder.encode(syncSafe) - record["sshConfigJson"] = data as CKRecordValue + record[.sshConfigJson] = data as CKRecordValue } catch { logger.warning("Failed to encode SSH config for sync: \(error.localizedDescription)") } @@ -72,7 +84,7 @@ public enum SyncRecordMapper { if let sslConfig = connection.sslConfiguration { do { let data = try encoder.encode(sslConfig) - record["sslConfigJson"] = data as CKRecordValue + record[.sslConfigJson] = data as CKRecordValue } catch { logger.warning("Failed to encode SSL config for sync: \(error.localizedDescription)") } @@ -81,14 +93,14 @@ public enum SyncRecordMapper { if !connection.additionalFields.isEmpty { do { let data = try encoder.encode(connection.additionalFields) - record["additionalFieldsJson"] = data as CKRecordValue + record[.additionalFieldsJson] = data as CKRecordValue } catch { logger.warning("Failed to encode additional fields for sync: \(error.localizedDescription)") } } - record["modifiedAtLocal"] = Date() as CKRecordValue - record["schemaVersion"] = schemaVersion as CKRecordValue + record[.modifiedAtLocal] = Date() as CKRecordValue + record[.schemaVersion] = schemaVersion as CKRecordValue return record } @@ -96,59 +108,59 @@ public enum SyncRecordMapper { // MARK: - CKRecord -> Connection public static func toConnection(_ record: CKRecord) -> DatabaseConnection? { - guard let idString = record["connectionId"] as? String, + guard let idString = record[.connectionId] as? String, let id = UUID(uuidString: idString), - let name = record["name"] as? String, - let typeRaw = record["type"] as? String + let name = record[.name] as? String, + let typeRaw = record[.type] as? String else { logger.warning("Failed to decode connection from CKRecord: missing required fields") return nil } - let host = record["host"] as? String ?? "127.0.0.1" - let port = (record["port"] as? Int64).map { Int($0) } ?? 3306 - let database = record["database"] as? String ?? "" - let username = record["username"] as? String ?? "" - let colorTag = record["color"] as? String ?? record["colorTag"] as? String - let groupId = (record["groupId"] as? String).flatMap { UUID(uuidString: $0) } + let host = record[.host] as? String ?? "127.0.0.1" + let port = (record[.port] as? Int64).map { Int($0) } ?? 3306 + let database = record[.database] as? String ?? "" + let username = record[.username] as? String ?? "" + let colorTag = record[.colorTag] as? String + let groupId = (record[.groupId] as? String).flatMap { UUID(uuidString: $0) } let tagIds: [UUID] - if let rawIds = record["tagIds"] as? [String], !rawIds.isEmpty { + if let rawIds = record[.tagIds] as? [String], !rawIds.isEmpty { tagIds = rawIds.compactMap { UUID(uuidString: $0) } - } else if let single = (record["tagId"] as? String).flatMap({ UUID(uuidString: $0) }) { + } else if let single = (record[.tagId] as? String).flatMap({ UUID(uuidString: $0) }) { tagIds = [single] } else { tagIds = [] } - let sortOrder = (record["sortOrder"] as? Int64).map { Int($0) } ?? 0 - let isReadOnly = (record["isReadOnly"] as? Int64 ?? 0) != 0 - let safeModeLevel = safeModeLevel(fromWire: record["safeModeLevel"] as? String, isReadOnly: isReadOnly) - let queryTimeout = (record["queryTimeoutSeconds"] as? Int64).map { Int($0) } + let sortOrder = (record[.sortOrder] as? Int64).map { Int($0) } ?? 0 + let isReadOnly = (record[.isReadOnly] as? Int64 ?? 0) != 0 + let safeModeLevel = safeModeLevel(fromWire: record[.safeModeLevel] as? String, isReadOnly: isReadOnly) + let queryTimeout = (record[.queryTimeoutSeconds] as? Int64).map { Int($0) } var sshConfig: SSHConfiguration? - if let sshData = record["sshConfigJson"] as? Data { + if let sshData = record[.sshConfigJson] as? Data { sshConfig = try? decoder.decode(SSHConfiguration.self, from: sshData) } // macOS stores SSH enabled inside sshConfigJson ("enabled" field), // not as a top-level CKRecord field. Fall back to checking the JSON. let sshEnabled: Bool - if let explicit = record["sshEnabled"] as? Int64 { + if let explicit = record[.sshEnabled] as? Int64 { sshEnabled = explicit != 0 - } else if let sshData = record["sshConfigJson"] as? Data, + } else if let sshData = record[.sshConfigJson] as? Data, let json = try? JSONSerialization.jsonObject(with: sshData) as? [String: Any] { sshEnabled = json["enabled"] as? Bool ?? (sshConfig != nil && !(sshConfig?.host.isEmpty ?? true)) } else { sshEnabled = false } - let sslEnabled = (record["sslEnabled"] as? Int64 ?? 0) != 0 + let sslEnabled = (record[.sslEnabled] as? Int64 ?? 0) != 0 var sslConfig: SSLConfiguration? - if let sslData = record["sslConfigJson"] as? Data { + if let sslData = record[.sslConfigJson] as? Data { sslConfig = try? decoder.decode(SSLConfiguration.self, from: sslData) } var additionalFields: [String: String] = [:] - if let fieldsData = record["additionalFieldsJson"] as? Data { + if let fieldsData = record[.additionalFieldsJson] as? Data { additionalFields = (try? decoder.decode([String: String].self, from: fieldsData)) ?? [:] } @@ -188,75 +200,59 @@ public enum SyncRecordMapper { // MARK: - Update Existing CKRecord (preserves macOS-only fields) public static func updateRecord(_ record: CKRecord, with connection: DatabaseConnection) { - record["connectionId"] = connection.id.uuidString as CKRecordValue - record["name"] = connection.name as CKRecordValue - record["host"] = connection.host as CKRecordValue - record["port"] = Int64(connection.port) as CKRecordValue - record["database"] = connection.database as CKRecordValue - record["username"] = connection.username as CKRecordValue - record["type"] = connection.type.rawValue as CKRecordValue - record["sortOrder"] = Int64(connection.sortOrder) as CKRecordValue - record["isReadOnly"] = Int64(connection.isReadOnly ? 1 : 0) as CKRecordValue - record["safeModeLevel"] = connection.safeModeLevel.rawValue as CKRecordValue - record["sshEnabled"] = Int64(connection.sshEnabled ? 1 : 0) as CKRecordValue - record["sslEnabled"] = Int64(connection.sslEnabled ? 1 : 0) as CKRecordValue - - if let colorTag = connection.colorTag { - record["color"] = colorTag as CKRecordValue - record["colorTag"] = colorTag as CKRecordValue - } else { - record["color"] = nil - record["colorTag"] = nil - } - - if let groupId = connection.groupId { - record["groupId"] = groupId.uuidString as CKRecordValue - } else { - record["groupId"] = nil - } + record[.connectionId] = connection.id.uuidString as CKRecordValue + record[.name] = connection.name as CKRecordValue + record[.host] = connection.host as CKRecordValue + record[.port] = Int64(connection.port) as CKRecordValue + record[.database] = connection.database as CKRecordValue + record[.username] = connection.username as CKRecordValue + record[.type] = connection.type.rawValue as CKRecordValue + record[.sortOrder] = Int64(connection.sortOrder) as CKRecordValue + record[.isReadOnly] = Int64(connection.isReadOnly ? 1 : 0) as CKRecordValue + record[.safeModeLevel] = connection.safeModeLevel.rawValue as CKRecordValue + record[.sshEnabled] = Int64(connection.sshEnabled ? 1 : 0) as CKRecordValue + record[.sslEnabled] = Int64(connection.sslEnabled ? 1 : 0) as CKRecordValue + record[.colorTag] = connection.colorTag as CKRecordValue? + record[.groupId] = connection.groupId?.uuidString as CKRecordValue? if !connection.tagIds.isEmpty { let tagIdStrings = connection.tagIds.map { $0.uuidString } - record["tagIds"] = tagIdStrings as CKRecordValue - record["tagId"] = tagIdStrings[0] as CKRecordValue + record[.tagIds] = tagIdStrings as CKRecordValue + record[.tagId] = tagIdStrings[0] as CKRecordValue } else { - record["tagIds"] = nil - record["tagId"] = nil + record[.tagIds] = nil + record[.tagId] = nil } - if let queryTimeout = connection.queryTimeoutSeconds { - record["queryTimeoutSeconds"] = Int64(queryTimeout) as CKRecordValue - } else { - record["queryTimeoutSeconds"] = nil - } + record[.queryTimeoutSeconds] = connection.queryTimeoutSeconds.map { Int64($0) } as CKRecordValue? if let sshConfig = connection.sshConfiguration { var syncSafe = sshConfig syncSafe.privateKeyData = nil if let data = try? encoder.encode(syncSafe) { - record["sshConfigJson"] = data as CKRecordValue + record[.sshConfigJson] = data as CKRecordValue } } else { - record["sshConfigJson"] = nil + record[.sshConfigJson] = nil } if let sslConfig = connection.sslConfiguration { if let data = try? encoder.encode(sslConfig) { - record["sslConfigJson"] = data as CKRecordValue + record[.sslConfigJson] = data as CKRecordValue } } else { - record["sslConfigJson"] = nil + record[.sslConfigJson] = nil } if !connection.additionalFields.isEmpty { if let data = try? encoder.encode(connection.additionalFields) { - record["additionalFieldsJson"] = data as CKRecordValue + record[.additionalFieldsJson] = data as CKRecordValue } } else { - record["additionalFieldsJson"] = nil + record[.additionalFieldsJson] = nil } - record["modifiedAtLocal"] = Date() as CKRecordValue + record[.modifiedAtLocal] = Date() as CKRecordValue } // MARK: - Group -> CKRecord diff --git a/Packages/TableProCore/Sources/TableProSync/CloudKitSyncEngine.swift b/Packages/TableProCore/Sources/TableProSyncTransport/CloudKitSyncEngine.swift similarity index 68% rename from Packages/TableProCore/Sources/TableProSync/CloudKitSyncEngine.swift rename to Packages/TableProCore/Sources/TableProSyncTransport/CloudKitSyncEngine.swift index b6394a47a..eb83aabb5 100644 --- a/Packages/TableProCore/Sources/TableProSync/CloudKitSyncEngine.swift +++ b/Packages/TableProCore/Sources/TableProSyncTransport/CloudKitSyncEngine.swift @@ -52,11 +52,13 @@ public actor CloudKitSyncEngine { // MARK: - Push - public func push(records: [CKRecord], deletions: [CKRecord.ID]) async throws { - guard !records.isEmpty || !deletions.isEmpty else { return } + @discardableResult + public func push(records: [CKRecord], deletions: [CKRecord.ID]) async throws -> PushOutcome { + guard !records.isEmpty || !deletions.isEmpty else { return PushOutcome() } var remainingSaves = records[...] var remainingDeletions = deletions[...] + var outcome = PushOutcome() while !remainingSaves.isEmpty || !remainingDeletions.isEmpty { let savesCount = min(remainingSaves.count, Self.maxBatchSize) @@ -67,37 +69,62 @@ public actor CloudKitSyncEngine { let batchDeletions = Array(remainingDeletions.prefix(deletionsCount)) remainingDeletions = remainingDeletions.dropFirst(deletionsCount) - try await pushBatch(records: batchSaves, deletions: batchDeletions) + outcome.merge(try await pushBatch(records: batchSaves, deletions: batchDeletions)) } - Self.logger.info("Pushed \(records.count) records, \(deletions.count) deletions") + let saved = outcome.savedRecords.count + let deleted = outcome.deletedRecordIDs.count + let failed = outcome.failures.count + Self.logger.info("Pushed \(saved) records, \(deleted) deletions, \(failed) rejected") + + for (recordID, failure) in outcome.failures { + Self.logger.error("CloudKit rejected \(recordID.recordName): \(failure.message)") + } + + return outcome } - private func pushBatch(records: [CKRecord], deletions: [CKRecord.ID]) async throws { + private func pushBatch(records: [CKRecord], deletions: [CKRecord.ID]) async throws -> PushOutcome { try await withRetry { let operation = CKModifyRecordsOperation( recordsToSave: records, recordIDsToDelete: deletions ) - // .changedKeys overwrites only the fields we set, safe for partial updates operation.savePolicy = .changedKeys - operation.isAtomic = true + operation.isAtomic = false return try await withCheckedThrowingContinuation { continuation in + var outcome = PushOutcome() + operation.perRecordSaveBlock = { recordID, result in - if case .failure(let error) = result { - Self.logger.error( - "Failed to save record \(recordID.recordName): \(error.localizedDescription)" - ) + switch result { + case .success(let record): + outcome.recordSave(record) + case .failure(let error): + outcome.recordFailure(SyncItemFailure(error: error), for: recordID) + } + } + + operation.perRecordDeleteBlock = { recordID, result in + switch result { + case .success: + outcome.recordDeletion(recordID) + case .failure(let error): + outcome.recordFailure(SyncItemFailure(error: error), for: recordID) } } operation.modifyRecordsResultBlock = { result in switch result { case .success: - continuation.resume() + continuation.resume(returning: outcome) case .failure(let error): - continuation.resume(throwing: error) + guard let ckError = error as? CKError, ckError.code == .partialFailure else { + continuation.resume(throwing: error) + return + } + outcome.absorbPartialErrors(from: ckError) + continuation.resume(returning: outcome) } } @@ -109,12 +136,35 @@ public actor CloudKitSyncEngine { // MARK: - Pull public func pull(since token: CKServerChangeToken?) async throws -> PullResult { - try await withRetry { - try await performPull(since: token) + var changedRecords: [CKRecord] = [] + var deletedRecordIDs: [CKRecord.ID] = [] + var cursor = token + + while true { + let page = try await withRetry { [cursor] in + try await performPull(since: cursor) + } + + changedRecords.append(contentsOf: page.result.changedRecords) + deletedRecordIDs.append(contentsOf: page.result.deletedRecordIDs) + cursor = page.result.newToken ?? cursor + + guard page.moreComing, page.result.newToken != nil else { + return PullResult( + changedRecords: changedRecords, + deletedRecordIDs: deletedRecordIDs, + newToken: cursor + ) + } } } - private func performPull(since token: CKServerChangeToken?) async throws -> PullResult { + private struct PullPage { + let result: PullResult + let moreComing: Bool + } + + private func performPull(since token: CKServerChangeToken?) async throws -> PullPage { let configuration = CKFetchRecordZoneChangesOperation.ZoneConfiguration() configuration.previousServerChangeToken = token @@ -126,6 +176,7 @@ public actor CloudKitSyncEngine { var changedRecords: [CKRecord] = [] var deletedRecordIDs: [CKRecord.ID] = [] var newToken: CKServerChangeToken? + var moreComing = false return try await withCheckedThrowingContinuation { continuation in operation.recordWasChangedBlock = { _, result in @@ -144,11 +195,12 @@ public actor CloudKitSyncEngine { operation.recordZoneFetchResultBlock = { _, result in switch result { - case .success(let (serverToken, _, _)): + case .success(let (serverToken, _, hasMore)): newToken = serverToken + moreComing = hasMore case .failure(let error): Self.logger.warning("Zone fetch result error: \(error.localizedDescription)") - // Zone-level failure with records collected so far is acceptable — + // Zone-level failure with records collected so far is acceptable: // newToken stays nil, forcing a full re-fetch on next sync cycle. } } @@ -156,10 +208,13 @@ public actor CloudKitSyncEngine { operation.fetchRecordZoneChangesResultBlock = { result in switch result { case .success: - continuation.resume(returning: PullResult( - changedRecords: changedRecords, - deletedRecordIDs: deletedRecordIDs, - newToken: newToken + continuation.resume(returning: PullPage( + result: PullResult( + changedRecords: changedRecords, + deletedRecordIDs: deletedRecordIDs, + newToken: newToken + ), + moreComing: moreComing )) case .failure(let error): // Map CKError.changeTokenExpired to SyncError.tokenExpired diff --git a/Packages/TableProCore/Sources/TableProSyncTransport/ConnectionSyncSchema.swift b/Packages/TableProCore/Sources/TableProSyncTransport/ConnectionSyncSchema.swift new file mode 100644 index 000000000..ac46a787d --- /dev/null +++ b/Packages/TableProCore/Sources/TableProSyncTransport/ConnectionSyncSchema.swift @@ -0,0 +1,89 @@ +import CloudKit +import Foundation + +public enum ProductionSchemaState: Sendable { + case verified + case unverified +} + +public enum ConnectionSyncField: String, CaseIterable, Sendable { + case connectionId + case name + case host + case port + case database + case username + case type + case color + case colorTag + case safeModeLevel + case sortOrder + case groupId + case tagId + case tagIds + case isReadOnly + case sshEnabled + case sslEnabled + case queryTimeoutSeconds + case sshConfigJson + case sslConfigJson + case additionalFieldsJson + case aiPolicy + case aiRules + case aiAlwaysAllowedTools + case redisDatabase + case startupCommands + case sshProfileId + case isFavorite + case modifiedAtLocal + case schemaVersion + + public var key: String { rawValue } + + public var productionSchemaState: ProductionSchemaState { + switch self { + case .connectionId, .name, .host, .port, .database, .username, .type, + .color, .colorTag, .safeModeLevel, .sortOrder, .groupId, .tagId, + .isReadOnly, .sshEnabled, .sslEnabled, + .sshConfigJson, .sslConfigJson, .additionalFieldsJson, + .aiPolicy, .redisDatabase, .startupCommands, .sshProfileId, + .modifiedAtLocal, .schemaVersion: + return .verified + default: + return .unverified + } + } + + public var isWritable: Bool { productionSchemaState == .verified } + + public static var writableKeys: Set { + Set(allCases.filter(\.isWritable).map(\.key)) + } + + public static var declaredKeys: Set { + Set(allCases.map(\.key)) + } +} + +public extension CKRecord { + subscript(field: ConnectionSyncField) -> Any? { + get { self[field.key] } + set { + guard field.isWritable else { return } + let replacement = newValue as? any CKRecordValueProtocol + guard !Self.isEqualRecordValue(self[field.key], replacement) else { return } + self[field.key] = replacement + } + } + + static func isEqualRecordValue(_ lhs: Any?, _ rhs: Any?) -> Bool { + switch (lhs, rhs) { + case (nil, nil): + return true + case (let lhs as NSObject, let rhs as NSObject): + return lhs.isEqual(rhs) + default: + return false + } + } +} diff --git a/Packages/TableProCore/Sources/TableProSyncTransport/PushOutcome.swift b/Packages/TableProCore/Sources/TableProSyncTransport/PushOutcome.swift new file mode 100644 index 000000000..90ef9b156 --- /dev/null +++ b/Packages/TableProCore/Sources/TableProSyncTransport/PushOutcome.swift @@ -0,0 +1,85 @@ +import CloudKit +import Foundation + +public struct SyncItemFailure: Sendable { + public let code: CKError.Code + public let serverRecord: CKRecord? + public let clientRecord: CKRecord? + public let message: String + + public init(code: CKError.Code, serverRecord: CKRecord?, clientRecord: CKRecord?, message: String) { + self.code = code + self.serverRecord = serverRecord + self.clientRecord = clientRecord + self.message = message + } + + public init(error: Error) { + let ckError = error as? CKError + self.code = ckError?.code ?? .internalError + self.serverRecord = ckError?.serverRecord + self.clientRecord = ckError?.clientRecord + self.message = error.localizedDescription + } + + public var isConflict: Bool { code == .serverRecordChanged } +} + +public struct PushOutcome: Sendable { + public private(set) var savedRecords: [CKRecord.ID: CKRecord] + public private(set) var deletedRecordIDs: Set + public private(set) var failures: [CKRecord.ID: SyncItemFailure] + + public init( + savedRecords: [CKRecord.ID: CKRecord] = [:], + deletedRecordIDs: Set = [], + failures: [CKRecord.ID: SyncItemFailure] = [:] + ) { + self.savedRecords = savedRecords + self.deletedRecordIDs = deletedRecordIDs + self.failures = failures + } + + public var hasFailures: Bool { !failures.isEmpty } + + public var conflicts: [CKRecord.ID: SyncItemFailure] { + failures.filter(\.value.isConflict) + } + + public func didSave(_ recordID: CKRecord.ID) -> Bool { + savedRecords[recordID] != nil + } + + public func didDelete(_ recordID: CKRecord.ID) -> Bool { + deletedRecordIDs.contains(recordID) + } + + public mutating func recordSave(_ record: CKRecord) { + savedRecords[record.recordID] = record + failures[record.recordID] = nil + } + + public mutating func recordDeletion(_ recordID: CKRecord.ID) { + deletedRecordIDs.insert(recordID) + failures[recordID] = nil + } + + public mutating func recordFailure(_ failure: SyncItemFailure, for recordID: CKRecord.ID) { + guard savedRecords[recordID] == nil, !deletedRecordIDs.contains(recordID) else { return } + failures[recordID] = failure + } + + public mutating func merge(_ other: PushOutcome) { + savedRecords.merge(other.savedRecords) { _, new in new } + deletedRecordIDs.formUnion(other.deletedRecordIDs) + failures.merge(other.failures) { _, new in new } + } + + public mutating func absorbPartialErrors(from error: CKError) { + guard let partial = error.partialErrorsByItemID else { return } + for (itemID, itemError) in partial { + guard let recordID = itemID as? CKRecord.ID else { continue } + recordFailure(SyncItemFailure(error: itemError), for: recordID) + } + } +} diff --git a/Packages/TableProCore/Sources/TableProSync/SyncError.swift b/Packages/TableProCore/Sources/TableProSyncTransport/SyncError.swift similarity index 100% rename from Packages/TableProCore/Sources/TableProSync/SyncError.swift rename to Packages/TableProCore/Sources/TableProSyncTransport/SyncError.swift diff --git a/Packages/TableProCore/Sources/TableProSync/SyncMetadataStorage.swift b/Packages/TableProCore/Sources/TableProSyncTransport/SyncMetadataStorage.swift similarity index 100% rename from Packages/TableProCore/Sources/TableProSync/SyncMetadataStorage.swift rename to Packages/TableProCore/Sources/TableProSyncTransport/SyncMetadataStorage.swift diff --git a/Packages/TableProCore/Sources/TableProSyncTransport/SyncRecordCache.swift b/Packages/TableProCore/Sources/TableProSyncTransport/SyncRecordCache.swift new file mode 100644 index 000000000..8e394b4a4 --- /dev/null +++ b/Packages/TableProCore/Sources/TableProSyncTransport/SyncRecordCache.swift @@ -0,0 +1,69 @@ +import CloudKit +import Foundation +import os + +public final class SyncRecordCache { + private static let logger = Logger(subsystem: "com.TablePro", category: "SyncRecordCache") + + private let defaults: UserDefaults + private let storageKey: String + + public init(defaults: UserDefaults = .standard, storageKey: String = "com.TablePro.sync.recordCache") { + self.defaults = defaults + self.storageKey = storageKey + } + + public func record(for recordID: CKRecord.ID) -> CKRecord? { + guard let data = archives()[recordID.recordName] else { return nil } + return unarchive(data) + } + + public func store(_ records: [CKRecord]) { + guard !records.isEmpty else { return } + var current = archives() + for record in records { + guard let data = archive(record) else { continue } + current[record.recordID.recordName] = data + } + defaults.set(current, forKey: storageKey) + } + + public func remove(_ recordIDs: [CKRecord.ID]) { + guard !recordIDs.isEmpty else { return } + var current = archives() + for recordID in recordIDs { + current[recordID.recordName] = nil + } + if current.isEmpty { + defaults.removeObject(forKey: storageKey) + } else { + defaults.set(current, forKey: storageKey) + } + } + + public func removeAll() { + defaults.removeObject(forKey: storageKey) + } + + private func archives() -> [String: Data] { + defaults.dictionary(forKey: storageKey) as? [String: Data] ?? [:] + } + + private func archive(_ record: CKRecord) -> Data? { + do { + return try NSKeyedArchiver.archivedData(withRootObject: record, requiringSecureCoding: true) + } catch { + Self.logger.error("Failed to archive record \(record.recordID.recordName): \(error.localizedDescription)") + return nil + } + } + + private func unarchive(_ data: Data) -> CKRecord? { + do { + return try NSKeyedUnarchiver.unarchivedObject(ofClass: CKRecord.self, from: data) + } catch { + Self.logger.error("Failed to unarchive a cached record: \(error.localizedDescription)") + return nil + } + } +} diff --git a/Packages/TableProCore/Sources/TableProSync/SyncRecordType.swift b/Packages/TableProCore/Sources/TableProSyncTransport/SyncRecordType.swift similarity index 100% rename from Packages/TableProCore/Sources/TableProSync/SyncRecordType.swift rename to Packages/TableProCore/Sources/TableProSyncTransport/SyncRecordType.swift diff --git a/Packages/TableProCore/Sources/TableProSyncTransport/SyncStatus.swift b/Packages/TableProCore/Sources/TableProSyncTransport/SyncStatus.swift new file mode 100644 index 000000000..c29d69493 --- /dev/null +++ b/Packages/TableProCore/Sources/TableProSyncTransport/SyncStatus.swift @@ -0,0 +1,7 @@ +import Foundation + +public enum SyncStatus: Equatable, Sendable { + case idle + case syncing + case error(String) +} diff --git a/Packages/TableProCore/Tests/TableProSyncTests/ConnectionSyncSchemaTests.swift b/Packages/TableProCore/Tests/TableProSyncTests/ConnectionSyncSchemaTests.swift new file mode 100644 index 000000000..3f68e20bb --- /dev/null +++ b/Packages/TableProCore/Tests/TableProSyncTests/ConnectionSyncSchemaTests.swift @@ -0,0 +1,129 @@ +import CloudKit +import Foundation +import Testing + +@testable import TableProModels +@testable import TableProSync +@testable import TableProSyncTransport + +@Suite("Connection sync schema") +struct ConnectionSyncSchemaTests { + private let zoneID = CKRecordZone.ID(zoneName: "TestZone", ownerName: CKCurrentUserDefaultName) + + private func makeFullyPopulatedConnection() -> DatabaseConnection { + DatabaseConnection( + id: UUID(), + name: "Production", + type: DatabaseType(rawValue: "PostgreSQL"), + host: "db.example.com", + port: 5432, + username: "admin", + database: "app", + colorTag: "#FF0000", + isReadOnly: true, + safeModeLevel: .readOnly, + queryTimeoutSeconds: 30, + additionalFields: ["schema": "public"], + sshEnabled: true, + sshConfiguration: SSHConfiguration(host: "bastion.example.com"), + sslEnabled: true, + sslConfiguration: SSLConfiguration(), + groupId: UUID(), + tagIds: [UUID(), UUID()], + sortOrder: 3 + ) + } + + @Test("an unverified field is never written to a record") + func unverifiedFieldsAreNeverWritten() { + let record = SyncRecordMapper.toRecord(makeFullyPopulatedConnection(), zoneID: zoneID) + let written = Set(record.allKeys()) + let unverified = Set(ConnectionSyncField.allCases.filter { !$0.isWritable }.map(\.key)) + + #expect(written.isDisjoint(with: unverified)) + } + + @Test("updateRecord never writes an unverified field either") + func updateRecordSkipsUnverifiedFields() { + let connection = makeFullyPopulatedConnection() + let recordID = SyncRecordMapper.recordID(type: .connection, id: connection.id.uuidString, in: zoneID) + let record = CKRecord(recordType: SyncRecordType.connection.rawValue, recordID: recordID) + + SyncRecordMapper.updateRecord(record, with: connection) + + let written = Set(record.allKeys()) + let unverified = Set(ConnectionSyncField.allCases.filter { !$0.isWritable }.map(\.key)) + + #expect(written.isDisjoint(with: unverified)) + } + + @Test("every written key is declared in the schema") + func everyWrittenKeyIsDeclared() { + let record = SyncRecordMapper.toRecord(makeFullyPopulatedConnection(), zoneID: zoneID) + + #expect(Set(record.allKeys()).isSubset(of: ConnectionSyncField.declaredKeys)) + } + + @Test("isFavorite stays out of the wire until the production schema is verified") + func isFavoriteIsNotWritten() { + #expect(ConnectionSyncField.isFavorite.isWritable == false) + } + + @Test("a colour tag is written to colorTag and never to the macOS color field") + func colourTagDoesNotCollideWithMacColor() { + var connection = makeFullyPopulatedConnection() + connection.colorTag = "#00FF00" + + let record = SyncRecordMapper.toRecord(connection, zoneID: zoneID) + + #expect(record[ConnectionSyncField.colorTag] as? String == "#00FF00") + #expect(record[ConnectionSyncField.color] == nil) + } + + @Test("a macOS colour name is not adopted as an iOS colour tag") + func macColorNameIsNotDecodedAsColourTag() { + let connection = makeFullyPopulatedConnection() + let recordID = SyncRecordMapper.recordID(type: .connection, id: connection.id.uuidString, in: zoneID) + let record = CKRecord(recordType: SyncRecordType.connection.rawValue, recordID: recordID) + record[ConnectionSyncField.connectionId] = connection.id.uuidString as CKRecordValue + record[ConnectionSyncField.name] = "From Mac" as CKRecordValue + record[ConnectionSyncField.type] = "PostgreSQL" as CKRecordValue + record[ConnectionSyncField.color] = "Blue" as CKRecordValue + + let decoded = SyncRecordMapper.toConnection(record) + + #expect(decoded?.colorTag == nil) + } + + @Test("a fully populated connection round-trips through the wire") + func roundTripPreservesVerifiedFields() { + let connection = makeFullyPopulatedConnection() + let record = SyncRecordMapper.toRecord(connection, zoneID: zoneID) + + let decoded = SyncRecordMapper.toConnection(record) + + #expect(decoded?.id == connection.id) + #expect(decoded?.name == connection.name) + #expect(decoded?.host == connection.host) + #expect(decoded?.port == connection.port) + #expect(decoded?.database == connection.database) + #expect(decoded?.username == connection.username) + #expect(decoded?.sortOrder == connection.sortOrder) + #expect(decoded?.isReadOnly == connection.isReadOnly) + #expect(decoded?.safeModeLevel == connection.safeModeLevel) + #expect(decoded?.groupId == connection.groupId) + #expect(decoded?.colorTag == connection.colorTag) + } + + @Test("A gated field does not survive the round trip") + func gatedFieldsDoNotRoundTrip() { + let connection = makeFullyPopulatedConnection() + let record = SyncRecordMapper.toRecord(connection, zoneID: zoneID) + + let decoded = SyncRecordMapper.toConnection(record) + + #expect(connection.queryTimeoutSeconds != nil) + #expect(ConnectionSyncField.queryTimeoutSeconds.isWritable == false) + #expect(decoded?.queryTimeoutSeconds == nil) + } +} diff --git a/Packages/TableProCore/Tests/TableProSyncTests/FieldLevelMergeTests.swift b/Packages/TableProCore/Tests/TableProSyncTests/FieldLevelMergeTests.swift new file mode 100644 index 000000000..0cfd22cf4 --- /dev/null +++ b/Packages/TableProCore/Tests/TableProSyncTests/FieldLevelMergeTests.swift @@ -0,0 +1,158 @@ +import CloudKit +import Foundation +import Testing + +@testable import TableProModels +@testable import TableProSync +@testable import TableProSyncTransport + +@Suite("Field level merge") +struct FieldLevelMergeTests { + private let zoneID = CKRecordZone.ID(zoneName: "TestZone", ownerName: CKCurrentUserDefaultName) + + private func makeConnection(name: String = "Production", port: Int = 5432) -> DatabaseConnection { + DatabaseConnection( + id: UUID(), + name: name, + type: DatabaseType(rawValue: "PostgreSQL"), + host: "db.example.com", + port: port, + username: "admin", + database: "app", + sortOrder: 1 + ) + } + + private func roundTripped(_ record: CKRecord) throws -> CKRecord { + let data = try NSKeyedArchiver.archivedData(withRootObject: record, requiringSecureCoding: true) + return try #require(try NSKeyedUnarchiver.unarchivedObject(ofClass: CKRecord.self, from: data)) + } + + @Test("Updating a cached server record keeps fields this platform never writes") + func foreignFieldsSurviveAnUpdate() throws { + var connection = makeConnection() + let serverRecord = try roundTripped(SyncRecordMapper.toRecord(connection, zoneID: zoneID)) + serverRecord["aiPolicy"] = "askEachTime" as CKRecordValue + serverRecord["startupCommands"] = "SET search_path TO public" as CKRecordValue + + connection.name = "Renamed" + SyncRecordMapper.updateRecord(serverRecord, with: connection) + + #expect(serverRecord["aiPolicy"] as? String == "askEachTime") + #expect(serverRecord["startupCommands"] as? String == "SET search_path TO public") + #expect(serverRecord[ConnectionSyncField.name] as? String == "Renamed") + } + + @Test("An equal value is recognised so the field is left untouched") + func equalValuesAreRecognised() { + #expect(CKRecord.isEqualRecordValue("Production" as CKRecordValue, "Production" as CKRecordValue)) + #expect(CKRecord.isEqualRecordValue(Int64(5432) as CKRecordValue, Int64(5432) as CKRecordValue)) + #expect(CKRecord.isEqualRecordValue(nil, nil)) + #expect(CKRecord.isEqualRecordValue( + ["a", "b"] as CKRecordValue, + ["a", "b"] as CKRecordValue + )) + #expect(CKRecord.isEqualRecordValue( + Data([1, 2, 3]) as CKRecordValue, + Data([1, 2, 3]) as CKRecordValue + )) + } + + @Test("A differing value is recognised so the field is rewritten") + func differingValuesAreRecognised() { + #expect(CKRecord.isEqualRecordValue("Production" as CKRecordValue, "Staging" as CKRecordValue) == false) + #expect(CKRecord.isEqualRecordValue(Int64(5432) as CKRecordValue, Int64(6543) as CKRecordValue) == false) + #expect(CKRecord.isEqualRecordValue(nil, "Production" as CKRecordValue) == false) + #expect(CKRecord.isEqualRecordValue("Production" as CKRecordValue, nil) == false) + #expect(CKRecord.isEqualRecordValue( + ["a", "b"] as CKRecordValue, + ["a"] as CKRecordValue + ) == false) + #expect(CKRecord.isEqualRecordValue( + Data([1, 2, 3]) as CKRecordValue, + Data([1, 2]) as CKRecordValue + ) == false) + } + + @Test("Clearing a field that was already absent leaves it absent") + func clearingAnAbsentFieldIsANoOp() throws { + var connection = makeConnection() + connection.groupId = nil + let serverRecord = try roundTripped(SyncRecordMapper.toRecord(connection, zoneID: zoneID)) + + SyncRecordMapper.updateRecord(serverRecord, with: connection) + + #expect(serverRecord[ConnectionSyncField.groupId] == nil) + } + + @Test("Clearing a field that had a value removes it") + func clearingAPopulatedFieldRemovesIt() throws { + var connection = makeConnection() + connection.groupId = UUID() + let serverRecord = try roundTripped(SyncRecordMapper.toRecord(connection, zoneID: zoneID)) + + connection.groupId = nil + SyncRecordMapper.updateRecord(serverRecord, with: connection) + + #expect(serverRecord[ConnectionSyncField.groupId] == nil) + } +} + +@Suite("Sync record cache") +struct SyncRecordCacheTests { + private let zoneID = CKRecordZone.ID(zoneName: "TestZone", ownerName: CKCurrentUserDefaultName) + + private func makeCache() throws -> SyncRecordCache { + let defaults = try #require(UserDefaults(suiteName: "com.TablePro.tests.\(UUID().uuidString)")) + return SyncRecordCache(defaults: defaults, storageKey: "recordCache") + } + + private func makeRecord(_ name: String) -> CKRecord { + let record = CKRecord( + recordType: "Connection", + recordID: CKRecord.ID(recordName: name, zoneID: zoneID) + ) + record["name"] = "Production" as CKRecordValue + return record + } + + @Test("A stored record round-trips with its values") + func storedRecordRoundTrips() throws { + let cache = try makeCache() + let record = makeRecord("Connection_A") + + cache.store([record]) + + #expect(cache.record(for: record.recordID)?["name"] as? String == "Production") + } + + @Test("Each read returns an independent copy so a failed push cannot poison the cache") + func readsAreIndependent() throws { + let cache = try makeCache() + let record = makeRecord("Connection_A") + cache.store([record]) + + let first = try #require(cache.record(for: record.recordID)) + first["name"] = "Mutated" as CKRecordValue + + #expect(cache.record(for: record.recordID)?["name"] as? String == "Production") + } + + @Test("A removed record is gone") + func removedRecordIsGone() throws { + let cache = try makeCache() + let record = makeRecord("Connection_A") + cache.store([record]) + + cache.remove([record.recordID]) + + #expect(cache.record(for: record.recordID) == nil) + } + + @Test("An unknown record is absent") + func unknownRecordIsAbsent() throws { + let cache = try makeCache() + + #expect(cache.record(for: CKRecord.ID(recordName: "Connection_Z", zoneID: zoneID)) == nil) + } +} diff --git a/Packages/TableProCore/Tests/TableProSyncTests/ProductionSchemaParityTests.swift b/Packages/TableProCore/Tests/TableProSyncTests/ProductionSchemaParityTests.swift new file mode 100644 index 000000000..a3b5407b3 --- /dev/null +++ b/Packages/TableProCore/Tests/TableProSyncTests/ProductionSchemaParityTests.swift @@ -0,0 +1,95 @@ +import Foundation +import Testing + +@testable import TableProSyncTransport + +@Suite("Production CloudKit schema parity") +struct ProductionSchemaParityTests { + private static let repositoryRoot = URL(fileURLWithPath: #filePath) + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + .deletingLastPathComponent() + + private static let schemaURL = repositoryRoot + .appendingPathComponent("CloudKit") + .appendingPathComponent("production-schema.ckdb") + + struct MissingRecordType: Error, CustomStringConvertible { + let recordType: String + + var description: String { + """ + No "RECORD TYPE \(recordType)" block in CloudKit/production-schema.ckdb. \ + The snapshot is stale or malformed, so nothing can be verified against it. \ + Re-export it with scripts/export-cloudkit-schema.sh and commit the result. + """ + } + } + + private static func fields(ofRecordType recordType: String) throws -> Set { + let contents = try String(contentsOf: schemaURL, encoding: .utf8) + guard let block = contents + .components(separatedBy: "RECORD TYPE \(recordType) (") + .dropFirst() + .first? + .components(separatedBy: ");") + .first + else { + throw MissingRecordType(recordType: recordType) + } + + var names: Set = [] + for line in block.components(separatedBy: .newlines) { + let trimmed = line.trimmingCharacters(in: .whitespaces) + guard !trimmed.isEmpty, !trimmed.hasPrefix("\""), !trimmed.hasPrefix("GRANT") else { continue } + guard let name = trimmed.components(separatedBy: .whitespaces).first, !name.isEmpty else { continue } + names.insert(name) + } + return names + } + + @Test("The exported production schema is checked in") + func schemaSnapshotExists() { + #expect(FileManager.default.fileExists(atPath: Self.schemaURL.path)) + } + + @Test("The snapshot parses into a recognisable Connection record") + func snapshotParsesConnectionFields() throws { + let deployed = try Self.fields(ofRecordType: "Connection") + + #expect(deployed.contains("connectionId"), """ + Parsed the Connection block but found no connectionId field, so the parser no longer \ + understands the snapshot format and every parity check below it would pass vacuously. \ + Parsed \(deployed.count) field(s): \(deployed.sorted()). + """) + } + + @Test("Every field the app writes exists in the production schema") + func writableFieldsExistInProduction() throws { + let deployed = try Self.fields(ofRecordType: "Connection") + let missing = ConnectionSyncField.writableKeys.subtracting(deployed) + + #expect(missing.isEmpty, """ + Writable fields absent from the production schema: \(missing.sorted()). \ + CloudKit rejects any record carrying an undeclared field, so these would stop \ + Connection syncing entirely. Add each field in CloudKit Console, deploy Development \ + to Production, run scripts/export-cloudkit-schema.sh, and commit the refreshed snapshot. \ + Until then mark them unverified in ConnectionSyncSchema.swift. + """) + } + + @Test("A field marked unverified is genuinely absent from the production schema") + func unverifiedFieldsAreAbsentFromProduction() throws { + let deployed = try Self.fields(ofRecordType: "Connection") + let unverified = ConnectionSyncField.declaredKeys.subtracting(ConnectionSyncField.writableKeys) + let deployedButGated = unverified.intersection(deployed) + + #expect(deployedButGated.isEmpty, """ + Deployed fields still gated off: \(deployedButGated.sorted()). \ + These exist in Production, so the gate is costing you data the app could sync. \ + Mark them verified in ConnectionSyncSchema.swift. + """) + } +} diff --git a/Packages/TableProCore/Tests/TableProSyncTests/PushOutcomeTests.swift b/Packages/TableProCore/Tests/TableProSyncTests/PushOutcomeTests.swift new file mode 100644 index 000000000..a529e1dca --- /dev/null +++ b/Packages/TableProCore/Tests/TableProSyncTests/PushOutcomeTests.swift @@ -0,0 +1,124 @@ +import CloudKit +import Foundation +import Testing + +@testable import TableProSyncTransport + +@Suite("Push outcome accounting") +struct PushOutcomeTests { + private let zoneID = CKRecordZone.ID(zoneName: "TestZone", ownerName: CKCurrentUserDefaultName) + + private func makeRecord(_ name: String) -> CKRecord { + CKRecord(recordType: "Connection", recordID: CKRecord.ID(recordName: name, zoneID: zoneID)) + } + + private func recordID(_ name: String) -> CKRecord.ID { + CKRecord.ID(recordName: name, zoneID: zoneID) + } + + @Test("A saved record is reported saved") + func savedRecordIsReported() { + var outcome = PushOutcome() + let record = makeRecord("Connection_A") + + outcome.recordSave(record) + + #expect(outcome.didSave(record.recordID)) + #expect(outcome.hasFailures == false) + } + + @Test("One rejected record leaves the others reported as saved") + func oneRejectionDoesNotPoisonTheBatch() { + var outcome = PushOutcome() + let saved = makeRecord("Connection_A") + let rejected = recordID("Connection_B") + + outcome.recordSave(saved) + outcome.recordFailure( + SyncItemFailure(code: .invalidArguments, serverRecord: nil, clientRecord: nil, message: "unknown field"), + for: rejected + ) + + #expect(outcome.didSave(saved.recordID)) + #expect(outcome.didSave(rejected) == false) + #expect(outcome.failures.count == 1) + } + + @Test("Per-item errors from a partial failure are absorbed") + func partialErrorsAreAbsorbed() { + var outcome = PushOutcome() + let rejected = recordID("Connection_B") + let partial = CKError( + .partialFailure, + userInfo: [CKPartialErrorsByItemIDKey: [rejected: CKError(.invalidArguments)]] + ) + + outcome.absorbPartialErrors(from: partial) + + #expect(outcome.failures[rejected]?.code == .invalidArguments) + } + + @Test("A per-item conflict is separated from an outright rejection") + func conflictsAreSeparatedFromRejections() { + var outcome = PushOutcome() + let conflicted = recordID("Connection_A") + let rejected = recordID("Connection_B") + + outcome.recordFailure( + SyncItemFailure(code: .serverRecordChanged, serverRecord: nil, clientRecord: nil, message: "changed"), + for: conflicted + ) + outcome.recordFailure( + SyncItemFailure(code: .invalidArguments, serverRecord: nil, clientRecord: nil, message: "unknown field"), + for: rejected + ) + + #expect(outcome.conflicts.keys.contains(conflicted)) + #expect(outcome.conflicts.keys.contains(rejected) == false) + #expect(outcome.failures.count == 2) + } + + @Test("A record that saved is never also reported as failed") + func saveWinsOverAStaleFailure() { + var outcome = PushOutcome() + let record = makeRecord("Connection_A") + + outcome.recordSave(record) + outcome.recordFailure( + SyncItemFailure(code: .invalidArguments, serverRecord: nil, clientRecord: nil, message: "late"), + for: record.recordID + ) + + #expect(outcome.didSave(record.recordID)) + #expect(outcome.failures.isEmpty) + } + + @Test("A confirmed deletion is reported separately from a save") + func deletionsAreReported() { + var outcome = PushOutcome() + let deleted = recordID("Connection_C") + + outcome.recordDeletion(deleted) + + #expect(outcome.didDelete(deleted)) + #expect(outcome.didSave(deleted) == false) + } + + @Test("Merging batches keeps every saved record and every failure") + func mergeKeepsAllResults() { + var first = PushOutcome() + first.recordSave(makeRecord("Connection_A")) + + var second = PushOutcome() + second.recordSave(makeRecord("Connection_B")) + second.recordFailure( + SyncItemFailure(code: .invalidArguments, serverRecord: nil, clientRecord: nil, message: "unknown field"), + for: recordID("Connection_C") + ) + + first.merge(second) + + #expect(first.savedRecords.count == 2) + #expect(first.failures.count == 1) + } +} diff --git a/Packages/TableProCore/Tests/TableProSyncTests/SyncRecordMapperTests.swift b/Packages/TableProCore/Tests/TableProSyncTests/SyncRecordMapperTests.swift index 5ebc43993..13a59f560 100644 --- a/Packages/TableProCore/Tests/TableProSyncTests/SyncRecordMapperTests.swift +++ b/Packages/TableProCore/Tests/TableProSyncTests/SyncRecordMapperTests.swift @@ -4,6 +4,7 @@ import Testing @testable import TableProModels @testable import TableProSync +@testable import TableProSyncTransport @Suite("SyncRecordMapper safe mode") struct SyncRecordMapperTests { diff --git a/TablePro.xcodeproj/project.pbxproj b/TablePro.xcodeproj/project.pbxproj index 262341b00..f367cd81c 100644 --- a/TablePro.xcodeproj/project.pbxproj +++ b/TablePro.xcodeproj/project.pbxproj @@ -18,6 +18,7 @@ 5A32BBFB2F9D5EAB00BAEB5F /* X509 in Frameworks */ = {isa = PBXBuildFile; productRef = 5A32BBFA2F9D5EAB00BAEB5F /* X509 */; }; 5A32BC0B2F9D659100BAEB5F /* tablepro-mcp in Copy Files */ = {isa = PBXBuildFile; fileRef = 5A32BC002F9D5F1300BAEB5F /* tablepro-mcp */; settings = {ATTRIBUTES = (CodeSignOnCopy, ); }; }; 5A3BE6FC2F97DB0000611C1F /* TableProPluginKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 5A860000100000000 /* TableProPluginKit.framework */; }; + 5A5YNCTR000000000000A011 /* TableProSyncTransport in Frameworks */ = {isa = PBXBuildFile; productRef = 5A5YNCTR000000000000A010 /* TableProSyncTransport */; }; 5A7E78A02F95F02A00EEF236 /* TableProAnalytics in Frameworks */ = {isa = PBXBuildFile; productRef = 5ACE00012F4F000000000010 /* TableProAnalytics */; }; 5A860000A00000000 /* TableProPluginKit.framework in Embed Frameworks */ = {isa = PBXBuildFile; fileRef = 5A860000100000000 /* TableProPluginKit.framework */; settings = {ATTRIBUTES = (CodeSignOnCopy, RemoveHeadersOnCopy, ); }; }; 5A861000A00000000 /* TableProPluginKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 5A860000100000000 /* TableProPluginKit.framework */; }; @@ -951,6 +952,7 @@ files = ( 5A7E78A02F95F02A00EEF236 /* TableProAnalytics in Frameworks */, 5A1MPORT000000000000A011 /* TableProImport in Frameworks */, + 5A5YNCTR000000000000A011 /* TableProSyncTransport in Frameworks */, 5ACE00012F4F000000000004 /* CodeEditSourceEditor in Frameworks */, 5ACE00012F4F000000000005 /* CodeEditLanguages in Frameworks */, 5A32BBFB2F9D5EAB00BAEB5F /* X509 in Frameworks */, @@ -1482,6 +1484,7 @@ 5ACE00012F4F000000000010 /* TableProAnalytics */, 5A32BBFA2F9D5EAB00BAEB5F /* X509 */, 5A1MPORT000000000000A010 /* TableProImport */, + 5A5YNCTR000000000000A010 /* TableProSyncTransport */, 5AYAMS00000000000000A002 /* Yams */, ); productName = TablePro; @@ -5489,6 +5492,11 @@ package = 5A32BBF92F9D5EAB00BAEB5F /* XCRemoteSwiftPackageReference "swift-certificates" */; productName = X509; }; + 5A5YNCTR000000000000A010 /* TableProSyncTransport */ = { + isa = XCSwiftPackageProductDependency; + package = 5A0000012F4F000000000102 /* XCLocalSwiftPackageReference "Packages/TableProCore" */; + productName = TableProSyncTransport; + }; 5ACE00012F4F000000000002 /* CodeEditSourceEditor */ = { isa = XCSwiftPackageProductDependency; productName = CodeEditSourceEditor; diff --git a/TablePro/Core/Services/AppServices.swift b/TablePro/Core/Services/AppServices.swift index b3afa4d99..19dfd39ff 100644 --- a/TablePro/Core/Services/AppServices.swift +++ b/TablePro/Core/Services/AppServices.swift @@ -24,7 +24,6 @@ struct AppServices { let tagStorage: TagStorage let sshProfileStorage: SSHProfileStorage let licenseManager: LicenseManager - let conflictResolver: ConflictResolver let syncMetadataStorage: SyncMetadataStorage let favoritesExpansionState: FavoritesExpansionState let linkedFolderWatcher: LinkedFolderWatcher @@ -54,7 +53,6 @@ struct AppServices { tagStorage: .shared, sshProfileStorage: .shared, licenseManager: .shared, - conflictResolver: .shared, syncMetadataStorage: .shared, favoritesExpansionState: .shared, linkedFolderWatcher: .shared, diff --git a/TablePro/Core/Sync/CloudKitSyncEngine.swift b/TablePro/Core/Sync/CloudKitSyncEngine.swift index c0129b780..3eaa2b02c 100644 --- a/TablePro/Core/Sync/CloudKitSyncEngine.swift +++ b/TablePro/Core/Sync/CloudKitSyncEngine.swift @@ -9,6 +9,7 @@ import CloudKit import Foundation import os import Security +import TableProSyncTransport /// Result of a pull operation struct PullResult: Sendable { @@ -73,12 +74,14 @@ actor CloudKitSyncEngine { /// CloudKit allows at most 400 items (saves + deletions) per modify operation private static let maxBatchSize = 400 - func push(records: [CKRecord], deletions: [CKRecord.ID]) async throws { - guard !records.isEmpty || !deletions.isEmpty else { return } + @discardableResult + func push(records: [CKRecord], deletions: [CKRecord.ID]) async throws -> PushOutcome { + guard !records.isEmpty || !deletions.isEmpty else { return PushOutcome() } // Split into batches that fit within CloudKit's 400-item limit var remainingSaves = records[...] var remainingDeletions = deletions[...] + var outcome = PushOutcome() while !remainingSaves.isEmpty || !remainingDeletions.isEmpty { let batchSaves: [CKRecord] @@ -92,39 +95,63 @@ actor CloudKitSyncEngine { batchDeletions = Array(remainingDeletions.prefix(deletionsCount)) remainingDeletions = remainingDeletions.dropFirst(deletionsCount) - try await pushBatch(records: batchSaves, deletions: batchDeletions) + outcome.merge(try await pushBatch(records: batchSaves, deletions: batchDeletions)) } - Self.logger.info("Pushed \(records.count) records, \(deletions.count) deletions") + let saved = outcome.savedRecords.count + let deleted = outcome.deletedRecordIDs.count + let failed = outcome.failures.count + Self.logger.info("Pushed \(saved) records, \(deleted) deletions, \(failed) rejected") + + for (recordID, failure) in outcome.failures { + Self.logger.error("CloudKit rejected \(recordID.recordName): \(failure.message)") + } + + return outcome } - private func pushBatch(records: [CKRecord], deletions: [CKRecord.ID]) async throws { + private func pushBatch(records: [CKRecord], deletions: [CKRecord.ID]) async throws -> PushOutcome { guard let database else { throw SyncError.accountUnavailable } try await withRetry { let operation = CKModifyRecordsOperation( recordsToSave: records, recordIDsToDelete: deletions ) - // Use .changedKeys so we don't need to track server change tags - // This overwrites only the fields we set, which is safe for our use case operation.savePolicy = .changedKeys operation.isAtomic = false return try await withCheckedThrowingContinuation { continuation in + var outcome = PushOutcome() + operation.perRecordSaveBlock = { recordID, result in - if case .failure(let error) = result { - Self.logger.error( - "Failed to save record \(recordID.recordName): \(error.localizedDescription)" - ) + switch result { + case .success(let record): + outcome.recordSave(record) + case .failure(let error): + outcome.recordFailure(SyncItemFailure(error: error), for: recordID) + } + } + + operation.perRecordDeleteBlock = { recordID, result in + switch result { + case .success: + outcome.recordDeletion(recordID) + case .failure(let error): + outcome.recordFailure(SyncItemFailure(error: error), for: recordID) } } operation.modifyRecordsResultBlock = { result in switch result { case .success: - continuation.resume() + continuation.resume(returning: outcome) case .failure(let error): - continuation.resume(throwing: error) + guard let ckError = error as? CKError, ckError.code == .partialFailure else { + continuation.resume(throwing: error) + return + } + outcome.absorbPartialErrors(from: ckError) + continuation.resume(returning: outcome) } } database.add(operation) @@ -135,12 +162,35 @@ actor CloudKitSyncEngine { // MARK: - Pull func pull(since token: CKServerChangeToken?) async throws -> PullResult { - try await withRetry { - try await performPull(since: token) + var changedRecords: [CKRecord] = [] + var deletedRecordIDs: [CKRecord.ID] = [] + var cursor = token + + while true { + let page = try await withRetry { [cursor] in + try await performPull(since: cursor) + } + + changedRecords.append(contentsOf: page.result.changedRecords) + deletedRecordIDs.append(contentsOf: page.result.deletedRecordIDs) + cursor = page.result.newToken ?? cursor + + guard page.moreComing, page.result.newToken != nil else { + return PullResult( + changedRecords: changedRecords, + deletedRecordIDs: deletedRecordIDs, + newToken: cursor + ) + } } } - private func performPull(since token: CKServerChangeToken?) async throws -> PullResult { + private struct PullPage { + let result: PullResult + let moreComing: Bool + } + + private func performPull(since token: CKServerChangeToken?) async throws -> PullPage { guard let database else { throw SyncError.accountUnavailable } let configuration = CKFetchRecordZoneChangesOperation.ZoneConfiguration() configuration.previousServerChangeToken = token @@ -153,6 +203,7 @@ actor CloudKitSyncEngine { var changedRecords: [CKRecord] = [] var deletedRecordIDs: [CKRecord.ID] = [] var newToken: CKServerChangeToken? + var moreComing = false return try await withCheckedThrowingContinuation { continuation in operation.recordWasChangedBlock = { _, result in @@ -171,8 +222,9 @@ actor CloudKitSyncEngine { operation.recordZoneFetchResultBlock = { _, result in switch result { - case .success(let (serverToken, _, _)): + case .success(let (serverToken, _, hasMore)): newToken = serverToken + moreComing = hasMore case .failure(let error): Self.logger.warning("Zone fetch result error: \(error.localizedDescription)") } @@ -181,12 +233,15 @@ actor CloudKitSyncEngine { operation.fetchRecordZoneChangesResultBlock = { result in switch result { case .success: - let pullResult = PullResult( - changedRecords: changedRecords, - deletedRecordIDs: deletedRecordIDs, - newToken: newToken + let page = PullPage( + result: PullResult( + changedRecords: changedRecords, + deletedRecordIDs: deletedRecordIDs, + newToken: newToken + ), + moreComing: moreComing ) - continuation.resume(returning: pullResult) + continuation.resume(returning: page) case .failure(let error): continuation.resume(throwing: error) } diff --git a/TablePro/Core/Sync/ConflictResolver.swift b/TablePro/Core/Sync/ConflictResolver.swift deleted file mode 100644 index 3d265bf41..000000000 --- a/TablePro/Core/Sync/ConflictResolver.swift +++ /dev/null @@ -1,87 +0,0 @@ -// -// ConflictResolver.swift -// TablePro -// -// Queues and resolves sync conflicts one at a time -// - -import CloudKit -import Foundation -import Observation -import os - -/// Represents a sync conflict between local and remote versions -struct SyncConflict: Identifiable { - let id: UUID - let recordType: SyncRecordType - let entityName: String - let localRecord: CKRecord - let serverRecord: CKRecord - let localModifiedAt: Date - let serverModifiedAt: Date - - init( - recordType: SyncRecordType, - entityName: String, - localRecord: CKRecord, - serverRecord: CKRecord, - localModifiedAt: Date, - serverModifiedAt: Date - ) { - self.id = UUID() - self.recordType = recordType - self.entityName = entityName - self.localRecord = localRecord - self.serverRecord = serverRecord - self.localModifiedAt = localModifiedAt - self.serverModifiedAt = serverModifiedAt - } -} - -/// Manages a queue of sync conflicts for user resolution -@MainActor @Observable -final class ConflictResolver { - static let shared = ConflictResolver() - private static let logger = Logger(subsystem: "com.TablePro", category: "ConflictResolver") - - private(set) var pendingConflicts: [SyncConflict] = [] - - var hasConflicts: Bool { !pendingConflicts.isEmpty } - - var currentConflict: SyncConflict? { pendingConflicts.first } - - private init() {} - - func addConflict(_ conflict: SyncConflict) { - pendingConflicts.append(conflict) - let count = pendingConflicts.count - Self.logger.trace( - "Conflict queued: \(conflict.recordType.rawValue)/\(conflict.entityName) (\(count) pending)" - ) - } - - /// Resolve the current (first) conflict. - /// Returns the CKRecord to push if keeping local; nil if keeping server version. - @discardableResult - func resolveCurrentConflict(keepLocal: Bool) -> CKRecord? { - guard let conflict = pendingConflicts.first else { return nil } - - pendingConflicts.removeFirst() - let resolution = keepLocal ? "local" : "server" - let remaining = pendingConflicts.count - Self.logger.trace( - "Resolved conflict: \(conflict.recordType.rawValue)/\(conflict.entityName) — kept \(resolution) (\(remaining) remaining)" - ) - - if keepLocal { - // Copy local field values onto the server record to update its change tag - let resolved = conflict.serverRecord - for key in conflict.localRecord.allKeys() { - resolved[key] = conflict.localRecord[key] - } - return resolved - } - - return nil - } -} diff --git a/TablePro/Core/Sync/SyncCoordinator.swift b/TablePro/Core/Sync/SyncCoordinator.swift index 712018216..41fdd0d7e 100644 --- a/TablePro/Core/Sync/SyncCoordinator.swift +++ b/TablePro/Core/Sync/SyncCoordinator.swift @@ -10,6 +10,7 @@ import Combine import Foundation import Observation import os +import TableProSyncTransport /// Central coordinator for iCloud sync @MainActor @Observable @@ -25,7 +26,7 @@ final class SyncCoordinator { @ObservationIgnored private let engine = CloudKitSyncEngine() @ObservationIgnored private let changeTracker: SyncChangeTracker @ObservationIgnored private let metadataStorage: SyncMetadataStorage - @ObservationIgnored private let conflictResolver: ConflictResolver + @ObservationIgnored private let recordCache = SyncRecordCache() @ObservationIgnored private var accountObserver: NSObjectProtocol? @ObservationIgnored private var changeCancellable: AnyCancellable? @ObservationIgnored private var licenseCancellable: AnyCancellable? @@ -36,7 +37,6 @@ final class SyncCoordinator { self.services = services self.changeTracker = services.syncTracker self.metadataStorage = services.syncMetadataStorage - self.conflictResolver = services.conflictResolver lastSyncDate = metadataStorage.lastSyncDate } @@ -97,9 +97,22 @@ final class SyncCoordinator { do { try await engine.ensureZoneExists() - await performPush() + + var pushError: Error? + do { + try await performPush() + } catch { + pushError = error + Self.logger.error("Push failed: \(error.localizedDescription)") + } + await performPull() + if let pushError { + syncStatus = .error(SyncError.from(pushError)) + return + } + lastSyncDate = Date() metadataStorage.lastSyncDate = lastSyncDate syncStatus = .idle @@ -267,7 +280,7 @@ final class SyncCoordinator { // MARK: - Push - private func performPush() async { + private func performPush() async throws { let settings = services.appSettingsStorage.loadSync() var recordsToSave: [CKRecord] = [] var recordIDsToDelete: [CKRecord.ID] = [] @@ -280,8 +293,13 @@ final class SyncCoordinator { for id in dirtyConnectionIds { if let connection = connections.first(where: { $0.id.uuidString == id }), !connection.localOnly { + let recordID = SyncRecordMapper.recordID(type: .connection, id: id, in: zoneID) recordsToSave.append( - SyncRecordMapper.toCKRecord(connection, in: zoneID) + SyncRecordMapper.toCKRecord( + connection, + in: zoneID, + base: recordCache.record(for: recordID) + ) ) } } @@ -328,75 +346,30 @@ final class SyncCoordinator { guard !recordsToSave.isEmpty || !uniqueDeletions.isEmpty else { return } - do { - try await engine.push(records: recordsToSave, deletions: uniqueDeletions) + let outcome = try await engine.push(records: recordsToSave, deletions: uniqueDeletions) - if settings.syncConnections { - changeTracker.clearAllDirty(.connection) - } - if settings.syncGroupsAndTags { - changeTracker.clearAllDirty(.group) - changeTracker.clearAllDirty(.tag) - } - if settings.syncSSHProfiles { - changeTracker.clearAllDirty(.sshProfile) - } - if settings.syncSettings { - changeTracker.clearAllDirty(.settings) - } - if settings.syncTableFavorites { - changeTracker.clearAllDirty(.tableFavorite) - } - if settings.syncSQLFavorites { - changeTracker.clearAllDirty(.favorite) - changeTracker.clearAllDirty(.favoriteFolder) - } + recordCache.store(Array(outcome.savedRecords.values)) + recordCache.remove(Array(outcome.deletedRecordIDs)) - // Clear tombstones only for types that were actually pushed - if settings.syncConnections { - for tombstone in metadataStorage.tombstones(for: .connection) { - metadataStorage.removeTombstone(type: .connection, id: tombstone.id) - } - } - if settings.syncGroupsAndTags { - for tombstone in metadataStorage.tombstones(for: .group) { - metadataStorage.removeTombstone(type: .group, id: tombstone.id) - } - for tombstone in metadataStorage.tombstones(for: .tag) { - metadataStorage.removeTombstone(type: .tag, id: tombstone.id) - } - } - if settings.syncSSHProfiles { - for tombstone in metadataStorage.tombstones(for: .sshProfile) { - metadataStorage.removeTombstone(type: .sshProfile, id: tombstone.id) - } - } - if settings.syncSettings { - for tombstone in metadataStorage.tombstones(for: .settings) { - metadataStorage.removeTombstone(type: .settings, id: tombstone.id) - } - } - if settings.syncTableFavorites { - for tombstone in metadataStorage.tombstones(for: .tableFavorite) { - metadataStorage.removeTombstone(type: .tableFavorite, id: tombstone.id) - } - } - if settings.syncSQLFavorites { - for tombstone in metadataStorage.tombstones(for: .favorite) { - metadataStorage.removeTombstone(type: .favorite, id: tombstone.id) - } - for tombstone in metadataStorage.tombstones(for: .favoriteFolder) { - metadataStorage.removeTombstone(type: .favoriteFolder, id: tombstone.id) - } - } + for recordID in outcome.savedRecords.keys { + guard let parsed = SyncRecordMapper.parse(recordName: recordID.recordName) else { continue } + changeTracker.clearDirty(parsed.type, id: parsed.id) + } - Self.logger.info("Push completed: \(recordsToSave.count) saved, \(recordIDsToDelete.count) deleted") - } catch let error as CKError where error.code == .serverRecordChanged { - Self.logger.warning("Server record changed during push — conflicts detected") - handlePushConflicts(error) - } catch { - Self.logger.error("Push failed: \(error.localizedDescription)") + for recordID in outcome.deletedRecordIDs { + guard let parsed = SyncRecordMapper.parse(recordName: recordID.recordName) else { continue } + metadataStorage.removeTombstone(type: parsed.type, id: parsed.id) } + + Self.logger.info( + "Push completed: \(outcome.savedRecords.count) saved, " + + "\(outcome.deletedRecordIDs.count) deleted, \(outcome.failures.count) rejected" + ) + + guard outcome.hasFailures else { return } + + guard let firstFailure = outcome.failures.values.first else { return } + throw SyncError.pushRejected(count: outcome.failures.count, detail: firstFailure.message) } // MARK: - Pull @@ -430,6 +403,9 @@ final class SyncCoordinator { applyRemoteChanges(result) + recordCache.store(result.changedRecords) + recordCache.remove(result.deletedRecordIDs) + Self.logger.info( "Pull completed: \(result.changedRecords.count) changed, \(result.deletedRecordIDs.count) deleted" ) @@ -595,6 +571,25 @@ final class SyncCoordinator { } @discardableResult + private func mergeLocalEdits(into remoteRecord: CKRecord, localConnection: DatabaseConnection) -> DatabaseConnection? { + guard let base = recordCache.record(for: remoteRecord.recordID) else { return nil } + + let localRecord = SyncRecordMapper.toCKRecord(localConnection, in: remoteRecord.recordID.zoneID) + guard let merged = remoteRecord.copy() as? CKRecord else { return nil } + + for field in ConnectionSyncField.allCases where field != .modifiedAtLocal { + guard !CKRecord.isEqualRecordValue(localRecord[field], base[field]) else { continue } + merged[field] = localRecord[field] + } + + do { + return try SyncRecordMapper.toConnection(merged) + } catch { + Self.logger.error("Failed to merge local edits: \(error.localizedDescription)") + return nil + } + } + private func applyRemoteConnection(_ record: CKRecord, tombstoneIds: Set) -> Bool { let remoteConnection: DatabaseConnection do { @@ -610,26 +605,17 @@ final class SyncCoordinator { var connections = services.connectionStorage.loadConnections() if let index = connections.firstIndex(where: { $0.id == remoteConnection.id }) { + var incoming = remoteConnection if changeTracker.dirtyRecords(for: .connection).contains(remoteConnection.id.uuidString) { - let localRecord = SyncRecordMapper.toCKRecord( - connections[index], - in: CKRecordZone.ID( - zoneName: "TableProSync", - ownerName: CKCurrentUserDefaultName - ) - ) - let conflict = SyncConflict( - recordType: .connection, - entityName: remoteConnection.name, - localRecord: localRecord, - serverRecord: record, - localModifiedAt: (localRecord["modifiedAtLocal"] as? Date) ?? Date(), - serverModifiedAt: (record["modifiedAtLocal"] as? Date) ?? Date() - ) - conflictResolver.addConflict(conflict) - return false + guard let reconciled = mergeLocalEdits( + into: record, + localConnection: connections[index] + ) else { + return false + } + incoming = reconciled } - var merged = remoteConnection + var merged = incoming merged.localOnly = connections[index].localOnly merged.passwordSource = connections[index].passwordSource connections[index] = merged @@ -803,53 +789,6 @@ final class SyncCoordinator { // MARK: - Conflict Handling - private func handlePushConflicts(_ error: CKError) { - guard let partialErrors = error.partialErrorsByItemID else { return } - - for (_, itemError) in partialErrors { - guard let ckError = itemError as? CKError, - ckError.code == .serverRecordChanged, - let serverRecord = ckError.serverRecord, - let clientRecord = ckError.clientRecord - else { continue } - - let recordType = serverRecord.recordType - let entityName = (serverRecord["name"] as? String) ?? recordType - - let syncRecordType: SyncRecordType - switch recordType { - case SyncRecordType.connection.rawValue: syncRecordType = .connection - case SyncRecordType.group.rawValue: syncRecordType = .group - case SyncRecordType.tag.rawValue: syncRecordType = .tag - case SyncRecordType.settings.rawValue: syncRecordType = .settings - case SyncRecordType.sshProfile.rawValue: syncRecordType = .sshProfile - case SyncRecordType.tableFavorite.rawValue: syncRecordType = .tableFavorite - default: continue - } - - let conflict = SyncConflict( - recordType: syncRecordType, - entityName: entityName, - localRecord: clientRecord, - serverRecord: serverRecord, - localModifiedAt: (clientRecord["modifiedAtLocal"] as? Date) ?? Date(), - serverModifiedAt: (serverRecord["modifiedAtLocal"] as? Date) ?? Date() - ) - conflictResolver.addConflict(conflict) - } - } - - /// Push a resolved conflict record back to CloudKit - func pushResolvedConflict(_ record: CKRecord) { - Task { - do { - try await engine.push(records: [record], deletions: []) - } catch { - Self.logger.error("Failed to push resolved conflict: \(error.localizedDescription)") - } - } - } - // MARK: - Settings Helpers private func settingsData(for category: String) -> Data? { diff --git a/TablePro/Core/Sync/SyncError.swift b/TablePro/Core/Sync/SyncError.swift index c3ed1afe0..f00809207 100644 --- a/TablePro/Core/Sync/SyncError.swift +++ b/TablePro/Core/Sync/SyncError.swift @@ -17,6 +17,7 @@ enum SyncError: LocalizedError, Equatable { case serverError(String) case conflictDetected case encodingFailed(String) + case pushRejected(count: Int, detail: String) case unknown(String) var errorDescription: String? { @@ -35,6 +36,12 @@ enum SyncError: LocalizedError, Equatable { return String(localized: "A sync conflict was detected and needs to be resolved.") case .encodingFailed(let detail): return String(format: String(localized: "Failed to encode sync data: %@"), detail) + case .pushRejected(let count, let detail): + return String( + format: String(localized: "iCloud rejected %d change(s). They stay on this Mac and will retry: %@"), + count, + detail + ) case .unknown(let message): return String(format: String(localized: "An unknown sync error occurred: %@"), message) } @@ -76,6 +83,8 @@ enum SyncError: LocalizedError, Equatable { return a == b case (.encodingFailed(let a), .encodingFailed(let b)): return a == b + case (.pushRejected(let aCount, let aDetail), .pushRejected(let bCount, let bDetail)): + return aCount == bCount && aDetail == bDetail case (.unknown(let a), .unknown(let b)): return a == b default: diff --git a/TablePro/Core/Sync/SyncRecordMapper.swift b/TablePro/Core/Sync/SyncRecordMapper.swift index 0e969be36..a1aba9508 100644 --- a/TablePro/Core/Sync/SyncRecordMapper.swift +++ b/TablePro/Core/Sync/SyncRecordMapper.swift @@ -10,6 +10,7 @@ import Foundation import os import TableProImport import TableProPluginKit +import TableProSyncTransport /// CloudKit record types for sync enum SyncRecordType: String, CaseIterable { @@ -63,52 +64,78 @@ struct SyncRecordMapper { return CKRecord.ID(recordName: recordName, zoneID: zone) } + static func parse(recordName: String) -> (type: SyncRecordType, id: String)? { + let prefixes: [(SyncRecordType, String)] = [ + (.connection, "Connection_"), + (.group, "Group_"), + (.tag, "Tag_"), + (.settings, "Settings_"), + (.favoriteFolder, "FavoriteFolder_"), + (.tableFavorite, "FavoriteTable_"), + (.favorite, "Favorite_"), + (.sshProfile, "SSHProfile_") + ] + for (type, prefix) in prefixes where recordName.hasPrefix(prefix) { + return (type, String(recordName.dropFirst(prefix.count))) + } + return nil + } + // MARK: - Connection - static func toCKRecord(_ connection: DatabaseConnection, in zone: CKRecordZone.ID) -> CKRecord { + static func toCKRecord( + _ connection: DatabaseConnection, + in zone: CKRecordZone.ID, + base: CKRecord? = nil + ) -> CKRecord { let recordID = recordID(type: .connection, id: connection.id.uuidString, in: zone) - let record = CKRecord(recordType: SyncRecordType.connection.rawValue, recordID: recordID) - - record["connectionId"] = connection.id.uuidString as CKRecordValue - record["name"] = connection.name as CKRecordValue - record["host"] = connection.host as CKRecordValue - record["port"] = Int64(connection.port) as CKRecordValue - record["database"] = connection.database as CKRecordValue - record["username"] = connection.username as CKRecordValue - record["type"] = connection.type.rawValue as CKRecordValue - record["color"] = connection.color.rawValue as CKRecordValue - record["safeModeLevel"] = connection.safeModeLevel.rawValue as CKRecordValue - record["modifiedAtLocal"] = Date() as CKRecordValue - record["schemaVersion"] = schemaVersion as CKRecordValue - record["sortOrder"] = Int64(connection.sortOrder) as CKRecordValue - record["isFavorite"] = Int64(connection.isFavorite ? 1 : 0) as CKRecordValue + let record: CKRecord + if let base, base.recordID == recordID { + record = base + } else { + record = CKRecord(recordType: SyncRecordType.connection.rawValue, recordID: recordID) + } + + record[.connectionId] = connection.id.uuidString as CKRecordValue + record[.name] = connection.name as CKRecordValue + record[.host] = connection.host as CKRecordValue + record[.port] = Int64(connection.port) as CKRecordValue + record[.database] = connection.database as CKRecordValue + record[.username] = connection.username as CKRecordValue + record[.type] = connection.type.rawValue as CKRecordValue + record[.color] = connection.color.rawValue as CKRecordValue + record[.safeModeLevel] = connection.safeModeLevel.rawValue as CKRecordValue + record[.modifiedAtLocal] = Date() as CKRecordValue + record[.schemaVersion] = schemaVersion as CKRecordValue + record[.sortOrder] = Int64(connection.sortOrder) as CKRecordValue + record[.isFavorite] = Int64(connection.isFavorite ? 1 : 0) as CKRecordValue if !connection.tagIds.isEmpty { let tagIdStrings = connection.tagIds.map { $0.uuidString } - record["tagIds"] = tagIdStrings as CKRecordValue - record["tagId"] = tagIdStrings[0] as CKRecordValue + record[.tagIds] = tagIdStrings as CKRecordValue + record[.tagId] = tagIdStrings[0] as CKRecordValue } if let groupId = connection.groupId { - record["groupId"] = groupId.uuidString as CKRecordValue + record[.groupId] = groupId.uuidString as CKRecordValue } if let aiPolicy = connection.aiPolicy { - record["aiPolicy"] = aiPolicy.rawValue as CKRecordValue + record[.aiPolicy] = aiPolicy.rawValue as CKRecordValue } if let aiRules = connection.aiRules, !aiRules.isEmpty { - record["aiRules"] = aiRules as CKRecordValue + record[.aiRules] = aiRules as CKRecordValue } if !connection.aiAlwaysAllowedTools.isEmpty { let sorted = Array(connection.aiAlwaysAllowedTools).sorted() - record["aiAlwaysAllowedTools"] = sorted as CKRecordValue + record[.aiAlwaysAllowedTools] = sorted as CKRecordValue } if let redisDatabase = connection.redisDatabase { - record["redisDatabase"] = Int64(redisDatabase) as CKRecordValue + record[.redisDatabase] = Int64(redisDatabase) as CKRecordValue } if let startupCommands = connection.startupCommands { - record["startupCommands"] = startupCommands as CKRecordValue + record[.startupCommands] = startupCommands as CKRecordValue } if let sshProfileId = connection.sshProfileId { - record["sshProfileId"] = sshProfileId.uuidString as CKRecordValue + record[.sshProfileId] = sshProfileId.uuidString as CKRecordValue } // Encode complex structs as JSON Data — contract device-local paths @@ -123,20 +150,20 @@ struct SyncRecordMapper { // is device-local and may not exist or resolve on another Mac. do { let sshData = try encoder.encode(Self.makePortable(connection.sshConfig)) - record["sshConfigJson"] = sshData as CKRecordValue + record[.sshConfigJson] = sshData as CKRecordValue } catch { logger.warning("Failed to encode SSH config for sync: \(error.localizedDescription)") } do { let sslData = try encoder.encode(Self.makePortable(connection.sslConfig)) - record["sslConfigJson"] = sslData as CKRecordValue + record[.sslConfigJson] = sslData as CKRecordValue } catch { logger.warning("Failed to encode SSL config for sync: \(error.localizedDescription)") } if !connection.additionalFields.isEmpty { do { let fieldsData = try encoder.encode(connection.additionalFields) - record["additionalFieldsJson"] = fieldsData as CKRecordValue + record[.additionalFieldsJson] = fieldsData as CKRecordValue } catch { logger.warning("Failed to encode additional fields for sync: \(error.localizedDescription)") } @@ -146,44 +173,45 @@ struct SyncRecordMapper { } static func toConnection(_ record: CKRecord) throws -> DatabaseConnection { - guard let connectionIdString = record["connectionId"] as? String, + guard let connectionIdString = record[.connectionId] as? String, let connectionId = UUID(uuidString: connectionIdString) else { throw SyncDecodeError.missingRequiredField("connectionId") } - guard let name = record["name"] as? String else { + guard let name = record[.name] as? String else { throw SyncDecodeError.missingRequiredField("name") } - guard let typeRawValue = record["type"] as? String else { + guard let typeRawValue = record[.type] as? String else { throw SyncDecodeError.missingRequiredField("type") } - let host = record["host"] as? String ?? "localhost" - let port = (record["port"] as? Int64).map { Int($0) } ?? 0 - let database = record["database"] as? String ?? "" - let username = record["username"] as? String ?? "" - let colorRaw = record["color"] as? String ?? ConnectionColor.none.rawValue - let safeModeLevelRaw = record["safeModeLevel"] as? String ?? SafeModeLevel.silent.rawValue + let host = record[.host] as? String ?? "localhost" + let port = (record[.port] as? Int64).map { Int($0) } ?? 0 + let database = record[.database] as? String ?? "" + let username = record[.username] as? String ?? "" + let colorRaw = record[.color] as? String ?? ConnectionColor.none.rawValue + let isReadOnly = (record[.isReadOnly] as? Int64 ?? 0) != 0 + let safeModeLevel = Self.safeModeLevel(fromWire: record[.safeModeLevel] as? String, isReadOnly: isReadOnly) let tagIds: [UUID] - if let rawIds = record["tagIds"] as? [String], !rawIds.isEmpty { + if let rawIds = record[.tagIds] as? [String], !rawIds.isEmpty { tagIds = rawIds.compactMap { UUID(uuidString: $0) } - } else if let single = (record["tagId"] as? String).flatMap({ UUID(uuidString: $0) }) { + } else if let single = (record[.tagId] as? String).flatMap({ UUID(uuidString: $0) }) { tagIds = [single] } else { tagIds = [] } - let groupId = (record["groupId"] as? String).flatMap { UUID(uuidString: $0) } - let aiPolicyRaw = record["aiPolicy"] as? String - let aiRulesRaw = record["aiRules"] as? String - let aiAlwaysAllowedToolsArray = record["aiAlwaysAllowedTools"] as? [String] ?? [] - let redisDatabase = (record["redisDatabase"] as? Int64).map { Int($0) } - let startupCommands = record["startupCommands"] as? String - let sortOrder = (record["sortOrder"] as? Int64).map { Int($0) } ?? 0 - let isFavorite = (record["isFavorite"] as? Int64 ?? 0) != 0 - let sshProfileId = (record["sshProfileId"] as? String).flatMap { UUID(uuidString: $0) } + let groupId = (record[.groupId] as? String).flatMap { UUID(uuidString: $0) } + let aiPolicyRaw = record[.aiPolicy] as? String + let aiRulesRaw = record[.aiRules] as? String + let aiAlwaysAllowedToolsArray = record[.aiAlwaysAllowedTools] as? [String] ?? [] + let redisDatabase = (record[.redisDatabase] as? Int64).map { Int($0) } + let startupCommands = record[.startupCommands] as? String + let sortOrder = (record[.sortOrder] as? Int64).map { Int($0) } ?? 0 + let isFavorite = (record[.isFavorite] as? Int64 ?? 0) != 0 + let sshProfileId = (record[.sshProfileId] as? String).flatMap { UUID(uuidString: $0) } var sshConfig = SSHConfiguration() - if let sshData = record["sshConfigJson"] as? Data { + if let sshData = record[.sshConfigJson] as? Data { do { sshConfig = try decoder.decode(SSHConfiguration.self, from: sshData) } catch { @@ -194,7 +222,7 @@ struct SyncRecordMapper { let connectionType = DatabaseType(rawValue: typeRawValue) var sslConfig = SSLConfiguration(mode: connectionType.defaultSSLMode) - if let sslData = record["sslConfigJson"] as? Data { + if let sslData = record[.sslConfigJson] as? Data { do { sslConfig = try decoder.decode(SSLConfiguration.self, from: sslData) } catch { @@ -204,7 +232,7 @@ struct SyncRecordMapper { } var additionalFields: [String: String]? - if let fieldsData = record["additionalFieldsJson"] as? Data { + if let fieldsData = record[.additionalFieldsJson] as? Data { do { additionalFields = try decoder.decode([String: String].self, from: fieldsData) } catch { @@ -226,7 +254,7 @@ struct SyncRecordMapper { tagIds: tagIds, groupId: groupId, sshProfileId: sshProfileId, - safeModeLevel: SafeModeLevel(rawValue: safeModeLevelRaw) ?? .silent, + safeModeLevel: safeModeLevel, aiPolicy: aiPolicyRaw.flatMap { AIConnectionPolicy(rawValue: $0) }, aiRules: aiRulesRaw, aiAlwaysAllowedTools: Set(aiAlwaysAllowedToolsArray), @@ -238,6 +266,16 @@ struct SyncRecordMapper { ) } + static func safeModeLevel(fromWire raw: String?, isReadOnly: Bool) -> SafeModeLevel { + guard let raw else { return isReadOnly ? .readOnly : .silent } + if let level = SafeModeLevel(rawValue: raw) { return level } + switch raw { + case "off": return .silent + case "confirmWrites": return .alert + default: return isReadOnly ? .readOnly : .alert + } + } + // MARK: - Connection Group static func toCKRecord(_ group: ConnectionGroup, in zone: CKRecordZone.ID) -> CKRecord { diff --git a/TablePro/Views/Components/ConflictResolutionView.swift b/TablePro/Views/Components/ConflictResolutionView.swift deleted file mode 100644 index e30284890..000000000 --- a/TablePro/Views/Components/ConflictResolutionView.swift +++ /dev/null @@ -1,206 +0,0 @@ -// -// ConflictResolutionView.swift -// TablePro -// -// Sheet for resolving sync conflicts between local and remote versions -// - -import CloudKit -import SwiftUI - -struct ConflictResolutionView: View { - @Bindable private var conflictResolver = ConflictResolver.shared - @Environment(\.dismiss) private var dismiss - - var body: some View { - if let conflict = conflictResolver.currentConflict { - VStack(spacing: 16) { - header(for: conflict) - description(for: conflict) - comparisonBoxes(for: conflict) - actionButtons(for: conflict) - progressIndicator - } - .padding(24) - .frame(width: 500) - } - } - - // MARK: - Header - - private func header(for conflict: SyncConflict) -> some View { - HStack(spacing: 8) { - Image(systemName: "exclamationmark.triangle.fill") - .font(.title2) - .foregroundStyle(.orange) - Text(String(localized: "Sync Conflict")) - .font(.headline) - } - } - - // MARK: - Description - - private func description(for conflict: SyncConflict) -> some View { - Group { - if conflict.recordType == .settings { - Text(String(localized: "Settings were changed on both this Mac and another device.")) - .font(.subheadline) - .foregroundStyle(.secondary) - } else { - Text( - String( - localized: "\"\(conflict.entityName)\" was modified on both this Mac and another device." - ) - ) - .font(.subheadline) - .foregroundStyle(.secondary) - } - } - .multilineTextAlignment(.center) - } - - // MARK: - Comparison Boxes - - private func comparisonBoxes(for conflict: SyncConflict) -> some View { - HStack(spacing: 12) { - GroupBox { - VStack(alignment: .leading, spacing: 8) { - Label(String(localized: "This Mac"), systemImage: "desktopcomputer") - .font(.subheadline.bold()) - - Divider() - - LabeledContent(String(localized: "Modified:")) { - Text(conflict.localModifiedAt, style: .date) - Text(conflict.localModifiedAt, style: .time) - } - .font(.caption) - - changedFields(from: conflict.localRecord, conflict: conflict) - } - .frame(maxWidth: .infinity, alignment: .leading) - .padding(4) - } - - GroupBox { - VStack(alignment: .leading, spacing: 8) { - Label(String(localized: "Other Device"), systemImage: "laptopcomputer") - .font(.subheadline.bold()) - - Divider() - - LabeledContent(String(localized: "Modified:")) { - Text(conflict.serverModifiedAt, style: .date) - Text(conflict.serverModifiedAt, style: .time) - } - .font(.caption) - - changedFields(from: conflict.serverRecord, conflict: conflict) - } - .frame(maxWidth: .infinity, alignment: .leading) - .padding(4) - } - } - } - - @ViewBuilder - private func changedFields(from record: CKRecord, conflict: SyncConflict) -> some View { - switch conflict.recordType { - case .connection: - if let host = record["host"] as? String { - fieldRow(label: "Host", value: host) - } - if let port = record["port"] as? Int64 { - fieldRow(label: "Port", value: "\(port)") - } - if let database = record["database"] as? String { - fieldRow(label: "Database", value: database) - } - if let username = record["username"] as? String { - fieldRow(label: "User", value: username) - } - case .settings: - Text(String(localized: "Settings were changed")) - .font(.caption) - .foregroundStyle(.secondary) - case .group, .tag: - if let name = record["name"] as? String { - fieldRow(label: "Name", value: name) - } - if let color = record["color"] as? String { - fieldRow(label: "Color", value: color) - } - case .favorite, .favoriteFolder, .tableFavorite: - if let name = record["name"] as? String { - fieldRow(label: String(localized: "Name"), value: name) - } - case .sshProfile: - if let name = record["name"] as? String { - fieldRow(label: String(localized: "Name"), value: name) - } - if let host = record["host"] as? String { - fieldRow(label: "Host", value: host) - } - } - } - - private func fieldRow(label: String, value: String) -> some View { - LabeledContent(label + ":") { - Text(value) - .lineLimit(1) - } - .font(.caption) - } - - // MARK: - Action Buttons - - private func actionButtons(for conflict: SyncConflict) -> some View { - HStack(spacing: 12) { - Button(String(localized: "Keep Other Version")) { - resolveConflict(keepLocal: false) - } - .buttonStyle(.bordered) - - Button(String(localized: "Keep This Mac's Version")) { - resolveConflict(keepLocal: true) - } - .buttonStyle(.borderedProminent) - } - } - - // MARK: - Progress - - private var progressIndicator: some View { - Group { - let total = conflictResolver.pendingConflicts.count - if total > 1 { - Text( - String( - localized: "1 of \(total) conflicts" - ) - ) - .font(.caption) - .foregroundStyle(.secondary) - } - } - } - - // MARK: - Actions - - private func resolveConflict(keepLocal: Bool) { - let resolvedRecord = conflictResolver.resolveCurrentConflict(keepLocal: keepLocal) - - if let record = resolvedRecord { - SyncCoordinator.shared.pushResolvedConflict(record) - } - - if !conflictResolver.hasConflicts { - dismiss() - } - } -} - -#Preview { - ConflictResolutionView() - .frame(width: 500) -} diff --git a/TableProMobile/TableProMobile.xcodeproj/project.pbxproj b/TableProMobile/TableProMobile.xcodeproj/project.pbxproj index ccc0c5fd2..c7f8644c4 100644 --- a/TableProMobile/TableProMobile.xcodeproj/project.pbxproj +++ b/TableProMobile/TableProMobile.xcodeproj/project.pbxproj @@ -10,6 +10,7 @@ 5A72D6232F97A69500E2ADE0 /* Secrets.xcconfig in Resources */ = {isa = PBXBuildFile; fileRef = 5A72D6222F97A69500E2ADE0 /* Secrets.xcconfig */; }; 5A7E81B12F95F23600EEF236 /* TableProAnalytics in Frameworks */ = {isa = PBXBuildFile; productRef = 5A87EEED2F7F893000D028D1 /* TableProAnalytics */; }; 5A87EEED2F7F893000D028D0 /* TableProSync in Frameworks */ = {isa = PBXBuildFile; productRef = 5A87EEEC2F7F893000D028D0 /* TableProSync */; }; + 5A87EEEE2F7F893000D028E0 /* TableProSyncTransport in Frameworks */ = {isa = PBXBuildFile; productRef = 5A87EEEF2F7F893000D028E1 /* TableProSyncTransport */; }; 5AA136062F82610F00ADCD58 /* WidgetKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 5AA136052F82610F00ADCD58 /* WidgetKit.framework */; }; 5AA136082F82610F00ADCD58 /* SwiftUI.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 5AA136072F82610F00ADCD58 /* SwiftUI.framework */; }; 5AA136132F82611000ADCD58 /* TableProWidgetExtension.appex in Embed Foundation Extensions */ = {isa = PBXBuildFile; fileRef = 5AA136042F82610F00ADCD58 /* TableProWidgetExtension.appex */; settings = {ATTRIBUTES = (RemoveHeadersOnCopy, ); }; }; @@ -616,6 +617,7 @@ 5AD0CCDB2F0000000000F002 /* DuckDB.xcframework in Frameworks */, 5AD1F1B62FB4455700296783 /* TableProMSSQLCore in Frameworks */, 5A87EEED2F7F893000D028D0 /* TableProSync in Frameworks */, + 5A87EEEE2F7F893000D028E0 /* TableProSyncTransport in Frameworks */, 5AB9F3EB2F7C1D03001F3337 /* TableProModels in Frameworks */, 5A1MPORT00000000000000I1 /* TableProImport in Frameworks */, ); @@ -1622,17 +1624,25 @@ sourceTree = ""; }; 5A87EEEB2F7F891F00D028D0 /* TableProSync */ = { + isa = PBXGroup; + children = ( + 5A87EEE92F7F891F00D028D0 /* SyncRecordMapper.swift */, + ); + name = TableProSync; + path = ../Packages/TableProCore/Sources/TableProSync; + sourceTree = ""; + }; + 5A87EEF02F7F891F00D028E2 /* TableProSyncTransport */ = { isa = PBXGroup; children = ( 5A87EEE52F7F891F00D028D0 /* CloudKitSyncEngine.swift */, 5A87EEE62F7F891F00D028D0 /* SyncConflict.swift */, 5A87EEE72F7F891F00D028D0 /* SyncError.swift */, 5A87EEE82F7F891F00D028D0 /* SyncMetadataStorage.swift */, - 5A87EEE92F7F891F00D028D0 /* SyncRecordMapper.swift */, 5A87EEEA2F7F891F00D028D0 /* SyncRecordType.swift */, ); - name = TableProSync; - path = ../Packages/TableProCore/Sources/TableProSync; + name = TableProSyncTransport; + path = ../Packages/TableProCore/Sources/TableProSyncTransport; sourceTree = ""; }; 5AA313332F7EA5B4008EBA97 /* Frameworks */ = { @@ -1641,6 +1651,7 @@ 5AD1F1B72FB4456900296783 /* FreeTDS.xcframework */, 5AD0CCDB2F0000000000F001 /* DuckDB.xcframework */, 5A87EEEB2F7F891F00D028D0 /* TableProSync */, + 5A87EEF02F7F891F00D028E2 /* TableProSyncTransport */, 5A87EEE42F7F88F200D028D0 /* TablePro */, 5AA313532F7EC188008EBA97 /* LibSSH2.xcframework */, 5AA313352F7EA5B4008EBA97 /* Hiredis.xcframework */, @@ -1738,6 +1749,7 @@ 5AB9F3EC2F7C1D03001F3337 /* TableProPluginKit */, 5AB9F3EE2F7C1D03001F3337 /* TableProQuery */, 5A87EEEC2F7F893000D028D0 /* TableProSync */, + 5A87EEEF2F7F893000D028E1 /* TableProSyncTransport */, 5A87EEED2F7F893000D028D1 /* TableProAnalytics */, 5AD1F1B52FB4455700296783 /* TableProMSSQLCore */, ); @@ -2255,6 +2267,11 @@ package = 5AB9F3E72F7C1D03001F3337 /* XCLocalSwiftPackageReference "../Packages/TableProCore" */; productName = TableProSync; }; + 5A87EEEF2F7F893000D028E1 /* TableProSyncTransport */ = { + isa = XCSwiftPackageProductDependency; + package = 5AB9F3E72F7C1D03001F3337 /* XCLocalSwiftPackageReference "../Packages/TableProCore" */; + productName = TableProSyncTransport; + }; 5A87EEED2F7F893000D028D1 /* TableProAnalytics */ = { isa = XCSwiftPackageProductDependency; package = 5AB9F3E72F7C1D03001F3337 /* XCLocalSwiftPackageReference "../Packages/TableProCore" */; diff --git a/TableProMobile/TableProMobile/Sync/IOSSyncCoordinator.swift b/TableProMobile/TableProMobile/Sync/IOSSyncCoordinator.swift index 6a831c1fe..6685f4cc1 100644 --- a/TableProMobile/TableProMobile/Sync/IOSSyncCoordinator.swift +++ b/TableProMobile/TableProMobile/Sync/IOSSyncCoordinator.swift @@ -4,6 +4,7 @@ import Observation import os import TableProModels import TableProSync +import TableProSyncTransport @MainActor @Observable final class IOSSyncCoordinator { @@ -14,9 +15,7 @@ final class IOSSyncCoordinator { private var engine: CloudKitSyncEngine? private let metadata = SyncMetadataStorage() - private var cachedRecords: [UUID: CKRecord] = [:] - private var cachedGroupRecords: [UUID: CKRecord] = [:] - private var cachedTagRecords: [UUID: CKRecord] = [:] + private let recordCache = SyncRecordCache() private func getEngine() -> CloudKitSyncEngine { if let engine { return engine } @@ -75,6 +74,10 @@ final class IOSSyncCoordinator { onGroupsChanged?(mergedGroups) onTagsChanged?(mergedTags) + if let newToken = remoteChanges.newToken { + metadata.saveToken(newToken) + } + metadata.lastSyncDate = Date() lastSyncDate = metadata.lastSyncDate status = .idle @@ -104,9 +107,7 @@ final class IOSSyncCoordinator { ) async { debounceTask?.cancel() metadata.saveToken(nil) - cachedRecords.removeAll() - cachedGroupRecords.removeAll() - cachedTagRecords.removeAll() + recordCache.removeAll() Self.logger.info("Sync token cleared; forcing full pull from iCloud") await sync( localConnections: localConnections, @@ -178,7 +179,8 @@ final class IOSSyncCoordinator { // Dirty connections let dirtyConnIDs = metadata.dirtyIDs(for: .connection) for connection in localConnections where dirtyConnIDs.contains(connection.id.uuidString) { - if let existing = cachedRecords[connection.id] { + let recordID = SyncRecordMapper.recordID(type: .connection, id: connection.id.uuidString, in: zoneID) + if let existing = recordCache.record(for: recordID) { SyncRecordMapper.updateRecord(existing, with: connection) allRecords.append(existing) } else { @@ -194,7 +196,8 @@ final class IOSSyncCoordinator { // Dirty groups let dirtyGroupIDs = metadata.dirtyIDs(for: .group) for group in localGroups where dirtyGroupIDs.contains(group.id.uuidString) { - if let existing = cachedGroupRecords[group.id] { + let recordID = SyncRecordMapper.recordID(type: .group, id: group.id.uuidString, in: zoneID) + if let existing = recordCache.record(for: recordID) { SyncRecordMapper.updateRecord(existing, with: group) allRecords.append(existing) } else { @@ -210,7 +213,8 @@ final class IOSSyncCoordinator { // Dirty tags let dirtyTagIDs = metadata.dirtyIDs(for: .tag) for tag in localTags where dirtyTagIDs.contains(tag.id.uuidString) { - if let existing = cachedTagRecords[tag.id] { + let recordID = SyncRecordMapper.recordID(type: .tag, id: tag.id.uuidString, in: zoneID) + if let existing = recordCache.record(for: recordID) { SyncRecordMapper.updateRecord(existing, with: tag) allRecords.append(existing) } else { @@ -225,13 +229,28 @@ final class IOSSyncCoordinator { guard !allRecords.isEmpty || !allDeletions.isEmpty else { return } - try await getEngine().push(records: allRecords, deletions: allDeletions) - metadata.clearDirty(type: .connection) - metadata.clearTombstones(type: .connection) - metadata.clearDirty(type: .group) - metadata.clearTombstones(type: .group) - metadata.clearDirty(type: .tag) - metadata.clearTombstones(type: .tag) + let outcome = try await getEngine().push(records: allRecords, deletions: allDeletions) + + recordCache.store(Array(outcome.savedRecords.values)) + recordCache.remove(Array(outcome.deletedRecordIDs)) + + for recordID in outcome.savedRecords.keys { + guard let parsed = SyncRecordMapper.parse(recordName: recordID.recordName) else { continue } + metadata.removeDirty(parsed.id, type: parsed.type) + } + + for recordID in outcome.deletedRecordIDs { + guard let parsed = SyncRecordMapper.parse(recordName: recordID.recordName) else { continue } + metadata.removeTombstone(parsed.id, type: parsed.type) + } + + guard outcome.hasFailures else { return } + + for (recordID, failure) in outcome.failures { + Self.logger.error("iCloud rejected \(recordID.recordName): \(failure.message)") + } + + throw SyncError.pushFailed(outcome.failures.values.first?.message ?? "") } // MARK: - Pull @@ -243,33 +262,31 @@ final class IOSSyncCoordinator { var deletedGroupIDs: Set = [] var changedTags: [ConnectionTag] = [] var deletedTagIDs: Set = [] + var newToken: CKServerChangeToken? } private func pull() async throws -> PullChanges { let token = metadata.loadToken() let result = try await getEngine().pull(since: token) - if let newToken = result.newToken { - metadata.saveToken(newToken) - } - var changes = PullChanges() + changes.newToken = result.newToken + + recordCache.store(result.changedRecords) + recordCache.remove(result.deletedRecordIDs) for record in result.changedRecords { switch record.recordType { case SyncRecordType.connection.rawValue: if let connection = SyncRecordMapper.toConnection(record) { - cachedRecords[connection.id] = record changes.changedConnections.append(connection) } case SyncRecordType.group.rawValue: if let group = SyncRecordMapper.toGroup(record) { - cachedGroupRecords[group.id] = record changes.changedGroups.append(group) } case SyncRecordType.tag.rawValue: if let tag = SyncRecordMapper.toTag(record) { - cachedTagRecords[tag.id] = record changes.changedTags.append(tag) } default: diff --git a/TableProMobile/TableProMobile/Views/ConnectionListView.swift b/TableProMobile/TableProMobile/Views/ConnectionListView.swift index b0703ec3e..6b40e11c3 100644 --- a/TableProMobile/TableProMobile/Views/ConnectionListView.swift +++ b/TableProMobile/TableProMobile/Views/ConnectionListView.swift @@ -1,7 +1,7 @@ import SwiftUI import TableProImport import TableProModels -import TableProSync +import TableProSyncTransport import UniformTypeIdentifiers struct ConnectionListView: View { diff --git a/TableProMobile/TableProMobile/Views/SettingsView.swift b/TableProMobile/TableProMobile/Views/SettingsView.swift index d214f7f8a..e0ef2d176 100644 --- a/TableProMobile/TableProMobile/Views/SettingsView.swift +++ b/TableProMobile/TableProMobile/Views/SettingsView.swift @@ -1,6 +1,6 @@ import SwiftUI import TableProModels -import TableProSync +import TableProSyncTransport struct SettingsView: View { @Environment(AppState.self) private var appState diff --git a/TableProTests/Core/Sync/ConflictResolverTests.swift b/TableProTests/Core/Sync/ConflictResolverTests.swift deleted file mode 100644 index 7a2e93b8d..000000000 --- a/TableProTests/Core/Sync/ConflictResolverTests.swift +++ /dev/null @@ -1,89 +0,0 @@ -// -// ConflictResolverTests.swift -// TableProTests -// - -import CloudKit -import Foundation -import Testing - -@testable import TablePro - -@Suite("ConflictResolver", .serialized) -@MainActor -struct ConflictResolverTests { - private let resolver = ConflictResolver.shared - - init() { - while resolver.hasConflicts { - _ = resolver.resolveCurrentConflict(keepLocal: false) - } - } - - private func makeConflict(local: [String: String], server: [String: String]) -> SyncConflict { - let localRecord = CKRecord(recordType: "Connection") - for (key, value) in local { - localRecord[key] = value as CKRecordValue - } - let serverRecord = CKRecord(recordType: "Connection") - for (key, value) in server { - serverRecord[key] = value as CKRecordValue - } - return SyncConflict( - recordType: .connection, - entityName: "users", - localRecord: localRecord, - serverRecord: serverRecord, - localModifiedAt: Date(timeIntervalSince1970: 100), - serverModifiedAt: Date(timeIntervalSince1970: 200) - ) - } - - @Test("addConflict queues the conflict as current") - func addConflictQueues() { - #expect(!resolver.hasConflicts) - resolver.addConflict(makeConflict(local: ["name": "L"], server: ["name": "S"])) - - #expect(resolver.hasConflicts) - #expect(resolver.currentConflict?.entityName == "users") - - _ = resolver.resolveCurrentConflict(keepLocal: false) - } - - @Test("Keeping the server version discards the conflict and returns nil") - func keepServerReturnsNil() { - resolver.addConflict(makeConflict(local: ["name": "L"], server: ["name": "S"])) - - let result = resolver.resolveCurrentConflict(keepLocal: false) - - #expect(result == nil) - #expect(!resolver.hasConflicts) - } - - @Test("Keeping local copies local field values onto the server record") - func keepLocalCopiesFieldsOntoServerRecord() { - resolver.addConflict(makeConflict(local: ["name": "Local"], server: ["name": "Server"])) - - let resolved = resolver.resolveCurrentConflict(keepLocal: true) - - #expect(resolved?["name"] as? String == "Local") - #expect(!resolver.hasConflicts) - } - - @Test("Conflicts are resolved in FIFO order") - func conflictsResolveInFifoOrder() { - resolver.addConflict(makeConflict(local: ["name": "first"], server: ["name": "s1"])) - resolver.addConflict(makeConflict(local: ["name": "second"], server: ["name": "s2"])) - - #expect(resolver.currentConflict?.localRecord["name"] as? String == "first") - _ = resolver.resolveCurrentConflict(keepLocal: false) - #expect(resolver.currentConflict?.localRecord["name"] as? String == "second") - _ = resolver.resolveCurrentConflict(keepLocal: false) - #expect(!resolver.hasConflicts) - } - - @Test("Resolving with no pending conflicts returns nil") - func resolveWithNoConflictsReturnsNil() { - #expect(resolver.resolveCurrentConflict(keepLocal: false) == nil) - } -} diff --git a/TableProTests/Core/Sync/SyncRecordMapperConnectionTests.swift b/TableProTests/Core/Sync/SyncRecordMapperConnectionTests.swift new file mode 100644 index 000000000..848d71a15 --- /dev/null +++ b/TableProTests/Core/Sync/SyncRecordMapperConnectionTests.swift @@ -0,0 +1,129 @@ +import CloudKit +import Foundation +@testable import TablePro +import TableProSyncTransport +import Testing + +@Suite("SyncRecordMapper connection wire schema") +struct SyncRecordMapperConnectionTests { + private let zoneID = CKRecordZone.ID(zoneName: "TestZone", ownerName: CKCurrentUserDefaultName) + + private func makeFullyPopulatedConnection() -> DatabaseConnection { + var connection = DatabaseConnection(name: "Production") + connection.host = "db.example.com" + connection.port = 5432 + connection.database = "app" + connection.username = "admin" + connection.type = .postgresql + connection.color = .blue + connection.tagIds = [UUID(), UUID()] + connection.groupId = UUID() + connection.sshProfileId = UUID() + connection.safeModeLevel = .alertFull + connection.aiPolicy = .askEachTime + connection.aiRules = "Never drop tables" + connection.aiAlwaysAllowedTools = ["listTables"] + connection.redisDatabase = 3 + connection.startupCommands = "SET search_path TO public" + connection.sortOrder = 7 + connection.isFavorite = true + connection.additionalFields = ["schema": "public"] + return connection + } + + @Test("An unverified field is never written to a record") + func unverifiedFieldsAreNeverWritten() { + let record = SyncRecordMapper.toCKRecord(makeFullyPopulatedConnection(), in: zoneID) + let unverified = Set(ConnectionSyncField.allCases.filter { !$0.isWritable }.map(\.key)) + + #expect(Set(record.allKeys()).isDisjoint(with: unverified)) + } + + @Test("Every written key is declared in the shared schema") + func everyWrittenKeyIsDeclared() { + let record = SyncRecordMapper.toCKRecord(makeFullyPopulatedConnection(), in: zoneID) + + #expect(Set(record.allKeys()).isSubset(of: ConnectionSyncField.declaredKeys)) + } + + @Test("isFavorite stays off the wire until the production schema is verified") + func favoriteIsNotWritten() { + var connection = DatabaseConnection(name: "Local") + connection.isFavorite = true + + let record = SyncRecordMapper.toCKRecord(connection, in: zoneID) + + #expect(record["isFavorite"] == nil) + } + + @Test("A verified field still reaches the wire") + func verifiedFieldsAreWritten() { + let record = SyncRecordMapper.toCKRecord(makeFullyPopulatedConnection(), in: zoneID) + + #expect(record["name"] as? String == "Production") + #expect(record["host"] as? String == "db.example.com") + #expect(record["port"] as? Int64 == 5432) + #expect(record["startupCommands"] as? String == "SET search_path TO public") + } + + @Test("A fully populated connection round-trips through the wire") + func roundTripPreservesVerifiedFields() throws { + let connection = makeFullyPopulatedConnection() + let record = SyncRecordMapper.toCKRecord(connection, in: zoneID) + + let decoded = try SyncRecordMapper.toConnection(record) + + #expect(decoded.id == connection.id) + #expect(decoded.name == connection.name) + #expect(decoded.host == connection.host) + #expect(decoded.port == connection.port) + #expect(decoded.database == connection.database) + #expect(decoded.username == connection.username) + #expect(decoded.type == connection.type) + #expect(decoded.color == connection.color) + #expect(decoded.groupId == connection.groupId) + #expect(decoded.sshProfileId == connection.sshProfileId) + #expect(decoded.safeModeLevel == connection.safeModeLevel) + #expect(decoded.redisDatabase == connection.redisDatabase) + #expect(decoded.startupCommands == connection.startupCommands) + #expect(decoded.sortOrder == connection.sortOrder) + } + + @Test( + "iOS safe mode wire values map to the nearest macOS level", + arguments: [ + ("off", SafeModeLevel.silent), + ("confirmWrites", SafeModeLevel.alert), + ("readOnly", SafeModeLevel.readOnly) + ] + ) + func decodesIOSWireValues(raw: String, expected: SafeModeLevel) { + #expect(SyncRecordMapper.safeModeLevel(fromWire: raw, isReadOnly: false) == expected) + } + + @Test("An unrecognised safe mode value requires confirmation instead of failing open") + func unknownSafeModeFailsClosed() { + #expect(SyncRecordMapper.safeModeLevel(fromWire: "someFutureLevel", isReadOnly: false) == .alert) + #expect(SyncRecordMapper.safeModeLevel(fromWire: "someFutureLevel", isReadOnly: true) == .readOnly) + } + + @Test("A record without a safe mode level honours the read-only flag") + func missingSafeModeHonoursReadOnly() { + #expect(SyncRecordMapper.safeModeLevel(fromWire: nil, isReadOnly: true) == .readOnly) + #expect(SyncRecordMapper.safeModeLevel(fromWire: nil, isReadOnly: false) == .silent) + } + + @Test("An iOS read-only connection stays read-only on macOS") + func readOnlyConnectionFromIOSKeepsProtection() throws { + let recordID = SyncRecordMapper.recordID(type: .connection, id: UUID().uuidString, in: zoneID) + let record = CKRecord(recordType: SyncRecordType.connection.rawValue, recordID: recordID) + record["connectionId"] = UUID().uuidString as CKRecordValue + record["name"] = "From iPhone" as CKRecordValue + record["type"] = "PostgreSQL" as CKRecordValue + record["isReadOnly"] = Int64(1) as CKRecordValue + + let decoded = try SyncRecordMapper.toConnection(record) + + #expect(decoded.safeModeLevel == .readOnly) + } +} diff --git a/TableProTests/Core/Sync/SyncRecordMapperTagTests.swift b/TableProTests/Core/Sync/SyncRecordMapperTagTests.swift index d50aca924..06fe969bd 100644 --- a/TableProTests/Core/Sync/SyncRecordMapperTagTests.swift +++ b/TableProTests/Core/Sync/SyncRecordMapperTagTests.swift @@ -7,8 +7,8 @@ import Testing struct SyncRecordMapperTagTests { private let zoneID = CKRecordZone.ID(zoneName: "TestZone", ownerName: CKCurrentUserDefaultName) - @Test("Writes both tagIds array and legacy tagId") - func writesBothFields() { + @Test("Writes legacy tagId only while tagIds is unverified in the production schema") + func writesLegacyTagIdOnly() { let a = UUID() let b = UUID() var connection = DatabaseConnection(name: "Local") @@ -16,17 +16,18 @@ struct SyncRecordMapperTagTests { let record = SyncRecordMapper.toCKRecord(connection, in: zoneID) - #expect(record["tagIds"] as? [String] == [a.uuidString, b.uuidString]) #expect(record["tagId"] as? String == a.uuidString) + #expect(record["tagIds"] == nil) } - @Test("Prefers tagIds array when reading a record") + @Test("Prefers tagIds array when a record carries one") func prefersTagIds() throws { let a = UUID() let b = UUID() var connection = DatabaseConnection(name: "Local") connection.tagIds = [a, b] let record = SyncRecordMapper.toCKRecord(connection, in: zoneID) + record["tagIds"] = [a.uuidString, b.uuidString] as CKRecordValue let decoded = try SyncRecordMapper.toConnection(record) #expect(decoded.tagIds == [a, b]) diff --git a/TableProTests/ViewModels/WelcomeViewModelTests.swift b/TableProTests/ViewModels/WelcomeViewModelTests.swift index dc91a3474..55e18f60c 100644 --- a/TableProTests/ViewModels/WelcomeViewModelTests.swift +++ b/TableProTests/ViewModels/WelcomeViewModelTests.swift @@ -89,7 +89,6 @@ final class WelcomeViewModelTests: XCTestCase { tagStorage: live.tagStorage, sshProfileStorage: live.sshProfileStorage, licenseManager: live.licenseManager, - conflictResolver: live.conflictResolver, syncMetadataStorage: live.syncMetadataStorage, favoritesExpansionState: live.favoritesExpansionState, linkedFolderWatcher: live.linkedFolderWatcher, diff --git a/docs/features/icloud-sync.mdx b/docs/features/icloud-sync.mdx index 254785d3d..32ab0a0be 100644 --- a/docs/features/icloud-sync.mdx +++ b/docs/features/icloud-sync.mdx @@ -1,9 +1,9 @@ --- title: iCloud Sync -description: Sync connections, table favorites, settings, and SSH profiles across Macs via iCloud +description: Sync connections, table favorites, settings, and SSH profiles across Macs and iOS via iCloud --- -TablePro syncs your connections, groups, table favorites, saved queries, settings, and SSH profiles across all your Macs via CloudKit. iCloud Sync requires an active license (Starter or Team, see [Licensing](/features/licensing)). +TablePro syncs your connections, groups, table favorites, saved queries, settings, and SSH profiles across all your Macs via CloudKit. The iPhone and iPad app shares the same iCloud data, but syncs connections, groups, and tags only. iCloud Sync requires an active license (Starter or Team, see [Licensing](/features/licensing)). ## What syncs (and what doesn't) @@ -45,7 +45,7 @@ Local-only connections show an `icloud.slash` icon in the sidebar. The flag is p TablePro auto-syncs on app launch, when you switch back to it, and 2 seconds after you modify synced data. -When the same record changes on two Macs, you choose to keep the local or remote version. Conflicts are per-record, not per-category. +When the same connection changes on two devices, TablePro merges them field by field: a name changed on one device and a port changed on the other both survive. If the same field changed in both places, the device that syncs last wins. When a license expires, sync stops but local data remains. Re-activate to resume. diff --git a/scripts/export-cloudkit-schema.sh b/scripts/export-cloudkit-schema.sh new file mode 100755 index 000000000..5a93eda3a --- /dev/null +++ b/scripts/export-cloudkit-schema.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Refresh the checked-in snapshot of the Production CloudKit schema. +# +# CloudKit only creates record fields automatically in the Development +# environment. TablePro pins both apps to Production, so a field added to a +# record type in code never reaches the server on its own: saving a record +# that carries an undeclared field makes CloudKit reject that record. +# +# ConnectionSyncField refuses to write any field that is not verified against +# this snapshot, and ProductionSchemaParityTests fails if the two disagree. +# After deploying a schema change in CloudKit Console, run this script and +# commit the result, then mark the field verified in ConnectionSyncSchema.swift. +# +# Requires a CloudKit management token: +# xcrun cktool save-token --type management +# +# Usage: +# scripts/export-cloudkit-schema.sh + +TEAM_ID="D7HJ5TFYCU" +CONTAINER_ID="iCloud.com.TablePro" +ENVIRONMENT="production" + +REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +OUTPUT_FILE="${REPO_ROOT}/CloudKit/production-schema.ckdb" + +mkdir -p "$(dirname "${OUTPUT_FILE}")" + +echo "Exporting ${ENVIRONMENT} schema for ${CONTAINER_ID}..." +xcrun cktool export-schema \ + --team-id "${TEAM_ID}" \ + --container-id "${CONTAINER_ID}" \ + --environment "${ENVIRONMENT}" \ + --output-file "${OUTPUT_FILE}" + +echo "Wrote ${OUTPUT_FILE}" +echo +echo "Review the diff, then commit it:" +echo " git diff -- CloudKit/production-schema.ckdb"