Skip to content

fix(metadata): accept raw-DEFLATE appData and surface decode causes#91

Open
insipx wants to merge 1 commit into
mainfrom
insipx/appdata-dual-reader
Open

fix(metadata): accept raw-DEFLATE appData and surface decode causes#91
insipx wants to merge 1 commit into
mainfrom
insipx/appdata-dual-reader

Conversation

@insipx

@insipx insipx commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

convos-cli cannot read appData written by convos-ios: iOS compresses with Apple's COMPRESSION_ZLIB, which emits raw DEFLATE (no zlib header), while decompressIfNeeded uses Node inflateSync (zlib-wrapped only). Any >100-byte iOS-written appData hits Z_DATA_ERROR, which the lenient parser swallows into empty metadata — the root cause of herald's GET /metadata 502s (CON-627) and a contributor to iOS-side profile loss ("Somebody" names, since the reverse direction is equally broken).

  • decompressIfNeeded: try inflateSync, fall back to inflateRawSync — both formats decode; when both fail the zlib error (our canonical format) is surfaced. Size guard unchanged.
  • Strict decode extracted (decodeAppData) so parseAppDataForWrite attaches the ORIGINAL decode error as cause on its guard throw (undecodable case), while a decoded-but-tagless blob throws without cause — the two are now distinguishable in telemetry. Throw message text unchanged.
  • Strict base64url alphabet check: Buffer.from(str, "base64url") silently drops invalid chars, so garbage previously masqueraded as decoded-but-empty metadata; it now fails the decode with a real error. Lenient parseAppData behavior is unchanged (still returns empty on any failure).
  • Writer intentionally untouched (still zlib): phase 1 of a two-phase rollout. Phase 2 flips the writer to raw deflate once dual readers are deployed fleet-wide (herald, hermes image pins, iOS reader).

Verification

  • 35 metadata tests pass (5 new: iOS-format round-trip, garbage rejection, cause-threading ×3); full suite 497 passed.
  • Golden-validated against the real CON-627 blob (dev group 990db8c5…): previously parsed as empty → now decodes to its actual contents (tag, 32-byte imageEncryptionKey, 4 profiles) and parseAppDataForWrite succeeds.
  • pnpm typecheck has one pre-existing error on main (send-remote-attachment.ts:97, unrelated file).

Follow-ups (not this PR)

  • Phase 2: writer → deflateRawSync after fleet-wide reader deployment.
  • convos-ios reader should accept zlib-wrapped blobs too (historic CLI-written appData exists in groups).
  • Write paths in commands/conversation/lock.ts, invite.ts, agent/serve.ts lenient-parse before serializeAppData — same clobber class the strict guard protects against; flagged during adversarial review.

🤖 Generated with Claude Code


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

Note

Fix parseAppData and parseAppDataForWrite to accept raw-DEFLATE compressed appData

  • Updates decompressIfNeeded in metadata.ts to first attempt zlib-wrapped inflate, then fall back to raw DEFLATE (inflateRawSync) to support iOS-produced compressed payloads.
  • Introduces a decodeAppData internal function that centralizes strict decoding with explicit base64url validation, replacing silent acceptance of invalid inputs that previously decoded to empty bytes.
  • Updates parseAppDataForWrite to use strict decoding and attach the underlying decode error as cause when it throws, improving debuggability.
  • Behavioral Change: non-base64url strings now trigger a strict failure in parseAppDataForWrite (with cause), while parseAppData continues to return empty metadata leniently.

Macroscope summarized 1e07c13.

@macroscopeapp

macroscopeapp Bot commented Jul 2, 2026

Copy link
Copy Markdown

Approvability

Verdict: Needs human review

This PR adds a new decompression fallback path (raw DEFLATE for iOS compatibility) and new input validation to the appData parsing logic. While well-tested and maintaining backward compatibility on the lenient path, these changes alter runtime data processing behavior and warrant human review.

You can customize Macroscope's approvability policy. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant