Channels baseline: email, Signal, Matrix, Mattermost, Teams, Google Chat, SMS + media and voice - #4
Merged
Merged
Conversation
Channels can now carry media. Inbound gains Attachments (downloaded into a content-addressed spool at ~/.config/memcode/media by each adapter); Outbound gains VoicePath for later synthesized replies. The spool is the trust boundary: everything downstream — the durable inbox row, the job context envelope, the spawned child — addresses media by spool ID (bare <sha256>.<ext> filename), and the child resolves IDs strictly inside the spool, so a corrupted context file can't point a job at arbitrary files. Images/PDFs reach the engine through the existing input.Bundle attachment path via a new Session.SetTaskAttachments seam; audio never does (it will be transcribed gateway-side). Adapters: Telegram downloads photos/voice/audio/documents (getFile) and accepts captioned media; WhatsApp downloads image/audio/document media, uses captions as task text, and now routes outbound through the shared chunker (it was the one adapter bypassing it); Discord downloads message attachments; Slack lets file_share messages through as text (this SDK version doesn't surface the file list — noted limitation). Spool prune rides the inbox retention window; per-attachment cap 25 MiB.
A dedicated mailbox the agent answers — the biggest capability gap vs both competitors (OpenClaw has no email at all; Hermes ships a plaintext poller). IMAP-SSL poll (default 15s, email.poll to tune) + SMTP-STARTTLS replies with real threading (In-Reply-To/References, single Re:). Durable dedup key is <mailbox>/<UIDVALIDITY>/<UID> — the provider-side ack identity, robust against malformed/duplicated Message-IDs (those serve threading only). BODY.PEEK on fetch and \Seen set ONLY after the durable record, so a crash re-fetches instead of losing mail. Filters: Auto-Submitted, noreply/bounce senders, self (loop prevention). Attachments (images/PDFs) land in the media spool and reach the engine — where this beats Hermes' plaintext-only design. Allow-list + pairing work unchanged: an unknown sender's pairing code goes out as an email reply. Dep: emersion/go-imap/v2, guard-homed to internal/channels/email.
Inbound voice notes (Telegram voice, WhatsApp audio, email audio attachments, Discord uploads) are transcribed gateway-side and become the task text; audio never reaches the coding engine. The STT seam lives in the provider homes so SDK containment holds: openai (audio transcriptions endpoint, gpt-4o-mini-transcribe with whisper-1 fallback) and gemini (audio-in generate on flash). Provider picked by present credentials; with neither key a voice-only message gets an honest 'not configured' reply instead of silence, and mixed text+audio proceeds on the text. Transcription runs after the durable record and before the spawn, so a crash re-runs it rather than losing the note. Replies stay text (TTS is the later, opt-in phase).
The community-signal cluster (highest genuine demand after voice on both competitors' trackers) and the self-host credibility story. Signal: adapter to a signal-cli daemon in native HTTP mode (SSE events in, JSON-RPC send out) — the same companion every gateway in this space requires; we spawn nothing. Linked-device model, dedicated-number advice in docs. Principal = E.164 (uuid fallback); dedup key = sender:timestamp (Signal's own message identity); groups via groupId conversations with mention/quote-reply gating; attachments come from the daemon's store into the media spool; a synthesized voice reply rides send as an attachment. Matrix: raw client-server API, stdlib only, NO E2EE v1 (plain rooms — stated loudly). /sync long-poll with the since-token persisted in a new string-cursor state table, advanced only after every Deliver in the batch is durably recorded; first sync records the token without replaying history. Sends as m.notice (bot convention, loop prevention); m.mentions for group gating; authenticated media download; voice replies upload via the media repo with silent text fallback. Mattermost: WebSocket event stream + REST v4, bot token; posted events (double-encoded post JSON) with DM/type-D detection and word-boundary mention matching; file downloads via files API; chunked posts. gorilla/websocket promoted to a direct dep, guard-homed to mattermost.
The tactical lane — alerts, approvals, short tasks from a phone. Inbound webhook with HMAC-SHA1 signature verification over the exact configured public URL (channels.sms.webhook_url; fail closed without it), empty TwiML ack only after the durable record, 503 so Twilio retries when the record fails. MMS media downloads at receipt (URLs expire fast) with basic auth into the spool. Outbound via the Messages API, chunked at 1500. Principal/conversation = the sender's E.164 number. Also lands the env keys for the Teams and Google Chat adapters that follow.
The workplace bets, both riding the existing webhook mux.
Teams (Bot Framework): inbound activities validated against the Bot
Framework JWKS (RS256, issuer/audience/expiry; cached keys, bounded
refresh on unknown kid) — 401 before anything is parsed. Replies go to
the activity's serviceUrl with a cached Azure AD client-credentials
token; the conversation string encodes id|serviceUrl so a durable reply
survives a restart. DM via conversationType, mentions via entities,
<at> tags stripped structurally. 200 after durable record, 503 retries.
Google Chat: inbound events carry a Google-signed JWT verified against
Google's JWKS with a stdlib RSA verify (aud = project number,
channels.googlechat.audience). Replies via spaces.messages.create as the
service account (GOOGLE_CHAT_SA_KEY). argumentText preferred (mention-
stripped), DM/space detection, BOT senders skipped, async replies with
{} synchronous ack. Deps guard-homed: golang-jwt→msteams,
x/oauth2→googlechat.
channels.<name>.voice_replies: off (default — voice output costs money and speaks replies aloud, so it's a deliberate opt-in) | in_kind (voice note in → voice reply out) | always. The Speak seam lives in the openai provider home (gpt-4o-mini-tts, opus output — no ffmpeg anywhere); no key → silently text-only. The spoken rendition drops code blocks and caps at ~600 runes; the full text reply is ALWAYS sent alongside, and any synthesis/upload failure degrades to text — a reply is never lost to TTS. Carriers: Telegram sendVoice (native voice bubble), WhatsApp media upload + audio message, Discord file upload, Signal attachment, Matrix m.audio (already in their adapters).
One wizard case per new channel (email, signal, matrix, mattermost, msteams, googlechat, sms), merging into existing channel blocks as always. README: full channel table with transports and env keys, the webhook mount map, email dedup note, and a Media and voice section.
…t TTS, uuid Signal principals
P1: a queued task whose snapshotted project was disabled or deleted
before execution is now REFUSED with a durable reply — ResolveProject
failure no longer falls back to the gateway default root, closing the
contradiction with the 'never helpfully run elsewhere' claim. All
refusal paths share one helper; regression test added.
P2 (TTS): voice replies are synthesized exactly ONCE, at job completion,
and the spool ID is persisted on the replied row — a delivery retry or a
restart re-sends the same file instead of re-billing TTS ('always') or
losing the in_kind decision (attachments aren't loaded by the replay).
P2 (Signal): the principal is now the account UUID — Signal's stable
identity — with the phone number only as fallback; numbers change hands.
Pairing keeps uuids painless to allow-list.
P2/P3 (email): the From-address identity caveat is now stated in the
package doc and both docs (allow-list strength depends on the provider's
SPF/DKIM/DMARC filtering; dedicated mainstream account advised);
explicit Authentication-Results enforcement is a tracked follow-up.
Contributor
Author
|
Review fixes pushed (e08405d):
|
- internal/webjwt is now the ONE verifier for platform-signed inbound webhook tokens: RS256 via golang-jwt against a JWKS (direct URL or resolved through OpenID metadata), iss/aud/exp enforced, cached keys with one bounded refresh per unknown kid. Teams and Google Chat both use it — the hand-rolled Google Chat verifier is gone, and two divergent JWT implementations can't drift apart. Both adapters' JWT accept/reject test matrices now exercise the shared code from both configurations (metadata path and direct-JWKS path). - Guard map actually gains the new deps this time (the previous edit silently missed its anchor): golang-jwt→webjwt, gorilla/websocket→mattermost, x/oauth2→googlechat. - Email: attachment spooling uses bytes.NewReader (no 25 MiB string copy); reply threading headers and To are CRLF-stripped as defense in depth on top of mail.ReadMessage's parsing (injection test added). - SMS: comment documents that duplicate form keys would fail closed (signature mismatch), never bypass.
Contributor
Author
|
kimi review addressed (e21b796):
6 (Matrix map ordering) confirmed benign as noted; no change. |
…ening) Adversarial sweep cross-checked against Hermes/OpenClaw CVE history. No auth-bypass or RCE class reproduced (crypto core, stable-id principals, durable dedup, spool traversal all verified sound). Fixes: HIGH — media-download SSRF + credential leak. New channels.SafeHTTPClient blocks any fetch that resolves to loopback/private/link-local/CGNAT/ metadata (checked at dial on the resolved IP, so DNS names and redirects are caught too). Every message-derived download (Teams contentUrl, Google Chat downloadUri, WhatsApp media, Twilio MMS) routes through it. Teams additionally: the Azure connector bearer is attached ONLY to first-party Microsoft hosts, and an outbound reply is refused unless the serviceUrl is a Bot Framework host over https — so a forged serviceUrl/ contentUrl can't redirect the agent's output or harvest the token. This is exactly the class OpenClaw shipped dangerouslyAllowPrivateNetwork for. MED-HIGH — webjwt unknown-kid JWKS refetch is now throttled to one fetch per 30s (the package doc claimed this protection but the code lacked it); an unauthenticated forged-kid flood can no longer amplify into JWKS hammering. MED — email: cap parts (50), aggregate decoded bytes (50 MiB), and the raw IMAP fetch (60 MiB) so a crafted multipart can't exhaust memory; outbound replies now carry Auto-Submitted: auto-replied (loop prevention, matching the inbound filter). Signal: a learned number->uuid map keeps the dedup key and principal from flipping between uuid and number for the same account across deliveries (double-run guard). LOW — dispatcher retires idle per-conversation workers (goroutine/map leak over a long-lived daemon talking to many rooms); Telegram bot_command mention now matches @botusername on a token boundary, not a substring.
…ating Picking 'No, and tell me what to do differently' and typing feedback set Interrupt, which stopped the whole turn — the model never saw the typed feedback, so the turn just terminated with 'stopped this turn at your request'. That conflated two things: skipping the rejected action's sibling tool calls (right) and ending the turn (wrong for a redirect). Split them: a new ApprovalDecision.Redirect (and turn.redirected) denies the action, skips its siblings this batch, but lets the loop CONTINUE so the next model call reads the feedback (framed as 'The user declined that action and said: …') and responds — says something, re-plans, or asks. Esc / 'No, stop' still hard-interrupt. Regression test asserts redirect denies + marks redirected without interrupting.
- Image decompression-bomb guard (Medium): Downscale now reads only the header (image.DecodeConfig) and refuses an image over 50MP before the full decode, so a crafted PNG declaring huge dimensions can't balloon to gigabytes of RGBA and OOM the job process. Regression test with a dimension-forged PNG. - Webhook http.Server gets ReadHeaderTimeout/ReadTimeout/IdleTimeout (slowloris backstop). - waitForJob backstops at 30m so a hung agent child can't block its conversation's serial queue forever. - Media spool refreshes mtime on a dedup hit, so a task pending across the 30-day prune window doesn't lose its attachment. - Slack MessageID is now channel-qualified (channel:ts); a bare ts is unique only within a channel, so this removes a cross-channel INSERT-OR-IGNORE collision (fail-closed either way). - Pending/PendingReplies scans are LIMITed so a backlog flood can't turn each 2s worker tick into a full-table load. Residual noted (accepted): HostAllowed trusts any first-party suffix, so a content URL on an attacker-controlled *.googleusercontent.com blob still receives the SA bearer — low risk, the SSRF dial-guard still blocks any internal target.
Contributor
Author
|
Third-round audit (kimi, independent) addressed — commit 521f1b7. CI green. Fixed:
Accepted/noted: HostAllowed trusts any first-party suffix (a Audit verdict across all three reviewers: no auth bypass, no path traversal, no cross-conversation leak, no secret exposure. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One coherent channels baseline for the gateway: media, email, voice, and six new surfaces — every channel shipping only if its v1 meets the full gateway contract. No placeholder adapters.
What's in here (one commit per phase)
Inbound.Attachments+ a content-addressed spool (~/.config/memcode/media). The spool is the trust boundary: the durable inbox, the job context, and the spawned child all address media by spool ID (bare<sha256>.<ext>), and the child resolves IDs strictly inside the spool. Images/PDFs reach the engine as native blocks; WhatsApp outbound now uses the shared chunker.<mailbox>/<UIDVALIDITY>/<UID>(Message-ID is threading only),\Seenset only after the durable record, real reply threading, auto-mail filters, attachments in.gpt-4o-mini-transcribe→whisper-1fallback, or Gemini; picked by present key) into task text; audio never reaches the engine; honest "not configured" reply when no key./syncwith a durable since-token, no E2EE v1 (stated loudly). Mattermost WS + REST v4.voice_replies: off|in_kind|always, default off. Opus straight fromgpt-4o-mini-tts(no ffmpeg anywhere); code blocks never spoken; full text always sent; synthesis failure degrades to text.Why these channels (three separate signals)
Contract compliance
Non-goals (deliberate)
iMessage (macOS/private-API-fragile), WhatsApp-Web/Baileys (ban risk — official Cloud API stays), realtime voice calls (separate architecture pass — findings documented in the plan), Matrix E2EE, HTML email, regional apps (Feishu/WeChat/QQ/LINE), IRC, WebChat, Rocket.Chat.
Known limitations called out in code/docs: Slack file downloads (SDK doesn't surface the file list; file_share messages now at least deliver their text), Discord/Mattermost/Signal inbound is best-effort (no provider-side replay on push transports), email threading cache is in-memory (post-restart replies may start a new thread).
New deps, each guard-homed to exactly one package: emersion/go-imap→email, golang-jwt→msteams, x/oauth2→googlechat, gorilla/websocket→mattermost (promoted from indirect).