Skip to content

Add a federated peer picker to Image Gen and Video Gen - #4819

Merged
atomantic merged 2 commits into
mainfrom
claim/issue-4348
Aug 22, 2026
Merged

Add a federated peer picker to Image Gen and Video Gen#4819
atomantic merged 2 commits into
mainfrom
claim/issue-4348

Conversation

@atomantic

Copy link
Copy Markdown
Owner

Summary

Adds the Generation target picker to Image Gen and Video Gen, so a render can be routed to a federated PortOS peer from the UI. Both generate routes have accepted mediaProviderPeerId / mediaProviderEngine since #4676 — this was the last non-ADR-gated acceptance-criteria bullet on #4348, and until now a user with a GPU peer had to reach for the API to use it for a picture or a clip.

  • One shared RemoteMediaTargetPicker over a new useFederatedMediaTarget hook backs all three surfaces. The hook owns the peer list, the per-kind allowlist ∩ advertised-capabilities intersection, and the readiness verdict — Music Studio's private copy of that logic is deleted in favour of it, so a fourth surface cannot invent a fourth reading of the same peer.
  • Only the intersection is offerable, because that is exactly what assertFederatedMediaProviderSelection admits. The two ways it can come up empty carry different remedies (nothing allowlisted here vs. the peer no longer advertising it) and are now said out loud instead of collapsing into one "unavailable".
  • Wire v1 is text-to-image / text-to-video only. A form holding an init image, reference images, LoRAs, keyframes, a clip to extend, an audio track, chained chunks, or a non-text mode blocks Generate and names what has to go — rather than quietly dropping conditioning the user set up, which is the one thing the server refuses to do (MEDIA_PROVIDER_INPUT_UNSUPPORTED). Nothing is cleared for them, so switching the target back to This instance renders what they built.
  • Readiness is re-derived at click time, not read off the last poll: a capacity window expires on the clock, so an enabled button can already be pointing at a lapsed peer. blockingReason() runs the server's gates in the server's order, and every branch fails closed — an unparseable freshUntil, a missing snapshot, or an absent queue.accepting all read as blocked, never as available.
  • Picking a peer swaps the local model dropdown (and, on Image Gen, the quantization knob) for the peer's advertised models; resolution / steps / guidance / seed stay, since those do cross the wire.

No server changes, no wire changes, no new payload fields — an older peer behaves exactly as before, and omitting a peer leaves every local and cloud path byte-for-byte unchanged.

Test plan

  • cd client && npm test — 739 files / 9403 tests green; npm run lint clean.
  • New client/src/pages/ImageGen.federatedTarget.test.jsx and VideoGen.federatedTarget.test.jsx drive the real pages end to end: the submitted body carries the peer/engine/model triple and none of the local-dispatch fields (mode, quantize, cleaners, LoRAs, source frames, tiling, chunks…); a lapsed capacity window disables Generate and explains why; a non-text video mode blocks with the reason rather than submitting.
  • New RemoteMediaTargetPicker.test.jsx covers the hook + picker together: allowlist ∩ capabilities (including the engine/model pair, not the id alone), the two distinct empty-list remedies, stale-window refusal in both the caption and the dropdown suffix, a switched-off peer holding a fresh snapshot, and verify() re-deriving after the clock moves past the window.
  • federatedMediaReadiness.test.js gains unit coverage for federatedMediaModelsForPeer, including the shared empty-array identity callers memoize on.
  • One existing MusicGenPanel fixture was made genuinely usable (it advertised no capabilities while asserting the "usable peer" branch) — the panel now explains that gap instead of printing a queue reading beside a disabled button.
  • Local review pass: antigravity gemini-3.7-flash / effort medium against the branch diff — no material findings.

Remaining

Still open on #4348, all deliberately out of this slice:

Refs #4348

Both generate routes have accepted `mediaProviderPeerId` since #4676, but no
human-facing surface could set it — interactive remote selection existed only
in the Music Studio panel, so a user with a GPU peer had to reach for the API
to use it for a picture or a clip.

A shared `RemoteMediaTargetPicker` over a new `useFederatedMediaTarget` hook now
puts the same "Generation target" control on all three pages. The hook is where
the peer list, the per-kind allowlist ∩ advertised-capabilities intersection,
and the readiness verdict live, so a fourth surface cannot invent a fourth
reading of the same peer — Music Studio's own copy of that logic is deleted in
favour of it.

Two things the picker refuses to paper over:

- Only the intersection of the local allowlist and the peer's advertised
  capabilities is offerable, since that is exactly what the server admits. The
  two ways it can come up empty carry different remedies (nothing allowlisted
  here vs. the peer no longer advertising it), and they are now said out loud
  rather than collapsing into one "unavailable".
- Wire v1 is text-to-image / text-to-video only. A form holding an init image,
  references, LoRAs, keyframes, a clip to extend, an audio track, chained
  chunks or a non-text mode blocks Generate and names what has to go, instead
  of quietly dropping the conditioning the user set up. Nothing is cleared for
  them, so switching back to This instance renders what they built.

Readiness is re-derived at click time, not read off the last poll: a capacity
window expires on the clock, so an enabled button can already be pointing at a
lapsed peer.
…faults settle (#4348)

The engine-defaults effect is keyed on the engine, not on `modelId`, so the
`modelId` it read was whatever the render that queued it captured. React drains
passive effects on its own scheduler task — a separate task from the DOM
mutation a `findBy*` query resolves on — so a pick landing in that window was
invisible to the queued effect: it read the selection as absent and reset to the
engine default. The pick reverted silently, and every gate hanging off it went
quiet with it (the "selected model weights are not installed yet" warning, the
sized download button, the disabled Generate).

Reading the value at flush time through a functional updater cannot go stale.
Adding `modelId` to the dep array would NOT have fixed it — the already-queued
effect still runs with its own closure first.

The reason this was invisible until a loaded CI runner hit it: the Model select
was bound to the raw `modelId`, which is `''` until that effect drains, and
React leaves a `<select>` alone when no option matches its value. So the control
kept displaying its first option while the component held no selection at all —
the assertion that the select "has value 8-bit" passed against a component that
had not selected anything. Binding the control to the effective id makes it
agree with the readiness gates from the first commit.

Also stops `useFederatedMediaTarget` committing peer state when nothing is opted
in, so adding the picker to a form costs it no extra render on an install with
no media provider — the common case, and the one where this timing window is
narrowest to begin with.

Guarded by client/src/components/music/musicGenModelSelection.guard.test.js:
both shapes are asserted at the source level, because `act()` drains passive
effects between discrete events and so cannot reproduce the interleaving that a
loaded runner does.
@atomantic
atomantic merged commit 7a35a4f into main Aug 22, 2026
7 checks passed
@atomantic
atomantic deleted the claim/issue-4348 branch August 22, 2026 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant