perf(release): rank download sources by speed, resume transfers, and sign Linux archives

[bobleer]

Problem

A slow link, not an unreachable one, was the failure mode. Every download path only failed over on error, and used a wall-clock ceiling as its sole give-up condition, so success required speed > size / ceiling:

• Relay deploy used --max-time 900 with no resume, and re-rmed the partial file on every retry. A ~20 MB archive needed >23 KB/s; below that each attempt timed out mid-transfer and restarted from zero, so it could never finish. With --retry 3 and no --retry-max-time (curl resets --max-time per retry) that burned up to 60 minutes per source, silently — -s swallowed the output.
• CLI self-update set reqwest's whole-request .timeout(120), which includes the body, so a ~30 MB archive needed >250 KB/s. Both sources shared the cap, so a slow link failed both rather than degrading.
• Desktop updater had the same defect on the largest artifact. Tauri walks endpoints, stops at the first usable manifest, then downloads from the URL inside it. latest.json is ~2 KB, so a crawling-but-reachable GitHub always won the race to answer and pinned an 80–160 MB download to itself; the mirror was reached only when GitHub errored outright.

Change

All three paths probe candidates with a ranged request and order them by measured throughput (≥128 KB/s counts as healthy; if nothing clears the bar the fastest is still used, since a slow download beats a 20-minute source rebuild). Relay and CLI additionally resume rather than restart — across a source switch, and for the CLI across runs.

There is deliberately no wall-clock ceiling on a transfer. Any ceiling reintroduces the original bug further out: at the 8 KB/s floor a 30 MB archive needs ~3750 s, so even a generous 3600 s cap would kill a download that was progressing fine. The throughput floor is the give-up condition — it aborts a dead or hung link within --speed-time, and --connect-timeout covers setup. The harness asserts the archive fetch carries no --max-time while still requiring a floor.

Desktop is still routed through Update::download: minisign verification lives inside it, so hand-rolling the fetch and calling Update::install would silently skip signature checking. Its probe targets the package each manifest advertises, not the manifest itself, which would measure latency rather than throughput.

Signing

Before this PR only the five updater artifacts were signed, by the Tauri bundler during tauri build. The nine installers people download by hand from the release page — dmg, deb, rpm, the Windows installer, the direct AppImages — had no signature at all. The CLI and relay archives had none either, only a .sha256 fetched from the same host that served the archive, which proves the transfer was intact and nothing about the source. The CN relay path prefers a third-party GitHub proxy (ghfast.top) by design, so that mattered.

• Everything is signed now, with the same minisign key the updater already uses, via a shared scripts/sign-release-assets.sh. It skips anything the bundler already signed so updater signatures stay intact, and self-verifies before publishing — a signature nobody checked is worse than none. minisign.pub ships with every release, because a signature nobody can fetch a key for is not verifiable.
• CLI self-update enforces it. Official builds embed the public key and refuse an unsigned release; builds without it (forks, local) fall back to checksum-only.
• Relay hosts verify without minisign. A user's server has no trust root, and the archive signature can only be checked by whoever downloads the archive — the server. Resolved by signing the .sha256 files too: Desktop verifies that signature locally (a couple hundred bytes) and exports the proven hash into the deploy script as BITFUN_EXPECTED_SHA256_<TARGET>. The remote needs nothing but sha256sum, and no origin can override that hash.
• Cross-origin checksums where no verified hash exists: the .sha256 comes from the canonical GitHub URL whatever origin served the bytes, derivable from any candidate URL including the mirror's versioned path. Same-origin fallback only when GitHub is wholly unreachable, and it says so.

Not OS-level code signing — Gatekeeper and SmartScreen still need Apple/Authenticode certificates the project doesn't hold. Documented as such.

One implementation, two callers

The ranked/resumable download and runtime image moved to src/apps/relay-server/release-download.sh, sourced by deploy.sh and embedded by relay_deploy.rs with include_str!, exactly as mirror.sh already is. The documented manual path consequently stops compiling Rust on the user's VPS for ~20 minutes; --build-from-source restores the old behaviour.

Sync window

• Relay deploy takes the mirror URL from the mirror's own manifest instead of a pinned /<version>/ path, which 404ed for every Desktop build outside the two releases the mirror retained. KEEP_VERSIONS 2 → 6.
• The mirror publishes the four small Linux archives before the ~700 MB of Desktop packages, so the window where it advertises a release it cannot serve is minutes rather than however long the Desktop mirror takes.
• CLI self-update reads both manifests concurrently and installs whichever advertises the newer version, instead of silently reporting "up to date" when it fell back to a mirror still on the previous release.
• Optional post-publish webhook lets the mirror sync immediately; cron remains the source of truth and an unset secret keeps today's behaviour.

Other robustness fixes

• Startup no longer downloads: the automatic CLI check fetches only the manifest (10 s budget) and hands the archive to a detached child, serialised by an update.lock.
• Relay preflight reports free disk under $HOME and on Docker's data root; the source-build fallback refuses to start without ~6 GB rather than dying as an opaque compiler error.
• Nightly asset names drop SemVer build metadata — GitHub rewrites + in stored asset filenames, which would have made every nightly URL in linux-binaries.json a 404.
• linux-binaries.yml is the sole publisher of the Linux CLI. It and cli-package.yml both ran on release: published and uploaded identical asset names, and softprops/action-gh-release deletes a same-named asset before writing.

Defects found by executing rather than reading

• kill on a shell blocked in a long sleep is not delivered until that sleep returns, which stalled every relay download a full tick past completion (test wall-clock 120 s → 14 s).
• An unreachable mirror manifest made a command substitution fail, which only avoided aborting the caller because set -e happens to be suspended inside an if condition.
• A cancelled wizard left the relay stopped under its backup name, invisible to the "already deployed" probe.

Verification

• scripts/relay/release-download-harness.sh drives the generated bash against a stubbed curl — 9 checks: slow-GitHub-vs-fast-mirror, all-sources-slow, mid-transfer stall, checksum mismatch, unreachable mirror manifest, all-dead fallback, mirror bytes verified against GitHub's checksum (the stub gives every non-GitHub origin a wrong one), no wall-clock ceiling, floor present.
• self_update.rs tests run against real throttled sockets (ranking, slow-but-alive completion, cross-source resume, cross-run staging).
• Signature tests in both crates use a fixture produced with the real minisign CLI. The signing script, the CI invocation (password on stdin, Tauri's base64 wrapper, skip-already-signed) and the verification command the relay README documents were all validated locally against a generated key.
• cargo test: bitfun-services-integrations 57, bitfun-cli 10, bitfun-desktop 1. node --test 11. check:github-config, check:repo-hygiene. clippy clean on every changed file.

Notes for review

• The CI signing steps have not run in CI yet. Mechanics are validated locally and each step self-verifies, so a wrong assumption fails the release loudly rather than publishing bad signatures — but the first real run is the confirmation. Needs TAURI_UPDATER_PUBKEY, TAURI_SIGNING_PRIVATE_KEY, TAURI_SIGNING_PRIVATE_KEY_PASSWORD; wired at every call site, including BITFUN_RELEASE_PUBKEY into the Desktop and CLI builds as their trust root.
• Archive sizes (~20 MB relay / ~30 MB CLI) are still estimated from Desktop package sizes; the cliff is size / floor, worth measuring against a real release to confirm the 128 KB/s bar. All tuning constants are named and colocated.
• The CLI crate needed recursion_limit = "256" — the added async layers tipped trait resolution over the default depth.