From 8f8471e9673e4ff55613b31280cc1a21c1ef55f2 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Thu, 16 Jul 2026 11:39:07 +0000 Subject: [PATCH] Version Packages --- .changeset/bound-stalled-serve-memory.md | 5 ----- .changeset/memory-mode-meta-sweep.md | 7 ------- .changeset/recovery-hardening.md | 5 ----- .changeset/wal-cardinality-cliff.md | 13 ------------- packages/durable-streams-rust/CHANGELOG.md | 18 ++++++++++++++++++ packages/durable-streams-rust/package.json | 2 +- packages/sync-service/CHANGELOG.md | 6 ++++++ packages/sync-service/package.json | 2 +- 8 files changed, 26 insertions(+), 32 deletions(-) delete mode 100644 .changeset/bound-stalled-serve-memory.md delete mode 100644 .changeset/memory-mode-meta-sweep.md delete mode 100644 .changeset/recovery-hardening.md delete mode 100644 .changeset/wal-cardinality-cliff.md diff --git a/.changeset/bound-stalled-serve-memory.md b/.changeset/bound-stalled-serve-memory.md deleted file mode 100644 index 751c8401a4..0000000000 --- a/.changeset/bound-stalled-serve-memory.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@core/sync-service": patch ---- - -Bound the memory pinned by a shape response served to a slow or stalled client. Two compounding issues let every stalled connection pin its entire in-flight log chunk (~10 MB by default) for as long as the serve lived: the log file was read eagerly as one whole-range binary whose entries were served as sub-binary slices (pinning the full chunk plus the full entry list), and the JSON encoder batched response elements by item count only, so a batch of large rows grew into a multi-megabyte unit held in full by the request process and the socket's driver queue. Accumulated stalled serves could exhaust node memory (observed in production: ~400 stalled serves pinning ~3.9 GB, immune to GC since the references are live). The log is now read lazily in 64 KiB blocks and encoder batches are additionally capped at 256 KiB, bounding the pinned memory per stalled connection to well under 1 MB regardless of chunk size. diff --git a/.changeset/memory-mode-meta-sweep.md b/.changeset/memory-mode-meta-sweep.md deleted file mode 100644 index b0eb5d6074..0000000000 --- a/.changeset/memory-mode-meta-sweep.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -"@electric-ax/durable-streams-server-rust": patch ---- - -Memory-mode CPU fix: batch meta sidecar flushes into a periodic sweep (#4691). - -`--durability memory` appends no longer schedule a per-stream debounced sidecar flush (a timer task + full sidecar rewrite per stream per 100 ms — ~5x wal-mode CPU at high stream cardinality under low per-stream rates). Appends and TTL read touches now only mark the stream dirty in a store-level set; a single 1 s sweeper flushes all dirty sidecars in one pass, mirroring the batched checkpoint treatment wal mode got in the write-path overhaul. The sidecar's producer/access state remains a non-durable lagging flush; its lag bound moves from 100 ms to the 1 s sweep cadence. Durable flush-on-close/delete paths are unchanged, and a pending flush can no longer resurrect the sidecar of a hard-deleted stream. diff --git a/.changeset/recovery-hardening.md b/.changeset/recovery-hardening.md deleted file mode 100644 index c2379a0baa..0000000000 --- a/.changeset/recovery-hardening.md +++ /dev/null @@ -1,5 +0,0 @@ ---- -"@electric-ax/durable-streams-server-rust": patch ---- - -Recovery hardening: durability barriers (committer fdatasync, checkpoint syncfs, segment seal) are fail-stop instead of retryable-in-place (a retried fsync can falsely succeed on Linux and ack/recycle lost bytes); failed checkpoints re-register their dirty set (previously a transient error + restart truncated acked bytes); torn sidecars are quarantined instead of deleting the stream's data file; missing stream-lane mounts refuse to boot instead of letting the WAL reset destroy the lane's records; append stage failures roll back the data write and producer state (500'd bytes no longer resurrect; retries no longer swallowed as duplicates); sealing cuts at the durable frontier; unreadable sealed chunks fail the read instead of serving a response with missing interior bytes; dir fsyncs added across the WAL metadata lifecycle. diff --git a/.changeset/wal-cardinality-cliff.md b/.changeset/wal-cardinality-cliff.md deleted file mode 100644 index d38c7b7956..0000000000 --- a/.changeset/wal-cardinality-cliff.md +++ /dev/null @@ -1,13 +0,0 @@ ---- -"@electric-ax/durable-streams-server-rust": patch ---- - -Eliminate the WAL write cardinality cliff (10.4k → 383k appends/s @100k streams; 212k @1M). - -- Checkpoint durability now uses one `syncfs` barrier per stream lane on Linux (was O(touched-streams) per-file `fdatasync` — the barrier storm that collapsed throughput at high stream counts). -- New `--wal-checkpoint-interval-ms` (per-shard time trigger, default 3000) and `--wal-checkpoint-wal-bytes` (retained-WAL size budget, 0 = off): checkpoint cadence is an explicit crash-replay budget and shards self-stagger instead of storming together. -- New `--stream-lanes N` (default 1 = unchanged layout): hash stream data files across `streams/<0..N>/` dirs, one per device, spreading checkpoint writeback over N devices with N parallel barriers. The lane count is persisted and validated on open. -- New `--server-stats N` telemetry (`SRV_STATS`: cpu / inflight / service / lock / durability-wait per interval) — the dependency-free bottleneck diagnostics used to find all of the above. Memory-mode plain appends no longer queue redundant sidecar flushes. -- Removed dead/diagnostic flags: `--wal-fsync-parallel`, `--wal-meta-gate`, `--mem-meta-gate`, `--meta-sweep-disable`, `--meta-sweep-stats`, `--tier local` / `--tier-local-dir` (tier is `off|s3`). `--durability memory` combined with `--tier` is now rejected at startup. WAL records without `PAYLOAD_CHECKSUMMED` decode as torn (no released writer ever emitted them). - -Deployment guidance (device layout, CPU pinning, checkpoint budgets): `WAL_TUNING.md`. diff --git a/packages/durable-streams-rust/CHANGELOG.md b/packages/durable-streams-rust/CHANGELOG.md index d7d943860b..4e53837eb3 100644 --- a/packages/durable-streams-rust/CHANGELOG.md +++ b/packages/durable-streams-rust/CHANGELOG.md @@ -1,5 +1,23 @@ # @electric-ax/durable-streams-server-rust +## 0.1.5 + +### Patch Changes + +- 9acd04f: Memory-mode CPU fix: batch meta sidecar flushes into a periodic sweep (#4691). + + `--durability memory` appends no longer schedule a per-stream debounced sidecar flush (a timer task + full sidecar rewrite per stream per 100 ms — ~5x wal-mode CPU at high stream cardinality under low per-stream rates). Appends and TTL read touches now only mark the stream dirty in a store-level set; a single 1 s sweeper flushes all dirty sidecars in one pass, mirroring the batched checkpoint treatment wal mode got in the write-path overhaul. The sidecar's producer/access state remains a non-durable lagging flush; its lag bound moves from 100 ms to the 1 s sweep cadence. Durable flush-on-close/delete paths are unchanged, and a pending flush can no longer resurrect the sidecar of a hard-deleted stream. + +- 012dc4a: Recovery hardening: durability barriers (committer fdatasync, checkpoint syncfs, segment seal) are fail-stop instead of retryable-in-place (a retried fsync can falsely succeed on Linux and ack/recycle lost bytes); failed checkpoints re-register their dirty set (previously a transient error + restart truncated acked bytes); torn sidecars are quarantined instead of deleting the stream's data file; missing stream-lane mounts refuse to boot instead of letting the WAL reset destroy the lane's records; append stage failures roll back the data write and producer state (500'd bytes no longer resurrect; retries no longer swallowed as duplicates); sealing cuts at the durable frontier; unreadable sealed chunks fail the read instead of serving a response with missing interior bytes; dir fsyncs added across the WAL metadata lifecycle. +- db4977d: Eliminate the WAL write cardinality cliff (10.4k → 383k appends/s @100k streams; 212k @1M). + - Checkpoint durability now uses one `syncfs` barrier per stream lane on Linux (was O(touched-streams) per-file `fdatasync` — the barrier storm that collapsed throughput at high stream counts). + - New `--wal-checkpoint-interval-ms` (per-shard time trigger, default 3000) and `--wal-checkpoint-wal-bytes` (retained-WAL size budget, 0 = off): checkpoint cadence is an explicit crash-replay budget and shards self-stagger instead of storming together. + - New `--stream-lanes N` (default 1 = unchanged layout): hash stream data files across `streams/<0..N>/` dirs, one per device, spreading checkpoint writeback over N devices with N parallel barriers. The lane count is persisted and validated on open. + - New `--server-stats N` telemetry (`SRV_STATS`: cpu / inflight / service / lock / durability-wait per interval) — the dependency-free bottleneck diagnostics used to find all of the above. Memory-mode plain appends no longer queue redundant sidecar flushes. + - Removed dead/diagnostic flags: `--wal-fsync-parallel`, `--wal-meta-gate`, `--mem-meta-gate`, `--meta-sweep-disable`, `--meta-sweep-stats`, `--tier local` / `--tier-local-dir` (tier is `off|s3`). `--durability memory` combined with `--tier` is now rejected at startup. WAL records without `PAYLOAD_CHECKSUMMED` decode as torn (no released writer ever emitted them). + + Deployment guidance (device layout, CPU pinning, checkpoint budgets): `WAL_TUNING.md`. + ## 0.1.4 ### Patch Changes diff --git a/packages/durable-streams-rust/package.json b/packages/durable-streams-rust/package.json index fa8568149c..7dc2e18bfe 100644 --- a/packages/durable-streams-rust/package.json +++ b/packages/durable-streams-rust/package.json @@ -1,6 +1,6 @@ { "name": "@electric-ax/durable-streams-server-rust", - "version": "0.1.4", + "version": "0.1.5", "description": "High-performance Durable Streams server (Rust) — native binary distributed via npm.", "license": "Apache-2.0", "type": "module", diff --git a/packages/sync-service/CHANGELOG.md b/packages/sync-service/CHANGELOG.md index 40c2429336..baf1060a0c 100644 --- a/packages/sync-service/CHANGELOG.md +++ b/packages/sync-service/CHANGELOG.md @@ -1,5 +1,11 @@ # @core/sync-service +## 1.7.8 + +### Patch Changes + +- 5a298c6: Bound the memory pinned by a shape response served to a slow or stalled client. Two compounding issues let every stalled connection pin its entire in-flight log chunk (~10 MB by default) for as long as the serve lived: the log file was read eagerly as one whole-range binary whose entries were served as sub-binary slices (pinning the full chunk plus the full entry list), and the JSON encoder batched response elements by item count only, so a batch of large rows grew into a multi-megabyte unit held in full by the request process and the socket's driver queue. Accumulated stalled serves could exhaust node memory (observed in production: ~400 stalled serves pinning ~3.9 GB, immune to GC since the references are live). The log is now read lazily in 64 KiB blocks and encoder batches are additionally capped at 256 KiB, bounding the pinned memory per stalled connection to well under 1 MB regardless of chunk size. + ## 1.7.7 ### Patch Changes diff --git a/packages/sync-service/package.json b/packages/sync-service/package.json index 40fc9f4a72..305878f791 100644 --- a/packages/sync-service/package.json +++ b/packages/sync-service/package.json @@ -1,7 +1,7 @@ { "name": "@core/sync-service", "private": true, - "version": "1.7.7", + "version": "1.7.8", "scripts": { "publish:hex": "../../scripts/publish_hex.sh electric", "changeset": "pushd ../..; pnpm changeset; popd"