Skip to content

chore: publish new package versions#4694

Open
github-actions[bot] wants to merge 1 commit into
mainfrom
changeset-release/main
Open

chore: publish new package versions#4694
github-actions[bot] wants to merge 1 commit into
mainfrom
changeset-release/main

Conversation

@github-actions

@github-actions github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.

Releases

@electric-ax/durable-streams-server-rust@0.1.5

Patch Changes

  • 9acd04f: Memory-mode CPU fix: batch meta sidecar flushes into a periodic sweep (durable-streams-rust: memory mode uses ~5x more CPU than wal at identical low load (per-append debounced meta sidecar flush) #4691).

    --durability memory appends no longer schedule a per-stream debounced sidecar flush (a timer task + full sidecar rewrite per stream per 100 ms — ~5x wal-mode CPU at high stream cardinality under low per-stream rates). Appends and TTL read touches now only mark the stream dirty in a store-level set; a single 1 s sweeper flushes all dirty sidecars in one pass, mirroring the batched checkpoint treatment wal mode got in the write-path overhaul. The sidecar's producer/access state remains a non-durable lagging flush; its lag bound moves from 100 ms to the 1 s sweep cadence. Durable flush-on-close/delete paths are unchanged, and a pending flush can no longer resurrect the sidecar of a hard-deleted stream.

  • 012dc4a: Recovery hardening: durability barriers (committer fdatasync, checkpoint syncfs, segment seal) are fail-stop instead of retryable-in-place (a retried fsync can falsely succeed on Linux and ack/recycle lost bytes); failed checkpoints re-register their dirty set (previously a transient error + restart truncated acked bytes); torn sidecars are quarantined instead of deleting the stream's data file; missing stream-lane mounts refuse to boot instead of letting the WAL reset destroy the lane's records; append stage failures roll back the data write and producer state (500'd bytes no longer resurrect; retries no longer swallowed as duplicates); sealing cuts at the durable frontier; unreadable sealed chunks fail the read instead of serving a response with missing interior bytes; dir fsyncs added across the WAL metadata lifecycle.

  • db4977d: Eliminate the WAL write cardinality cliff (10.4k → 383k appends/s @100k streams; 212k @1m).

    • Checkpoint durability now uses one syncfs barrier per stream lane on Linux (was O(touched-streams) per-file fdatasync — the barrier storm that collapsed throughput at high stream counts).
    • New --wal-checkpoint-interval-ms (per-shard time trigger, default 3000) and --wal-checkpoint-wal-bytes (retained-WAL size budget, 0 = off): checkpoint cadence is an explicit crash-replay budget and shards self-stagger instead of storming together.
    • New --stream-lanes N (default 1 = unchanged layout): hash stream data files across streams/<0..N>/ dirs, one per device, spreading checkpoint writeback over N devices with N parallel barriers. The lane count is persisted and validated on open.
    • New --server-stats N telemetry (SRV_STATS: cpu / inflight / service / lock / durability-wait per interval) — the dependency-free bottleneck diagnostics used to find all of the above. Memory-mode plain appends no longer queue redundant sidecar flushes.
    • Removed dead/diagnostic flags: --wal-fsync-parallel, --wal-meta-gate, --mem-meta-gate, --meta-sweep-disable, --meta-sweep-stats, --tier local / --tier-local-dir (tier is off|s3). --durability memory combined with --tier is now rejected at startup. WAL records without PAYLOAD_CHECKSUMMED decode as torn (no released writer ever emitted them).

    Deployment guidance (device layout, CPU pinning, checkpoint budgets): WAL_TUNING.md.

@core/sync-service@1.7.8

Patch Changes

  • 5a298c6: Bound the memory pinned by a shape response served to a slow or stalled client. Two compounding issues let every stalled connection pin its entire in-flight log chunk (~10 MB by default) for as long as the serve lived: the log file was read eagerly as one whole-range binary whose entries were served as sub-binary slices (pinning the full chunk plus the full entry list), and the JSON encoder batched response elements by item count only, so a batch of large rows grew into a multi-megabyte unit held in full by the request process and the socket's driver queue. Accumulated stalled serves could exhaust node memory (observed in production: ~400 stalled serves pinning ~3.9 GB, immune to GC since the references are live). The log is now read lazily in 64 KiB blocks and encoder batches are additionally capped at 256 KiB, bounding the pinned memory per stalled connection to well under 1 MB regardless of chunk size.

@github-actions github-actions Bot force-pushed the changeset-release/main branch 6 times, most recently from bdb00c5 to eb2a339 Compare July 15, 2026 23:40
@github-actions github-actions Bot force-pushed the changeset-release/main branch from eb2a339 to 8f8471e Compare July 16, 2026 11:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants