feat(io): add streaming ItemCollection search writer - #1088
Merged
Conversation
Adds `stac_io::stream`: a streaming JSON writer for a search-response `ItemCollection`. The `features` array is written one item at a time, and the rest of the collection (links, context, counts) is supplied by a `finalize` callback after the items drain -- the `next` link needs the last item, and a `numberMatched` count may run concurrently. Public surface: `StreamSearch`, `StreamedSearch`, `ItemStream`, `Finalize`, and `write_item_collection`. This is the shared trait surface that pgstac 0.10 implements for its keyset-portal search, letting a pgstac database stream a search response with flat memory through the same writer rustac uses. Landing it on its own breaks the dependency cycle: pgstac 0.10 can build against a published stac-io instead of a git patch, and rustac's pgstac backend then builds against a published pgstac 0.10. Purely additive -- no manifest changes, no existing behavior touched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bitner
force-pushed
the
stac-io-streaming-writer
branch
from
July 21, 2026 14:48
4a43a5a to
4d72037
Compare
gadomski
requested changes
Jul 21, 2026
gadomski
left a comment
gadomski
left a comment
Member
There was a problem hiding this comment.
Just a couple of API surface questions/changes, otherwise looks good.
Addresses review feedback on #1088: the `BoxError` alias was public API that had nothing to do with streaming, and the writer returned it rather than the crate's own error. The stream types and `write_item_collection` now use `stac_io::Error`, and backends convert their own errors through a new `Error::Backend` boxed variant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bitner
added a commit
that referenced
this pull request
Jul 21, 2026
Addresses review feedback on #1088: the `BoxError` alias was public API that had nothing to do with streaming, and the writer returned it rather than the crate's own error. The stream types and `write_item_collection` now use `stac_io::Error`, and backends convert their own errors through a new `Error::Backend` boxed variant. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bitner
added a commit
that referenced
this pull request
Jul 21, 2026
Follows the #1088 review change: `write_search` now returns `stac_io::Error`, so the CLI no longer needs `anyhow::Error::from_boxed`. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Pete Gadomski <pete.gadomski@gmail.com>
gadomski
approved these changes
Jul 21, 2026
gadomski
enabled auto-merge (squash)
July 21, 2026 17:42
Merged
gadomski
pushed a commit
that referenced
this pull request
Jul 31, 2026
## 🤖 New release * `stac-derive`: 0.3.0 -> 0.3.1 * `stac-duckdb`: 0.3.9 -> 0.3.10 * `stac-io`: 0.3.0 -> 0.3.1 (✓ API compatible changes) * `stac-validate`: 0.6.10 -> 0.6.11 (✓ API compatible changes) * `stac`: 0.17.2 -> 0.17.3 * `stac-server`: 0.5.4 -> 0.5.5 * `rustac`: 0.2.12 -> 0.2.13 * `stac-extensions`: 0.1.12 -> 0.1.13 <details><summary><i><b>Changelog</b></i></summary><p> ## `stac-derive` <blockquote> ## [0.3.1](stac-derive-v0.3.0...stac-derive-v0.3.1) - 2026-07-31 ### Other - update Cargo.toml dependencies </blockquote> ## `stac-duckdb` <blockquote> ## [0.3.10](stac-duckdb-v0.3.9...stac-duckdb-v0.3.10) - 2026-07-31 ### Other - bump duckdb version in CI ([#1095](#1095)) </blockquote> ## `stac-io` <blockquote> ## [0.3.1](stac-io-v0.3.0...stac-io-v0.3.1) - 2026-07-31 ### Added - *(io)* add streaming ItemCollection search writer ([#1088](#1088)) </blockquote> ## `stac-validate` <blockquote> ## [0.6.11](stac-validate-v0.6.10...stac-validate-v0.6.11) - 2026-07-31 ### Other - update Cargo.toml dependencies </blockquote> ## `stac` <blockquote> ## [0.17.3](stac-v0.17.2...stac-v0.17.3) - 2026-07-31 ### Other - updated the following local packages: stac-derive </blockquote> ## `stac-server` <blockquote> ## [0.5.5](stac-server-v0.5.4...stac-server-v0.5.5) - 2026-07-31 ### Other - updated the following local packages: stac-duckdb, stac </blockquote> ## `rustac` <blockquote> ## [0.2.13](rustac-v0.2.12...rustac-v0.2.13) - 2026-07-31 ### Other - updated the following local packages: stac-duckdb, stac-io, stac-validate, stac, stac-server </blockquote> ## `stac-extensions` <blockquote> ## [0.1.13](stac-extensions-v0.1.12...stac-extensions-v0.1.13) - 2026-07-31 ### Other - updated the following local packages: stac </blockquote> </p></details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/). Co-authored-by: stac-utils-release-bot[bot] <249776822+stac-utils-release-bot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
stac_io::stream: a streaming JSON writer for a search-responseItemCollection. Thefeaturesarray is written one item at a time, and the rest of the collection (links, context, counts) is supplied by afinalizecallback after the items drain — thenextlink needs the last item, and anumberMatchedcount may run concurrently.Public surface:
StreamSearch,StreamedSearch,ItemStream,Finalize,write_item_collection.Why this is split out
This is the shared trait surface that pgstac 0.10 implements for its keyset-portal search, so a pgstac database can stream a search response with flat memory through the same writer rustac uses.
Right now there's a dependency cycle: #1077 needs the unreleased pgstac 0.10 crate, and pgstac's
v010iobranch needs thisstac-iocode (it currently carries a[patch.crates-io]git dependency on the rustac branch). Landing this on its own breaks it:stac-io0.3.1v010iodrops its[patch.crates-io]lines and builds against publishedstac-io→ release pgstac 0.10pgstac = "0.10"from crates.ioBecause the change is purely additive,
^0.3resolves 0.3.1 — pgstac needs no version bump, just the patch removal.Scope
Two files, no manifest changes:
crates/io/src/stream.rs(new)crates/io/src/lib.rs(+2: module declaration and re-export)futureswas already astac-iodependency. No existing behavior is touched.