Download as zip#1079
Merged
Merged
Conversation
Contributor
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (11)
📝 WalkthroughWalkthroughAdds ZIP download capability to the S3 Explorer. A StreamSaver service worker ( ChangesZIP Download Feature
Sequence DiagramsequenceDiagram
participant UI as S3ExplorerMainView
participant Page as Page.tsx (onDownload)
participant Thunk as downloadAsZip thunk
participant Fn as downloadS3UrisAsZip
participant SS as streamsaver (mitm.html + sw.js)
participant S3 as S3Client
participant Zip as ZipWriter
UI->>Page: onDownload({ s3Uris })
alt single non-folder URI
Page->>Thunk: downloadObject({ s3Uri })
else multiple or folder URIs
Page->>Thunk: downloadAsZip({ s3Uris })
Thunk->>Fn: downloadS3UrisAsZip(...)
Fn->>SS: createWriteStream(filename.zip)
SS-->>Fn: WritableStream
Fn->>S3: listObjects / crawl prefixes
S3-->>Fn: directories + objects
Fn->>Zip: add directories
loop each object
Fn->>S3: getObjectContent or HTTP fetch
S3-->>Fn: ReadableStream
Fn->>Zip: add(path, stream)
end
Fn->>Zip: close()
Zip-->>SS: piped bytes → browser download
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
garronej
force-pushed
the
download_as_zip
branch
from
e09f8d2 to
a2f2708
Compare
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@web/src/core/usecases/s3ExplorerUiController/decoupledLogic/downloadAsZip.ts`:
- Around line 302-309: The unbounded Promise.all() call when processing
result.prefixes in the crawlPrefix recursive logic creates excessive concurrent
requests to listObjects, which can freeze or timeout the ZIP operation. Replace
the direct Promise.all() approach with a shared bounded-concurrency control
mechanism (such as a semaphore or queue worker pool) that limits the number of
concurrent crawlPrefix calls across all recursive levels. This ensures that
regardless of the folder tree depth, the total concurrency remains controlled
and prevents request storms.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 1437e3ea-e954-451d-b277-e19e63dc758f
⛔ Files ignored due to path filters (1)
web/yarn.lockis excluded by!**/yarn.lock,!**/*.lock
📒 Files selected for processing (11)
web/nginx.confweb/package.jsonweb/public/streamsaver/mitm.htmlweb/public/streamsaver/sw.jsweb/src/core/usecases/s3ExplorerUiController/decoupledLogic/downloadAsZip.tsweb/src/core/usecases/s3ExplorerUiController/thunks.tsweb/src/streamsaver.d.tsweb/src/ui/pages/s3Explorer/Page.tsxweb/src/ui/shared/codex/S3ExplorerMainView/S3ExplorerMainView.spec.mdweb/src/ui/shared/codex/S3ExplorerMainView/S3ExplorerMainView.stories.tsxweb/src/ui/shared/codex/S3ExplorerMainView/S3ExplorerMainView.tsx
Comment on lines
+302
to
+309
| const nestedEntries = await Promise.all( | ||
| result.prefixes.map(s3Uri => | ||
| crawlPrefix({ | ||
| s3Client, | ||
| s3UriPrefix: s3Uri | ||
| }) | ||
| ) | ||
| ); |
Contributor
There was a problem hiding this comment.
Bound recursive prefix crawling concurrency
Line 302 currently expands child prefixes with unbounded Promise.all. On large folder trees, this creates a request storm against listObjects and can freeze/timeout the ZIP flow. Please switch to a shared bounded-concurrency crawl (queue/semaphore) across the whole traversal.
💡 Suggested direction
- const nestedEntries = await Promise.all(
- result.prefixes.map(s3Uri =>
- crawlPrefix({
- s3Client,
- s3UriPrefix: s3Uri
- })
- )
- );
+ const nestedEntries = await crawlPrefixesWithSharedLimiter({
+ s3Client,
+ prefixes: result.prefixes
+ });Use a single shared limiter (or iterative queue worker pool) so recursive levels do not multiply concurrency.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@web/src/core/usecases/s3ExplorerUiController/decoupledLogic/downloadAsZip.ts`
around lines 302 - 309, The unbounded Promise.all() call when processing
result.prefixes in the crawlPrefix recursive logic creates excessive concurrent
requests to listObjects, which can freeze or timeout the ZIP operation. Replace
the direct Promise.all() approach with a shared bounded-concurrency control
mechanism (such as a semaphore or queue worker pool) that limits the number of
concurrent crawlPrefix calls across all recursive levels. This ensures that
regardless of the folder tree depth, the total concurrency remains controlled
and prevents request storms.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by CodeRabbit
Release Notes