feat: add aria2c build-fullmap downloads - #74
Merged
Merged
Conversation
📝 WalkthroughWalkthroughThe PR adds optional Changesaria2c fullmap downloading
Estimated code review effort: 3 (Moderate) | ~30 minutes Sequence Diagram(s)sequenceDiagram
participant CLI
participant FullmapPipeline
participant aria2c
participant FullmapBuilder
CLI->>FullmapPipeline: Forward aria2c=True
FullmapPipeline->>aria2c: Download class and synonym files
aria2c-->>FullmapPipeline: Return downloaded paths
FullmapPipeline->>FullmapBuilder: Pass paths and thread settings
FullmapBuilder-->>CLI: Build fullmap
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/cli.md`:
- Line 98: Update the --aria2c CLI reference entry to document that downloads
also fail when aria2c exits successfully but the final file is missing or the
.aria2 control file remains, matching the validation performed by
download_babel_file_aria2c.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 2b0834bc-b201-4562-847a-a2acb9c7beb5
📒 Files selected for processing (5)
CHANGELOG.mddocs/cli.mddocs/fullmap.mdsrc/tablassert/cli.pytests/test_cover_cli.py
| | `--cache`, `-c` | Path | No | `./fullmap/downloads` | Directory for downloaded BABEL files (`classes/`, `synonyms/`) | | ||
| | `--version`, `-v` | str | No | `2026jul22` | BABEL snapshot date to fetch (a RENCI stamp, **not** Tablassert's version) | | ||
| | `--threads`, `-t` | int | No | `None` (auto) | Worker threads; auto-capped by memory on Linux (`/proc/meminfo`), else ~90% of CPUs | | ||
| | `--aria2c`, `-a` | Flag | No | `False` | Opt into the installed `aria2c` executable for resumable segmented BABEL downloads; fails loud if `aria2c` is missing or exits non-zero | |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Document incomplete-output failures.
download_babel_file_aria2c also raises when aria2c exits successfully but the final file is missing or the .aria2 control file remains. Include this case so the CLI reference matches src/tablassert/cli.py:425-509.
Proposed wording
-| `--aria2c`, `-a` | Flag | No | `False` | Opt into the installed `aria2c` executable for resumable segmented BABEL downloads; fails loud if `aria2c` is missing or exits non-zero |
+| `--aria2c`, `-a` | Flag | No | `False` | Opt into the installed `aria2c` executable for resumable segmented BABEL downloads; fails loud if `aria2c` is missing, exits non-zero, or leaves an incomplete file |📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| | `--aria2c`, `-a` | Flag | No | `False` | Opt into the installed `aria2c` executable for resumable segmented BABEL downloads; fails loud if `aria2c` is missing or exits non-zero | | |
| | `--aria2c`, `-a` | Flag | No | `False` | Opt into the installed `aria2c` executable for resumable segmented BABEL downloads; fails loud if `aria2c` is missing, exits non-zero, or leaves an incomplete file | |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/cli.md` at line 98, Update the --aria2c CLI reference entry to document
that downloads also fail when aria2c exits successfully but the final file is
missing or the .aria2 control file remains, matching the validation performed by
download_babel_file_aria2c.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds an opt-in
aria2cdownloader path forbuild-fullmapBABEL downloads, giving users segmented/resumable transfers when the executable is installed while keeping the existing Python downloader as the default.Aria2c Downloader
build-fullmap --aria2c/-aand threadsaria2c=Truethroughbuild_fullmap_pipeline.download_babel_file_aria2c()with--continue=true,--max-tries, and--retry-wait, preserving.aria2control files so dropped downloads can resume on rerun.BabelDownloadErrorwhenaria2cis missing, exits non-zero, leaves an incomplete file, or receives invalid retry settings.Downloadprogress bar; Python downloads retain byte-level detail, whilearia2cmode reportsaria2c downloadingand suppresses aria2's own console UI.Docs
--aria2c/-aindocs/cli.md,docs/fullmap.md, andCHANGELOG.md.Testing
uv run pytest tests/test_cover_cli.py tests/test_cli_progress.py tests/test_docs_cli_coverage.py -n 0→34 passed, 48 warnings.uv run ruff check .→ all checks passed.uv run ruff format --check .→69 files already formatted.uv run tablassert build-fullmap --help→ confirmsARIA2C --aria2c -a [default: False].Summary by CodeRabbit
New Features
--aria2c/-asupport tobuild-fullmapfor segmented, resumable downloads with retries and progress reporting.aria2cis unavailable or unsuccessful.Documentation