fix(config): set MD5 checksums for DigitalOcean and GCS flavors since they don't support CRC32C, and explicitly set CRC32C for MinIO#6487
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 52a5238847
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… they don't support CRC32C, and explicitly set CRC32C for MinIO
fulmicoton-dd
force-pushed
the
paul-masurel/fix-checksum-flavors
branch
from
52a5238 to
87d802c
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 87d802ccbd
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| self.force_path_style_access = true; | ||
| self.disable_multi_object_delete = true; | ||
| // doesn't support CRC32C | ||
| self.checksum_algorithm = ChecksumAlgorithm::Md5; |
There was a problem hiding this comment.
When an S3 config uses a flavor and also sets checksum_algorithm: disabled (or md5 for MinIO/Garage), apply_flavor() runs after deserialization and this unconditional assignment replaces the user-provided value; docs/configuration/storage-config.md documents checksum_algorithm as the supported control while disable_checksums is deprecated. For example, a DigitalOcean or GCS user disabling checksums to work around provider/proxy failures will still send Content-MD5, and MinIO/Garage users who choose disabled will get CRC32C. Please apply these flavor values only as defaults when the field was omitted, or otherwise keep honoring explicit config.
Useful? React with 👍 / 👎.
|
I remember Nadav mentioning that DO supports crc32c: |
2 participants
Summary
Md5now thatCrc32cis the defaultDisabled(which predated the CRC32C default); switch toMd5since it also lacks CRC32C S3 SDK supportCrc32cfor clarity (previously fell through to the new default)Follows up on #6442 which made CRC32C the default checksum algorithm.
Test plan
cargo nextest run -p quickwit-configpasses