Skip to content

[BUG] Fix s5cmd action placing subcommand flags before the subcommand - #84

Merged
John McCall (lowlydba) merged 2 commits into
mainfrom
lowlydba-cuddly-goggles
Aug 18, 2026
Merged

[BUG] Fix s5cmd action placing subcommand flags before the subcommand#84
John McCall (lowlydba) merged 2 commits into
mainfrom
lowlydba-cuddly-goggles

Conversation

@lowlydba

@lowlydba John McCall (lowlydba) commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

--delete is a sync-subcommand flag in s5cmd, not a global one, so it has to come after sync (s5cmd sync --delete src dst). The flags input on .github/actions/s5cmd/action.yml is documented as global flags and always gets placed before the subcommand, so OvertureMaps/stac's publish-catalog.yaml passing flags: --delete broke production publishing:

Running: s5cmd --delete sync public_releases/ s3://overturemaps-extras-us-west-2/stac/
Incorrect Usage: flag provided but not defined: -delete

Fixes #83.

Adds a subcommand-flags input, placed after the subcommand and before source/destination. flags and its documented global-flag behavior are untouched, so this is backwards compatible. Guarded subcommand-flags against the run command in the validation step, since its batch-file lines carry their own per-command flags already.

Since OvertureMaps/stac pins this action @main and its publish-catalog.yaml still passes flags: --delete, that alone would still fail after merging (flags keeps its documented before-the-subcommand placement). Rather than requiring a coordinated follow-up PR there first, this also hardcodes a targeted auto-correction: if flags contains --delete and the command is sync, the action moves it after the subcommand automatically and logs a warning pointing callers at subcommand-flags. Anything else misplaced in flags still fails as before. Extend this case-by-case if more subcommand flags need the same treatment.

Verified the ARGS construction locally with bash, both against the new subcommand-flags input and against the existing flags: --delete shape stac's workflow currently uses; both build s5cmd sync --delete public_releases/ s3://overturemaps-extras-us-west-2/stac/ correctly. Not tested against a real S3 bucket.

s5cmd flags like sync --delete are subcommand-scoped, not global, and
must come after the subcommand. The flags input places everything
before the subcommand, so passing flags: --delete broke with
'flag provided but not defined: -delete'. Add a separate
subcommand-flags input placed after the subcommand; flags keeps its
documented global-flag behavior.

Fixes #83

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Signed-off-by: John McCall <john@overturemaps.org>
@lowlydba
John McCall (lowlydba) marked this pull request as ready for review August 18, 2026 22:45
@lowlydba
John McCall (lowlydba) requested a review from a team as a code owner August 18, 2026 22:45
Copilot AI lite review requested due to automatic review settings August 18, 2026 22:45

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request fixes the .github/actions/s5cmd composite action’s argument ordering so subcommand-specific flags (e.g. sync --delete) can be placed after the subcommand, preventing production failures where subcommand flags were incorrectly treated as global flags.

Changes:

  • Adds a new subcommand-flags input to pass flags after the subcommand and before positional args.
  • Adds input validation to reject subcommand-flags when using the run command.
  • Updates the ARGS construction to insert subcommand-flags between the subcommand and source/destination.
Suppressed comments (1)

.github/actions/s5cmd/action.yml:128

  • The comment says “quote within flags if needed”, but the current read -ra ... <<< "$FLAGS" splitting does not honor shell-style quoting. Either update the comment to reflect whitespace-only splitting, or change parsing to a quote-aware approach.
        # Build argument array: global flags, then subcommand, then subcommand flags, then positional args
        ARGS=()

        # Word-split flags safely (simple space split; quote within flags if needed)
        if [[ -n "$FLAGS" ]]; then

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/actions/s5cmd/action.yml Outdated
Hardcode a targeted fix so existing callers (e.g. stac's
publish-catalog.yaml, which pins this action @main and predates
subcommand-flags) keep working without a coordinated follow-up PR:
if 'flags' contains --delete and the command is sync, move it after
the subcommand automatically and emit a warning pointing callers at
subcommand-flags. Anything else misplaced in 'flags' still fails as
before; extend this case-by-case if more subcommand flags need it.

Also documents the whitespace-only splitting behavior of flags and
subcommand-flags per PR #84 review feedback.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Signed-off-by: John McCall <john@overturemaps.org>
@lowlydba
John McCall (lowlydba) merged commit aca281b into main Aug 18, 2026
9 checks passed
@lowlydba
John McCall (lowlydba) deleted the lowlydba-cuddly-goggles branch August 18, 2026 23:23
John McCall (lowlydba) added a commit to OvertureMaps/stac that referenced this pull request Aug 18, 2026
OvertureMaps/workflows#84 merged (aca281b), so @main already has the sync
--delete fix. Drop the temporary e2e pin from earlier in this branch.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Signed-off-by: John McCall <john@overturemaps.org>
John McCall (lowlydba) added a commit to OvertureMaps/stac that referenced this pull request Aug 19, 2026
* test: pin s5cmd action to workflows#84 head sha for e2e verification

Temporary pin to 1e394d3cfcdc529069e9abe8643c3288ea940a52 (OvertureMaps/workflows#84)
to exercise the sync --delete fix via workflow_dispatch before merging the
upstream fix. Revert to @main once verified.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Signed-off-by: John McCall <john@overturemaps.org>

* fix(publish-catalog): skip session tagging on cloudfront-invalidator assume-role

sts:AssumeRole to cloudfront-invalidator failed: stac-publish-oidc-overturemaps
isn't authorized for sts:TagSession, which configure-aws-credentials sends by
default. Nothing in cloudfront-invalidator's trust or invalidate policy reads
session tags (no ABAC conditions), so skip tagging instead of granting the
permission.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Signed-off-by: John McCall <john@overturemaps.org>

* test: revert s5cmd pin to @main now that workflows#84 is merged

OvertureMaps/workflows#84 merged (aca281b), so @main already has the sync
--delete fix. Drop the temporary e2e pin from earlier in this branch.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Signed-off-by: John McCall <john@overturemaps.org>

---------

Signed-off-by: John McCall <john@overturemaps.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

s5cmd action places subcommand-level flags before the subcommand, breaking sync --delete

3 participants