Skip to content

Fail CI on R CMD check NOTEs (needs shiny-workflows error-on input) #195

Description

@schloerke

Follow-up to #184. Blocked on rstudio/shiny-workflows#60.

State today

R CMD check on pkg-r/ is clean — 0 errors | 0 warnings | 0 notes — and make r-check-package keeps it that way:

cd $(PATH_PKG_R) && Rscript -e "devtools::check(document = FALSE, error_on = \"note\")"

Verified both directions: a clean tree exits 0, and reintroducing a single shiny::: call fails with Error: R CMD check found NOTEs (exit 2).

The gap

That gate is local only. CI still passes on a NOTE.

.github/workflows/check-r.yaml delegates to rstudio/shiny-workflows/.github/workflows/R-CMD-check.yaml, which calls r-lib/actions/check-r-package@v2 without setting error-on. The action defaults to '"warning"', and the reusable workflow exposes no input to change it. So a NOTE introduced in a PR that nobody happens to run make r-check-package on lands on main silently.

Not workaroundable from here:

  • extra-check-args is for R CMD check arguments, not rcmdcheck's error_on.
  • The .github/shiny-workflows/check.{sh,R} hook runs as the step before check-r-package, so it can't influence that step's inputs.

The work

Once rstudio/shiny-workflows#60 ships the input, this is a two-line change:

  R-CMD-check:
    uses: rstudio/shiny-workflows/.github/workflows/R-CMD-check.yaml@v1
    with:
      working-directory: pkg-r
      error-on: '"note"'

Then confirm on a throwaway PR that a deliberately introduced NOTE actually turns CI red.

Watch out for

error_on = "note" can be flaky in CI even when the package is locally clean, because of environment NOTEs that never appear on a dev machine — most commonly:

  • checking CRAN incoming feasibility (unstable network / dev version numbers)
  • checking for future file timestamps (clock-check service unreachable)

Both are plausible on the 9-platform matrix. If they show up, prefer narrowing (e.g. apply error-on to a single representative platform, or set _R_CHECK_CRAN_INCOMING_=false) over reverting to "warning" — a check that never comes back clean is a check nobody reads, which is part of how the #182#186 parity bugs survived.

Alternative if the upstream input stalls

Add a dedicated single-platform job here running rcmdcheck::rcmdcheck(error_on = "note") directly, alongside the existing matrix. More YAML to own, but unblocked and easy to scope to one runner.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions