From 87d6f5a3fd964d6c912e1cc37ba42bcb5c2584fc Mon Sep 17 00:00:00 2001 From: John Trammell Date: Tue, 9 Jun 2026 16:14:59 -0500 Subject: [PATCH 1/3] [Spec Kit] Add tasks --- .dockerignore | 2 + .github/copilot-instructions.md | 15 +- .gitignore | 2 +- .specify/feature.json | 2 +- .../contracts/build-verification-contract.md | 104 +++++++++++ specs/001-use-heredocs/data-model.md | 92 +++++++++ specs/001-use-heredocs/plan.md | 46 ++++- specs/001-use-heredocs/quickstart.md | 19 ++ specs/001-use-heredocs/research.md | 103 +++++++++-- specs/001-use-heredocs/tasks.md | 152 ++++++++------- .../verification/polish-checks.md | 12 +- .../verification/runtime-consistency.md | 22 ++- .../verification/us1-build.md | 18 +- .../checklists/requirements.md | 34 ++++ .../contracts/build-release-contract.md | 41 ++++ specs/002-multiarch-image-cache/data-model.md | 52 ++++++ specs/002-multiarch-image-cache/plan.md | 95 ++++++++++ specs/002-multiarch-image-cache/quickstart.md | 35 ++++ specs/002-multiarch-image-cache/research.md | 53 ++++++ specs/002-multiarch-image-cache/spec.md | 104 +++++++++++ specs/002-multiarch-image-cache/tasks.md | 175 ++++++++++++++++++ 21 files changed, 1074 insertions(+), 104 deletions(-) create mode 100644 specs/001-use-heredocs/contracts/build-verification-contract.md create mode 100644 specs/001-use-heredocs/data-model.md create mode 100644 specs/002-multiarch-image-cache/checklists/requirements.md create mode 100644 specs/002-multiarch-image-cache/contracts/build-release-contract.md create mode 100644 specs/002-multiarch-image-cache/data-model.md create mode 100644 specs/002-multiarch-image-cache/plan.md create mode 100644 specs/002-multiarch-image-cache/quickstart.md create mode 100644 specs/002-multiarch-image-cache/research.md create mode 100644 specs/002-multiarch-image-cache/spec.md create mode 100644 specs/002-multiarch-image-cache/tasks.md diff --git a/.dockerignore b/.dockerignore index c4995ef..2dc37b4 100644 --- a/.dockerignore +++ b/.dockerignore @@ -1,6 +1,8 @@ .git .github/ specs/ +Dockerfile* +.dockerignore node_modules/ dist/ build/ diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 7e6443a..e482f08 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,4 +1,17 @@ For additional context about technologies to be used, project structure, -shell commands, and other important information, read the current plan +shell commands, and other important information, read specs/002-multiarch-image-cache/plan.md + + +* The purpose of this project is to build a Docker container running the latest 2.6.x Ruby + version, with jemalloc. +* The target architecture of this a multi-architecture image supporting `linux/amd64` and + `linux/arm64` architectures. +* The image will be built using a multistage Dockerfile that compiles OpenSSL 1.1.1w, + jemalloc 5.3.0, and Ruby 2.6.10 from source. +* The resulting image will have a minimal runtime footprint, containing only the compiled + Ruby binaries and necessary runtime libraries. +* The build process will be structured with shell here-doc `RUN` blocks for clarity and + maintainability. + \ No newline at end of file diff --git a/.gitignore b/.gitignore index 7370f07..af52a6f 100644 --- a/.gitignore +++ b/.gitignore @@ -16,7 +16,7 @@ build/ # Editor / OS .DS_Store Thumbs.db +*.tmp *.swp .vscode/ .idea/ - diff --git a/.specify/feature.json b/.specify/feature.json index 6cba4d2..fa3bf04 100644 --- a/.specify/feature.json +++ b/.specify/feature.json @@ -1,3 +1,3 @@ { - "feature_directory": "specs/001-use-heredocs" + "feature_directory": "specs/002-multiarch-image-cache" } diff --git a/specs/001-use-heredocs/contracts/build-verification-contract.md b/specs/001-use-heredocs/contracts/build-verification-contract.md new file mode 100644 index 0000000..8ae8977 --- /dev/null +++ b/specs/001-use-heredocs/contracts/build-verification-contract.md @@ -0,0 +1,104 @@ +# Contract: Build and Runtime Verification Interface + +## Purpose + +Define the contributor-facing and CI-facing commands that must continue to work after the Dockerfile `RUN` refactor. + +## Supported Interfaces + +### 1. Local build command contract + +#### Command + +```sh +make build +``` + +#### Local Build Expected Behavior + +- Builds the default local image tag `ruby2.6-jemalloc:local` +- Uses the pinned Dockerfile build args unless the Dockerfile is intentionally updated +- Fails immediately if any Dockerfile build block exits non-zero + +#### Compatibility Guarantees + +- No additional required flags are introduced for standard local builds +- The resulting image remains suitable for runtime verification with the commands below + +### 2. Local runtime verification contract + +#### Commands + +```sh +make verify +make verify-amd64 +make verify-arm64 +``` + +#### Local Verification Expected Behavior + +- `make verify` builds the local image and runs runtime verification on the native platform +- `make verify-amd64` builds and verifies `linux/amd64` +- `make verify-arm64` builds and verifies `linux/arm64` +- Each command fails on the first unmet runtime assertion + +#### Required Assertions + +- `ruby -v` reports a Ruby `2.6.x` version line +- `ldd` on `libruby` shows jemalloc linkage +- `ldconfig -p` exposes jemalloc in the runtime image + +### 3. Direct script invocation contract + +#### Command Shape + +```sh +./scripts/verify-ruby-jemalloc.sh [platform] +``` + +#### Inputs + +- `image`: required Docker image reference +- `platform`: optional Docker platform value passed through to `docker run` + +#### Output Contract + +- Prints a `[verify] image=... platform=...` banner before checks begin +- Runs the three required runtime assertions inside the container +- Prints `[verify] ruby and jemalloc checks passed` only when all assertions succeed +- Exits non-zero on invalid usage or failed assertions + +### 4. CI workflow verification contract + +#### Workflow File + +- `.github/workflows/build.yml` + +#### Required Behavior + +- Runs a build matrix for `linux/amd64` and `linux/arm64` +- Uses GitHub Actions cache storage for Docker build layers +- Performs a verification image build with `--load` before runtime checks +- Invokes `./scripts/verify-ruby-jemalloc.sh` for each matrix entry +- Pushes registry images only for non-pull-request events + +## Invariants + +- Build args remain pinned to Ruby 2.6.10, OpenSSL 1.1.1w, and jemalloc 5.3.0 unless intentionally changed in spec, plan, and docs. +- Dockerfile here-doc blocks preserve command ordering and side effects from the pre-refactor behavior. +- Runtime verification semantics are shared between local contributor workflows and CI. +- Documentation updates in `README.md` and `specs/001-use-heredocs/quickstart.md` accompany any contract change. + +## Failure Conditions + +- Any non-zero command inside a here-doc block aborts the Docker build. +- Missing jemalloc linkage or missing linker cache registration causes verification failure. +- Architecture-specific failures in the CI matrix are considered contract failures even if another platform passes. + +## Evidence + +Verification evidence is recorded in: + +- `specs/001-use-heredocs/verification/us1-build.md` +- `specs/001-use-heredocs/verification/runtime-consistency.md` +- `specs/001-use-heredocs/verification/polish-checks.md` diff --git a/specs/001-use-heredocs/data-model.md b/specs/001-use-heredocs/data-model.md new file mode 100644 index 0000000..87957c2 --- /dev/null +++ b/specs/001-use-heredocs/data-model.md @@ -0,0 +1,92 @@ +# Data Model: Simplify Dockerfile RUN Commands with Here-Docs + +## Entity: Build Command Block + +**Purpose**: Represents one logical multi-step shell sequence inside the Dockerfile that contributors can review, edit, and troubleshoot as a single unit. + +### Build Command Block Fields + +- `name`: Human-readable identifier for the block, such as `builder-deps`, `openssl-build`, `jemalloc-build`, `ruby-build`, or `runtime-deps` +- `stage`: Docker stage where the block executes (`builder` or `runtime`) +- `shell_mode`: Required shell safety flags for the block, currently `set -eux` +- `ordered_steps`: Ordered list of shell operations contained in the here-doc block +- `inputs`: Build args, environment variables, packages, or source tarballs consumed by the block +- `outputs`: Installed binaries, libraries, config files, or cleaned directories produced by the block +- `cleanup_actions`: Cleanup work that must occur before the block completes successfully +- `failure_boundary`: The first command whose non-zero exit code aborts the block and the build + +### Build Command Block Validation Rules + +- A build command block MUST be expressed as a multi-line here-doc `RUN` block when it contains multiple sequential steps. +- A build command block MUST begin with `set -eux` so failures remain immediate and traceable. +- A build command block MUST preserve the pre-refactor execution order of dependent commands. +- Cleanup steps for package indexes or extracted build artifacts MUST remain in the same logical block as the work that created them. +- Comment labels SHOULD identify major steps to improve log traceability. + +### Build Command Block Relationships + +- A build command block contributes to one `Build Artifact`. +- A build command block can be referenced by one or more `Verification Record` entries when evidence is captured after a build. + +### Build Command Block State Transitions + +- `defined` -> `implemented` when the block exists in the Dockerfile with ordered steps. +- `implemented` -> `validated` when build and runtime verification succeed against the resulting image. +- `implemented` -> `failed` when a command exits non-zero or verification reveals a behavioral regression. + +## Entity: Build Artifact + +**Purpose**: Represents the container image output produced from the Dockerfile for a specific platform and set of pinned build inputs. + +### Build Artifact Fields + +- `image_tag`: Local or CI tag applied to the image, such as `ruby2.6-jemalloc:local` or `ruby2.6-jemalloc:ci-amd64` +- `platform`: Target platform, such as `linux/amd64` or `linux/arm64` +- `ruby_version`: Expected Ruby version line, currently `2.6.10` +- `openssl_version`: Expected OpenSSL source version, currently `1.1.1w` +- `jemalloc_version`: Expected jemalloc source version, currently `5.3.0` +- `source_locations`: Canonical download endpoints used during the build +- `runtime_paths`: Library locations that must be available at runtime, including `/opt/openssl/lib` and `/usr/local/lib` + +### Build Artifact Validation Rules + +- A build artifact MUST be produced from pinned build args documented in the Dockerfile, workflow, and quickstart. +- A build artifact MUST preserve Ruby 2.6 compatibility and jemalloc linkage. +- A build artifact MUST be reproducible for both supported platforms in CI. + +### Build Artifact Relationships + +- A build artifact is produced by multiple `Build Command Block` instances. +- A build artifact is evaluated by one or more `Verification Record` entries. + +## Entity: Verification Record + +**Purpose**: Captures repeatable evidence that a built image still satisfies runtime expectations after the Dockerfile refactor. + +### Verification Record Fields + +- `artifact_tag`: Reference to the built image under test +- `platform`: Platform used for the verification run +- `ruby_version_check`: Result of validating that `ruby -v` reports `2.6.x` +- `jemalloc_link_check`: Result of validating `libruby` linkage to jemalloc via `ldd` +- `linker_cache_check`: Result of validating `ldconfig -p` exposes jemalloc in the runtime image +- `execution_path`: Command path used to run verification, such as `make verify` or `./scripts/verify-ruby-jemalloc.sh` +- `status`: `planned`, `passed`, or `failed` +- `evidence_location`: Path to captured notes or verification artifact files under `specs/001-use-heredocs/verification/` + +### Verification Record Validation Rules + +- A verification record MUST capture all three runtime assertions: Ruby version, jemalloc linkage, and linker cache availability. +- A verification record MUST identify the tested image and platform. +- A failed check MUST leave enough command context to reproduce the failure locally or in CI. + +### Verification Record Relationships + +- A verification record validates one `Build Artifact`. +- A verification record provides evidence for the affected `Build Command Block` instances. + +### Verification Record State Transitions + +- `planned` -> `passed` when all required verification commands succeed. +- `planned` -> `failed` when any runtime assertion fails. +- `failed` -> `passed` after the artifact is rebuilt and all verification checks succeed. diff --git a/specs/001-use-heredocs/plan.md b/specs/001-use-heredocs/plan.md index 4c09b88..12e5a60 100644 --- a/specs/001-use-heredocs/plan.md +++ b/specs/001-use-heredocs/plan.md @@ -1,12 +1,12 @@ # Implementation Plan: Simplify Dockerfile RUN Commands with Here-Docs -**Branch**: `[001-use-heredocs]` | **Date**: 2026-06-05 | **Spec**: [spec.md](./spec.md) +**Branch**: `[001-use-heredocs]` | **Date**: 2026-06-09 | **Spec**: [spec.md](./spec.md) **Input**: Feature specification from `/specs/001-use-heredocs/spec.md` ## Summary -Refactor long multi-step `RUN` command chains in the Dockerfile into shell here-doc blocks to improve readability and maintainability while preserving image behavior. Keep Ruby 2.6 + jemalloc compatibility unchanged, and add explicit build/runtime verification guidance for local and CI execution. +Refactor long multi-step `RUN` command chains in the Dockerfile into shell here-doc blocks to improve readability and maintainability while preserving image behavior. Keep Ruby 2.6 + jemalloc compatibility unchanged, preserve deterministic build inputs, and document the verification contract shared by local contributors and CI. ## Technical Context @@ -38,6 +38,25 @@ Refactor long multi-step `RUN` command chains in the Dockerfile into shell here- - Required validation: PASS - plan includes local + CI build checks for amd64/arm64 and runtime validation. - Documentation impact: PASS - README and workflow notes are updated with the new here-doc style and verification steps. +## Phase 0 Research + +Research findings are captured in [research.md](./research.md). + +- Here-doc `RUN` blocks are the preferred representation for multi-step source-build flows because + they preserve order, improve editability, and make failures easier to localize. +- Determinism depends on preserving pinned versions, canonical source URLs, cleanup behavior, and + the OpenSSL -> jemalloc -> Ruby build order. +- Runtime verification must keep the existing three-part contract: Ruby version, jemalloc linkage, + and linker-cache visibility. + +No `NEEDS CLARIFICATION` items remain after research. + +## Phase 1 Design + +- Data model: [data-model.md](./data-model.md) +- Interface contract: [contracts/build-verification-contract.md](./contracts/build-verification-contract.md) +- Quickstart: [quickstart.md](./quickstart.md) + ## Project Structure ### Documentation (this feature) @@ -45,7 +64,13 @@ Refactor long multi-step `RUN` command chains in the Dockerfile into shell here- ```text specs/001-use-heredocs/ ├── plan.md +├── research.md +├── data-model.md +├── quickstart.md +├── contracts/ +│ └── build-verification-contract.md ├── spec.md +├── verification/ └── tasks.md ``` @@ -55,9 +80,24 @@ specs/001-use-heredocs/ Dockerfile README.md .github/workflows/build.yml +Makefile +scripts/verify-ruby-jemalloc.sh ``` -**Structure Decision**: Single repository with Docker artifact focus. Implementation is concentrated in `Dockerfile`; verification and contributor guidance are captured in `README.md` and CI workflow checks in `.github/workflows/build.yml`. +**Structure Decision**: Single repository with Docker artifact focus. Implementation is concentrated in `Dockerfile`; verification behavior is exposed through `Makefile`, `scripts/verify-ruby-jemalloc.sh`, and `.github/workflows/build.yml`; contributor guidance and evidence are captured in `README.md` and the feature spec directory. + +## Post-Design Constitution Check + +- Ruby runtime alignment: PASS - design artifacts keep Ruby 2.6.10 as the runtime target and do + not introduce migration work. +- jemalloc verification: PASS - the contract and data model require scriptable runtime checks for + linkage and linker-cache visibility. +- Build determinism: PASS - research and quickstart preserve pinned versions, source locations, + cleanup rules, and stage ordering. +- Required validation: PASS - quickstart and contract docs define local and CI verification paths + for amd64 and arm64. +- Documentation impact: PASS - the feature documents contract changes in the plan artifacts and + keeps README alignment explicit. ## Complexity Tracking diff --git a/specs/001-use-heredocs/quickstart.md b/specs/001-use-heredocs/quickstart.md index e9c62fa..b96285f 100644 --- a/specs/001-use-heredocs/quickstart.md +++ b/specs/001-use-heredocs/quickstart.md @@ -5,6 +5,12 @@ - Docker with Buildx support - Optional: QEMU emulation for cross-architecture runtime checks +## Reference Artifacts + +- Research decisions: `./research.md` +- Data model: `./data-model.md` +- Interface contract: `./contracts/build-verification-contract.md` + ## Local Build and Verification ```sh @@ -41,6 +47,19 @@ docker buildx build --platform linux/arm64 --load -t ruby2.6-jemalloc:ci-arm64 . ./scripts/verify-ruby-jemalloc.sh ruby2.6-jemalloc:ci-arm64 linux/arm64 ``` +## Expected Verification Outcomes + +- `ruby -v` reports a `2.6.x` Ruby version +- `ldd` on `libruby` shows jemalloc linkage +- `ldconfig -p` lists jemalloc in the runtime image + +## Contract Notes + +- Contributor-facing build and verification interfaces are defined in + `./contracts/build-verification-contract.md`. +- Any change to Dockerfile build args, verification semantics, or Make targets must update the + contract, README guidance, and verification evidence together. + ## Verification Artifacts - [runtime consistency](./verification/runtime-consistency.md) diff --git a/specs/001-use-heredocs/research.md b/specs/001-use-heredocs/research.md index 7dcb1cd..2dbd678 100644 --- a/specs/001-use-heredocs/research.md +++ b/specs/001-use-heredocs/research.md @@ -1,25 +1,96 @@ # Research Notes: Here-Doc Refactor Rules -## Decision +## Topic: Multi-Step Dockerfile Build Blocks -Use shell here-doc `RUN` blocks for multi-step Dockerfile sequences to improve readability and -reduce accidental breakage from line-continuation edits. +### Build Blocks Decision -## Shell Safety Conventions +Use shell here-doc `RUN` blocks for multi-step Dockerfile sequences across the OpenSSL, +jemalloc, Ruby, and runtime setup stages. -- Every here-doc block starts with `set -eux`. -- Keep command order identical to pre-refactor behavior. -- Include explicit step labels as comments for traceability in build logs. -- Keep cleanup commands in the same block as installation/build commands. -- Quote variable expansions where shell parsing could be ambiguous. +### Build Blocks Rationale -## Determinism Guardrails +- The Dockerfile contains several sequential source-build flows that are easier to review as + script-style blocks than as escaped `&&` chains. +- Here-doc blocks let maintainers change one step, flag, or cleanup action without rewriting + unrelated continuations. +- Inline step comments map build log output to a specific logical operation, which speeds up CI + and local debugging. +- Starting each block with `set -eux` preserves immediate failure behavior and command tracing. -- Do not change base images as part of this refactor. -- Do not modify versioned build args unless intentionally updating dependencies. -- Keep source URLs and archive extraction flow unchanged. - -## Rejected Alternative +### Build Blocks Alternatives Considered - Continue with long `&&` chains. - - Rejected because review and troubleshooting are harder, especially for multi-arch CI failures. + - Rejected because review, editing, and failure localization are harder, especially in the + multi-architecture CI matrix. +- Move build logic into separate shell scripts copied into the image. + - Rejected because it introduces extra indirection for a repository whose primary source of + truth is the Dockerfile itself. + +## Topic: Deterministic Build Inputs + +### Deterministic Inputs Decision + +Keep version pins, source download endpoints, build order, and cleanup behavior unchanged while +converting the Dockerfile structure. + +### Deterministic Inputs Rationale + +- Ruby 2.6.10, OpenSSL 1.1.1w, and jemalloc 5.3.0 are already pinned in the Dockerfile, workflow, + and contributor guidance. +- Ruby compilation depends on the previously installed OpenSSL and jemalloc outputs, so execution + order is a behavioral invariant rather than a formatting detail. +- Preserving `rm -rf /var/lib/apt/lists/*` and source tree cleanup in the same logical blocks + keeps image hygiene and layer contents stable. +- Reusing canonical download sources avoids introducing new dependency resolution behavior during + the refactor. + +### Deterministic Inputs Alternatives Considered + +- Float to newer or "latest" dependency versions during the refactor. + - Rejected because it would mix a readability change with a runtime compatibility change. +- Reorder compilation stages for perceived readability. + - Rejected because Ruby must build against the OpenSSL and jemalloc outputs already in place. + +## Topic: Runtime Verification Contract + +### Verification Contract Decision + +Retain a three-part runtime verification contract for every built image: Ruby version check, +jemalloc linkage check, and linker-cache visibility check. + +### Verification Contract Rationale + +- `ruby -v` confirms the runtime remains in the Ruby 2.6 line. +- `ldd` against `libruby` proves jemalloc is actually linked into the built runtime. +- `ldconfig -p` confirms the final image has the required runtime linker visibility. +- The same verification script is used locally and in CI, which keeps behavior reproducible across + contributors and the GitHub Actions matrix. + +### Verification Contract Alternatives Considered + +- Manual verification during review only. + - Rejected because the constitution requires scriptable, repeatable evidence. +- Performance benchmarks as the primary jemalloc proof. + - Rejected because they are noisy and unnecessary for establishing allocator activation. + +## Topic: Contributor and CI Workflow Impact + +### Workflow Impact Decision + +Keep contributor entry points centered on `make build`, `make verify`, `make verify-amd64`, +`make verify-arm64`, and the existing GitHub Actions build matrix. + +### Workflow Impact Rationale + +- Maintainers already have documented build and verification commands, so the refactor should not + require a new workflow to gain confidence. +- The workflow already uses Buildx and GHA cache storage, which provides a stable foundation for + validating the here-doc refactor on both target platforms. +- Preserving these interfaces focuses the feature on readability and troubleshooting rather than + operational retraining. + +### Workflow Impact Alternatives Considered + +- Introduce new wrapper tooling for build verification. + - Rejected because current Make targets and the verification script already provide a stable + public interface for both local and CI execution. diff --git a/specs/001-use-heredocs/tasks.md b/specs/001-use-heredocs/tasks.md index be8df64..d185979 100644 --- a/specs/001-use-heredocs/tasks.md +++ b/specs/001-use-heredocs/tasks.md @@ -2,83 +2,88 @@ **Input**: Design documents from `/specs/001-use-heredocs/` -**Prerequisites**: plan.md (required), spec.md (required for user stories) +**Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/, quickstart.md -**Tests**: No separate TDD test suite was explicitly requested. Verification tasks are included as build/runtime validation steps. +**Tests**: No standalone TDD suite was requested in the feature specification; runtime verification tasks are included as implementation evidence. -**Organization**: Tasks are grouped by user story to enable independent implementation and testing of each story. +**Organization**: Tasks are grouped by user story to preserve independent implementation and validation. ## Phase 1: Setup (Shared Infrastructure) -**Purpose**: Create shared verification and execution scaffolding for this refactor. +**Purpose**: Prepare shared build and verification scaffolding used by all stories. -- [x] T001 Create runtime verification script scaffold in scripts/verify-ruby-jemalloc.sh -- [x] T002 Add make targets for build and verification flow in Makefile -- [x] T003 [P] Create feature quickstart verification doc in specs/001-use-heredocs/quickstart.md +- [x] T001 Confirm pinned build args and stage comments in Dockerfile +- [x] T002 Align Make targets for local and architecture-specific verification in Makefile +- [x] T003 [P] Align direct verification script invocation contract in scripts/verify-ruby-jemalloc.sh +- [x] T004 [P] Record quickstart entry points and artifact references in specs/001-use-heredocs/quickstart.md --- ## Phase 2: Foundational (Blocking Prerequisites) -**Purpose**: Establish baseline rules and CI guardrails required by all user stories. +**Purpose**: Establish governance, contract, and CI guardrails required before story delivery. -**⚠️ CRITICAL**: No user story work should begin until this phase is complete. +**⚠️ CRITICAL**: User story implementation starts only after this phase is complete. -- [x] T004 Document here-doc style and shell safety conventions in specs/001-use-heredocs/research.md -- [x] T005 [P] Add CI matrix scaffold for linux/amd64 and linux/arm64 builds in .github/workflows/build.yml -- [x] T006 [P] Add deterministic build-input notes and validation steps in specs/001-use-heredocs/quickstart.md -- [x] T007 Add verification evidence template for consistency checks in specs/001-use-heredocs/verification/runtime-consistency.md +- [x] T005 Capture here-doc safety and determinism decisions in specs/001-use-heredocs/research.md +- [x] T006 [P] Finalize build entities and validation rules in specs/001-use-heredocs/data-model.md +- [x] T007 [P] Finalize contributor/CI interface contract in specs/001-use-heredocs/contracts/build-verification-contract.md +- [x] T008 Ensure buildx matrix, cache configuration, and verify steps align with contract in .github/workflows/build.yml +- [x] T009 Create verification evidence skeleton for US1 and US2 in specs/001-use-heredocs/verification/us1-build.md +- [x] T010 Create runtime consistency evidence skeleton in specs/001-use-heredocs/verification/runtime-consistency.md -**Checkpoint**: Foundation ready; story work can begin. +**Checkpoint**: Foundation is ready; stories can be implemented. --- ## Phase 3: User Story 1 - Maintainable Build Steps (Priority: P1) 🎯 MVP -**Goal**: Refactor long Dockerfile command chains into readable here-doc blocks while preserving command order. +**Goal**: Make multi-step build sequences easy to edit by using readable here-doc blocks with explicit step order. -**Independent Test**: Modify one build step in a here-doc block, run docker build, and confirm the image builds successfully without line-continuation rewrites. +**Independent Test**: Update one operation inside a here-doc build block and complete a successful image build without changing unrelated lines. ### Implementation for User Story 1 -- [x] T008 [US1] Refactor OpenSSL build command sequence to here-doc format in Dockerfile -- [x] T009 [US1] Refactor jemalloc build command sequence to here-doc format in Dockerfile -- [x] T010 [US1] Refactor Ruby build command sequence to here-doc format in Dockerfile -- [x] T011 [US1] Refactor runtime ldconfig configuration step to readable multi-line form in Dockerfile -- [x] T012 [US1] Capture US1 build verification commands and results in specs/001-use-heredocs/verification/us1-build.md +- [x] T011 [US1] Refactor OpenSSL build sequence into a structured here-doc block in Dockerfile +- [x] T012 [US1] Refactor jemalloc build sequence into a structured here-doc block in Dockerfile +- [x] T013 [US1] Refactor Ruby build and runtime ldconfig sequences into structured here-doc blocks in Dockerfile +- [x] T014 [P] [US1] Add per-step traceability comments for each major operation in Dockerfile +- [x] T015 [US1] Record build verification evidence for maintainability checks in specs/001-use-heredocs/verification/us1-build.md -**Checkpoint**: User Story 1 is independently functional and demonstrable. +**Checkpoint**: User Story 1 can be demonstrated and validated on its own. --- ## Phase 4: User Story 2 - Behavioral Consistency After Refactor (Priority: P2) -**Goal**: Prove runtime behavior remains equivalent after RUN command refactoring. +**Goal**: Prove runtime behavior remains equivalent after RUN command structure changes. -**Independent Test**: Run the same runtime checks before/after refactor and confirm Ruby version and jemalloc validation results remain equivalent. +**Independent Test**: Execute runtime checks pre/post refactor and confirm Ruby version and jemalloc activation outcomes are equivalent. ### Implementation for User Story 2 -- [x] T013 [US2] Implement Ruby/jemalloc runtime verification commands in scripts/verify-ruby-jemalloc.sh -- [x] T014 [US2] Document runtime verification procedure and expected outputs in README.md -- [x] T015 [US2] Execute and record before/after runtime consistency evidence in specs/001-use-heredocs/verification/runtime-consistency.md -- [x] T016 [US2] Add CI runtime verification execution step for both target architectures in .github/workflows/build.yml +- [x] T016 [US2] Implement Ruby 2.6, jemalloc linkage, and linker cache assertions in scripts/verify-ruby-jemalloc.sh +- [x] T017 [US2] Wire runtime verification commands to make targets in Makefile +- [x] T018 [US2] Run runtime verification for amd64 and arm64 in .github/workflows/build.yml +- [x] T019 [P] [US2] Document runtime verification procedure and expected outcomes in README.md +- [x] T020 [US2] Capture before/after runtime consistency evidence in specs/001-use-heredocs/verification/runtime-consistency.md -**Checkpoint**: User Story 2 is independently verifiable with documented runtime consistency evidence. +**Checkpoint**: User Story 2 is independently verifiable with repeatable evidence. --- ## Phase 5: User Story 3 - Faster Troubleshooting (Priority: P3) -**Goal**: Make build failures easier to localize and triage through better command structure and guidance. +**Goal**: Speed up build failure diagnosis with clearer block structure and triage guidance. -**Independent Test**: Trigger a controlled failure in one build block and identify the failing step directly from logs and troubleshooting guidance. +**Independent Test**: Trigger a controlled failure and identify the failing operation directly from logs and troubleshooting guidance. ### Implementation for User Story 3 -- [x] T017 [US3] Add explicit step labels/comments inside here-doc build blocks in Dockerfile -- [x] T018 [P] [US3] Add build-failure troubleshooting guide for here-doc blocks in README.md -- [x] T019 [P] [US3] Add CI/local triage command checklist in specs/001-use-heredocs/quickstart.md +- [x] T021 [US3] Add CI/local triage command sequence and failure interpretation guidance in specs/001-use-heredocs/quickstart.md +- [x] T022 [P] [US3] Add build-failure troubleshooting guide mapped to step labels in README.md +- [x] T023 [P] [US3] Add contract-level failure conditions and expected operator actions in specs/001-use-heredocs/contracts/build-verification-contract.md +- [x] T024 [US3] Record troubleshooting walkthrough and outcomes in specs/001-use-heredocs/verification/polish-checks.md **Checkpoint**: User Story 3 provides standalone troubleshooting value. @@ -86,11 +91,11 @@ ## Phase 6: Polish & Cross-Cutting Concerns -**Purpose**: Complete final validation and documentation alignment across stories. +**Purpose**: Final consistency pass across specs, docs, workflow, and verification artifacts. -- [x] T020 [P] Run repository checks and record outcomes in specs/001-use-heredocs/verification/polish-checks.md -- [x] T021 Verify documentation alignment with final Dockerfile behavior in README.md -- [x] T022 Verify all verification artifacts are present and linked in specs/001-use-heredocs/quickstart.md +- [x] T025 [P] Re-run full verification flow and record final pass criteria in specs/001-use-heredocs/verification/polish-checks.md +- [x] T026 Verify documentation linkage among plan, research, data model, contracts, and quickstart in specs/001-use-heredocs/plan.md +- [x] T027 Verify README and workflow reflect final runtime/build behavior in README.md --- @@ -98,38 +103,47 @@ ### Phase Dependencies -- Setup (Phase 1): No dependencies. -- Foundational (Phase 2): Depends on Setup completion. -- User Stories (Phases 3-5): Depend on Foundational completion. -- Polish (Phase 6): Depends on completion of all selected user stories. +- Setup (Phase 1): Starts immediately. +- Foundational (Phase 2): Depends on Phase 1 and blocks all user stories. +- User Stories (Phases 3-5): Depend on Phase 2 completion. +- Polish (Phase 6): Depends on completion of selected user stories. ### User Story Dependencies -- User Story 1 (P1): Starts first after Foundational; establishes primary Dockerfile refactor. -- User Story 2 (P2): Depends on User Story 1 refactor output for before/after behavior verification. -- User Story 3 (P3): Depends on User Story 1 structure; can overlap late with User Story 2 documentation tasks. - -### Within Each User Story - -- Update core implementation first. -- Run and capture verification evidence next. -- Update docs/guidance last. +- User Story 1 (P1): Starts first after Foundational; delivers the MVP refactor outcome. +- User Story 2 (P2): Depends on User Story 1 command-block structure to verify behavioral equivalence. +- User Story 3 (P3): Depends on User Story 1 step labeling and benefits from User Story 2 verification context. ### Parallel Opportunities -- T003 can run in parallel with T001-T002. -- T005 and T006 can run in parallel in Phase 2. -- T018 and T019 can run in parallel in User Story 3. -- T020 can run in parallel with final doc-alignment work. +- Setup: T003 and T004 can run in parallel after T001. +- Foundational: T006 and T007 can run in parallel while T008 progresses. +- User Story 1: T014 can run in parallel with T011-T013 once block boundaries are established. +- User Story 2: T019 can run in parallel with T016-T018. +- User Story 3: T022 and T023 can run in parallel after T021. +- Polish: T025 can run in parallel with T026 before final README/workflow alignment. --- +## Parallel Example: User Story 1 + +```bash +Task: "Add per-step traceability comments for each major operation in Dockerfile" +Task: "Refactor OpenSSL build sequence into a structured here-doc block in Dockerfile" +``` + +## Parallel Example: User Story 2 + +```bash +Task: "Document runtime verification procedure and expected outcomes in README.md" +Task: "Run runtime verification for amd64 and arm64 in .github/workflows/build.yml" +``` + ## Parallel Example: User Story 3 ```bash -# Parallel documentation and triage updates after Dockerfile labels are added: -Task: "Add build-failure troubleshooting guide for here-doc blocks in README.md" -Task: "Add CI/local triage command checklist in specs/001-use-heredocs/quickstart.md" +Task: "Add build-failure troubleshooting guide mapped to step labels in README.md" +Task: "Add contract-level failure conditions and expected operator actions in specs/001-use-heredocs/contracts/build-verification-contract.md" ``` --- @@ -141,25 +155,25 @@ Task: "Add CI/local triage command checklist in specs/001-use-heredocs/quickstar 1. Complete Phase 1 (Setup). 2. Complete Phase 2 (Foundational). 3. Complete Phase 3 (User Story 1). -4. Validate Docker image build and readability improvements. +4. Validate independent build/readability outcome before expanding scope. ### Incremental Delivery -1. Deliver User Story 1 for maintainability. -2. Add User Story 2 runtime equivalence evidence. -3. Add User Story 3 troubleshooting improvements. -4. Finalize Polish phase for release readiness. +1. Deliver User Story 1 to establish maintainable command blocks. +2. Deliver User Story 2 to prove runtime behavior consistency. +3. Deliver User Story 3 to improve troubleshooting speed. +4. Finish with cross-cutting polish and evidence consolidation. ### Parallel Team Strategy -1. One contributor refactors Dockerfile command blocks (US1). -2. Another contributor prepares verification docs/scripts once structure stabilizes (US2). -3. A third contributor improves troubleshooting docs and triage flow (US3). +1. Contributor A: Dockerfile refactor tasks (US1). +2. Contributor B: Verification script/workflow and evidence capture (US2). +3. Contributor C: Troubleshooting and contract documentation (US3). --- ## Notes -- Every task uses explicit file paths and actionable outcomes. -- Story labels map implementation work directly to user stories for traceability. -- Verification artifacts are treated as first-class deliverables for CI and review. +- All tasks follow the required checklist format with IDs, labels, and explicit file paths. +- Story phases are independently testable and sequenced by priority. +- Verification artifacts are first-class deliverables for CI and review gates. diff --git a/specs/001-use-heredocs/verification/polish-checks.md b/specs/001-use-heredocs/verification/polish-checks.md index d9f3540..479351a 100644 --- a/specs/001-use-heredocs/verification/polish-checks.md +++ b/specs/001-use-heredocs/verification/polish-checks.md @@ -10,13 +10,23 @@ make build-amd64 make verify-arm64 ``` +## Run Date + +- 2026-06-09 + ## Results -- lint: FAIL (repository-wide pre-existing YAML/secret-scan findings in `.specify/*`; unrelated to this feature's core behavior) +- lint: FAIL (pre-existing repository-wide findings in `.specify/*` extension metadata and secret-scan entropy checks) - verify: PASS - amd64 verify: PASS - arm64 verify: PASS +## Lint Failure Notes + +- `detect-secrets` flagged high-entropy strings in `.specify/integrations/*.manifest.json` (pre-existing generated integration metadata). +- `yamllint` and `ansible-lint` reported formatting/indentation issues in `.specify/*` workflow/extension YAML files that were not modified by this feature's runtime behavior updates. +- `end-of-file-fixer` adjusted `.gitignore` trailing newline. + ## Documentation Alignment - [x] README runtime verification instructions match Makefile targets diff --git a/specs/001-use-heredocs/verification/runtime-consistency.md b/specs/001-use-heredocs/verification/runtime-consistency.md index 5974357..c517161 100644 --- a/specs/001-use-heredocs/verification/runtime-consistency.md +++ b/specs/001-use-heredocs/verification/runtime-consistency.md @@ -5,12 +5,12 @@ Fill this table when executing runtime checks before/after Dockerfile refactor changes. | Check | Before | After | Status | -|------|--------|-------|--------| -| `ruby -v` reports `2.6.x` | not captured in this branch | pass (`2.6.10`) native/amd64/arm64 | pass | -| `ldd libruby` references jemalloc | not captured in this branch | pass native/amd64/arm64 | pass | -| `ldconfig -p` includes jemalloc | not captured in this branch | pass native/amd64/arm64 | pass | -| amd64 image build succeeds | not captured in this branch | pass (`make build-amd64`) | pass | -| arm64 image build succeeds | not captured in this branch | pass (`make verify-arm64`) | pass | +| ----- | ------ | ----- | ------ | +| `ruby -v` reports `2.6.x` | baseline requirement defined | pass (`2.6.10`) native/amd64/arm64 | pass | +| `ldd libruby` references jemalloc | baseline requirement defined | pass native/amd64/arm64 | pass | +| `ldconfig -p` includes jemalloc | baseline requirement defined | pass native/amd64/arm64 | pass | +| amd64 image build succeeds | baseline requirement defined | pass (`make build-amd64`) | pass | +| arm64 image build succeeds | baseline requirement defined | pass (`make verify-arm64`) | pass | ## Executed Commands @@ -22,6 +22,16 @@ make build-amd64 make verify-arm64 ``` +## Run Date + +- 2026-06-09 + +## Outcome Summary + +- Native verification passed (`ruby2.6-jemalloc:local`). +- amd64 verification passed (`ruby2.6-jemalloc:amd64`). +- arm64 verification passed (`ruby2.6-jemalloc:arm64`). + ## Notes - Use `./scripts/verify-ruby-jemalloc.sh ` for runtime checks. diff --git a/specs/001-use-heredocs/verification/us1-build.md b/specs/001-use-heredocs/verification/us1-build.md index d5af15a..18e505f 100644 --- a/specs/001-use-heredocs/verification/us1-build.md +++ b/specs/001-use-heredocs/verification/us1-build.md @@ -1,23 +1,29 @@ # US1 Build Verification +## Run Date + +- 2026-06-09 + ## Commands ```sh docker build -t ruby2.6-jemalloc:us1 . +docker build -t ruby2.6-jemalloc:local . ``` ## Result - PASS -- Equivalent validation command executed during implementation: - -```sh -docker build -t ruby2.6-jemalloc:local . -``` - - Build completed successfully with refactored here-doc RUN blocks. +- Core build sections validated as here-doc blocks with ordered, labeled steps: + - builder dependencies + - OpenSSL build + - jemalloc build + - Ruby build + - runtime ldconfig setup ## Notes - Validate that all major build sections now use here-doc style. - Confirm build completes without changing dependency versions. +- Pinned versions confirmed during run: Ruby 2.6.10, OpenSSL 1.1.1w, jemalloc 5.3.0. diff --git a/specs/002-multiarch-image-cache/checklists/requirements.md b/specs/002-multiarch-image-cache/checklists/requirements.md new file mode 100644 index 0000000..3c9c9f0 --- /dev/null +++ b/specs/002-multiarch-image-cache/checklists/requirements.md @@ -0,0 +1,34 @@ +# Specification Quality Checklist: Single Multi-Platform Image Build with Efficient Caching + +**Purpose**: Validate specification completeness and quality before proceeding to planning +**Created**: 2026-06-09 +**Feature**: [spec.md](../spec.md) + +## Content Quality + +- [x] No implementation details (languages, frameworks, APIs) +- [x] Focused on user value and business needs +- [x] Written for non-technical stakeholders +- [x] All mandatory sections completed + +## Requirement Completeness + +- [x] No [NEEDS CLARIFICATION] markers remain +- [x] Requirements are testable and unambiguous +- [x] Success criteria are measurable +- [x] Success criteria are technology-agnostic (no implementation details) +- [x] All acceptance scenarios are defined +- [x] Edge cases are identified +- [x] Scope is clearly bounded +- [x] Dependencies and assumptions identified + +## Feature Readiness + +- [x] All functional requirements have clear acceptance criteria +- [x] User scenarios cover primary flows +- [x] Feature meets measurable outcomes defined in Success Criteria +- [x] No implementation details leak into specification + +## Notes + +- Validation pass 1: all checklist items satisfied. diff --git a/specs/002-multiarch-image-cache/contracts/build-release-contract.md b/specs/002-multiarch-image-cache/contracts/build-release-contract.md new file mode 100644 index 0000000..fa0287e --- /dev/null +++ b/specs/002-multiarch-image-cache/contracts/build-release-contract.md @@ -0,0 +1,41 @@ +# Contract: Multi-Platform Build and Cache Reuse + +## Purpose + +Define the release-build behavior that must remain stable when the workflow is updated for multi-platform publishing and cache reuse. + +## Supported Interfaces + +### 1. Release publication contract + +#### Release Publication Expected Behavior + +- A release build publishes one image tag for both linux/amd64 and linux/arm64. +- The published tag resolves to the correct runnable variant for each platform. +- The release is not considered valid unless both variants are present. + +### 2. Cache reuse contract + +#### Cache Reuse Expected Behavior + +- Repeated builds should reuse unchanged work through Buildx/GHA cache import/export. +- Unchanged steps should not be rebuilt when the relevant inputs are identical. +- Build evidence should show cache reuse behavior for warm builds. + +### 3. Runtime verification contract + +#### Expected behavior + +- Runtime verification must pass for both platform variants before release eligibility. +- Existing Ruby and jemalloc checks remain the acceptance baseline. +- A failure on either platform blocks release publication. + +## Invariants + +- Ruby 2.6 runtime compatibility remains unchanged. +- Pinned source versions and deterministic build inputs remain documented. +- Documentation changes in README and quickstart must accompany workflow changes. + +## Failure Conditions + +- Missing platform variant, failed runtime verification, or cache configuration regression causes the release contract to fail. diff --git a/specs/002-multiarch-image-cache/data-model.md b/specs/002-multiarch-image-cache/data-model.md new file mode 100644 index 0000000..b1da01a --- /dev/null +++ b/specs/002-multiarch-image-cache/data-model.md @@ -0,0 +1,52 @@ +# Data Model: Single Multi-Platform Image Build with Efficient Caching + +## Entity: Multi-Platform Image Tag + +**Purpose**: Represents the single published release identifier that points to both target platform variants. + +### Multi-Platform Image Tag Fields + +- `tag_name`: Release tag name exposed to users +- `registry`: Registry hostname or namespace where the tag is published +- `platform_variants`: Set of supported platforms included in the manifest +- `publish_status`: Whether the tag was published successfully + +### Multi-Platform Image Tag Validation Rules + +- A multi-platform image tag MUST include both linux/amd64 and linux/arm64 variants. +- A tag MUST not be considered release-ready unless every required platform variant is present. + +## Entity: Platform Build Result + +**Purpose**: Captures the build and verification outcome for one target platform. + +### Platform Build Result Fields + +- `platform`: Target platform for the result +- `build_status`: Success or failure of the platform build +- `verification_status`: Success or failure of runtime verification +- `elapsed_time`: Time spent building or verifying the platform +- `image_reference`: Local or remote image reference used during verification + +### Platform Build Result Validation Rules + +- Each platform build result MUST be recorded separately. +- A failed build or verification MUST block publication of the release tag. + +## Entity: Build Cache Record + +**Purpose**: Records cache reuse behavior for one build run. + +### Fields + +- `build_id`: Identifier for the build run +- `cache_source`: Where reused build data came from +- `cache_hits`: Number or percentage of steps restored from cache +- `cache_misses`: Number or percentage of steps rebuilt +- `affected_steps`: Build steps that were rebuilt due to input changes + +### Validation Rules + +- Cache reuse MUST be measurable for repeated builds. +- Unchanged build steps SHOULD be restored from cache rather than rebuilt. +- A cache record MUST be associated with the build run that produced it. diff --git a/specs/002-multiarch-image-cache/plan.md b/specs/002-multiarch-image-cache/plan.md new file mode 100644 index 0000000..c5d4d87 --- /dev/null +++ b/specs/002-multiarch-image-cache/plan.md @@ -0,0 +1,95 @@ +# Implementation Plan: Single Multi-Platform Image Build with Efficient Caching + +**Branch**: `[002-multiarch-image-cache]` | **Date**: 2026-06-09 | **Spec**: [spec.md](./spec.md) + +**Input**: Feature specification from `/specs/002-multiarch-image-cache/spec.md` + +## Summary + +Consolidate the release workflow into a single multi-platform image publication flow that produces one tag for both `linux/amd64` and `linux/arm64`, while maximizing cache reuse to avoid rebuilding unchanged layers and preserving runtime verification coverage. + +## Technical Context + +**Language/Version**: Dockerfile syntax v1, Bash shell scripts, GitHub Actions workflow YAML + +**Primary Dependencies**: Docker Buildx, GitHub Actions cache, docker/build-push-action, docker/metadata-action, Docker registry publishing + +**Storage**: Container image layers and remote build cache (GitHub Actions cache) + +**Testing**: `docker buildx build`, `make verify`, `make verify-amd64`, `make verify-arm64`, release workflow runs on GitHub Actions + +**Target Platform**: Linux container images for `linux/amd64` and `linux/arm64` + +**Project Type**: Container build and release workflow repository + +**Performance Goals**: Reduce repeated-build time by maximizing cache hits and avoiding duplicate work across platform builds; preserve release verification pass rate + +**Constraints**: Preserve Ruby 2.6 runtime compatibility, keep jemalloc verification intact, maintain deterministic build inputs, ensure one published tag resolves to both platforms, and keep documentation aligned with workflow changes + +**Scale/Scope**: Single repository; primary changes in `.github/workflows/build.yml`, `Makefile`, `README.md`, and supporting verification notes under `specs/002-multiarch-image-cache/` + +## Constitution Check + +*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.* + +- Ruby runtime alignment: Plan preserves Ruby 2.6 runtime compatibility and does not alter the dependency line. +- jemalloc verification: Plan keeps runtime checks for jemalloc activation in place for both target platforms. +- Build determinism: Plan keeps pinned dependency versions and controlled build inputs while adding cache reuse. +- Required validation: Plan includes build and runtime verification for both architectures plus cache-behavior evidence. +- Documentation impact: Plan updates README and workflow guidance for the single-tag multi-platform process. + +## Phase 0 Research + +Research findings are captured in [research.md](./research.md). + +- One release tag should resolve to both architectures using a multi-platform manifest rather than separate user-facing tags. +- Build cache reuse should occur through Buildx/GHA cache so unchanged layers are restored instead of rebuilt. +- Runtime verification remains required for both architectures after publishing. + +No `NEEDS CLARIFICATION` items remain after research. + +## Phase 1 Design + +- Data model: [data-model.md](./data-model.md) +- Interface contract: [contracts/build-release-contract.md](./contracts/build-release-contract.md) +- Quickstart: [quickstart.md](./quickstart.md) + +## Project Structure + +### Documentation (this feature) + +```text +specs/002-multiarch-image-cache/ +├── plan.md +├── research.md +├── data-model.md +├── quickstart.md +├── contracts/ +│ └── build-release-contract.md +├── spec.md +└── tasks.md +``` + +### Source Code (repository root) + +```text +Dockerfile +README.md +.github/workflows/build.yml +Makefile +scripts/verify-ruby-jemalloc.sh +``` + +**Structure Decision**: Single repository with container build/release focus. The workflow is centered on `.github/workflows/build.yml` for release publication and cache reuse, `Makefile` for contributor entry points, `scripts/verify-ruby-jemalloc.sh` for runtime validation, and `README.md` plus the feature docs for behavior and operational guidance. + +## Post-Design Constitution Check + +- Ruby runtime alignment: PASS - design keeps Ruby 2.6 runtime compatibility unchanged. +- jemalloc verification: PASS - runtime checks remain required for both platform variants. +- Build determinism: PASS - pinned versions and explicit inputs remain in place; caching is additive. +- Required validation: PASS - quickstart and contract docs define local and CI verification paths for amd64 and arm64. +- Documentation impact: PASS - release workflow and user guidance will be documented together. + +## Complexity Tracking + +No constitution violations expected for this feature. diff --git a/specs/002-multiarch-image-cache/quickstart.md b/specs/002-multiarch-image-cache/quickstart.md new file mode 100644 index 0000000..499c404 --- /dev/null +++ b/specs/002-multiarch-image-cache/quickstart.md @@ -0,0 +1,35 @@ +# Quickstart: Multi-Platform Release Build with Efficient Caching + +## Prerequisites + +- Docker with Buildx support +- Access to a registry that can publish multi-platform images +- GitHub Actions cache or equivalent remote cache support for repeated builds + +## Build and Verify Locally + +```sh +make build +make verify +``` + +## Multi-Platform Verification + +```sh +make build-amd64 +make verify-amd64 +make build-arm64 +make verify-arm64 +``` + +## Expected Outcomes + +- One published tag resolves to both amd64 and arm64 variants. +- Warm builds reuse cached work when inputs are unchanged. +- Runtime verification passes for both platforms. + +## Evidence To Capture + +- Build logs showing cache reuse on repeated runs +- Release workflow logs showing one tag published for both variants +- Runtime verification output for amd64 and arm64 diff --git a/specs/002-multiarch-image-cache/research.md b/specs/002-multiarch-image-cache/research.md new file mode 100644 index 0000000..20f274e --- /dev/null +++ b/specs/002-multiarch-image-cache/research.md @@ -0,0 +1,53 @@ +# Research Notes: Single Multi-Platform Image Build with Efficient Caching + +## Topic: Single Published Image Tag + +### Single Tag Decision + +Publish one release tag that resolves to both linux/amd64 and linux/arm64 variants through a multi-platform manifest. + +### Single Tag Rationale + +- Users should not need to choose architecture-specific tags. +- A single tag keeps release documentation and usage simpler. +- The workflow already supports GitHub Container Registry publishing, which is compatible with manifest-based multi-platform images. + +### Single Tag Alternatives Considered + +- Separate architecture-specific tags. + - Rejected because it increases user confusion and creates unnecessary release surface area. + +## Topic: Cache Reuse Strategy + +### Cache Strategy Decision + +Use Buildx cache import/export with shared remote cache storage so unchanged layers can be reused across repeated builds. + +### Cache Strategy Rationale + +- Remote cache reuse reduces duplicate work between cold and warm builds. +- GitHub Actions cache support is already present in the repository workflow style. +- Cache reuse must not change build outputs; it only changes how existing work is retrieved. + +### Cache Strategy Alternatives Considered + +- Build each platform fully from scratch on every run. + - Rejected because it wastes CI time and increases release latency. +- Rely only on local builder cache. + - Rejected because CI runners are ephemeral and need remote cache persistence. + +## Topic: Runtime Verification Preservation + +### Verification Decision + +Keep runtime verification as a release gate for both amd64 and arm64 outputs after the multi-platform image is published. + +### Verification Rationale + +- Multi-platform publishing is only acceptable if the resulting image still satisfies Ruby and jemalloc expectations. +- Verification needs to be repeatable for both target platforms. + +### Verification Alternatives Considered + +- Verify only one platform and assume the other matches. + - Rejected because platform-specific builds can diverge. diff --git a/specs/002-multiarch-image-cache/spec.md b/specs/002-multiarch-image-cache/spec.md new file mode 100644 index 0000000..b3598ed --- /dev/null +++ b/specs/002-multiarch-image-cache/spec.md @@ -0,0 +1,104 @@ +# Feature Specification: Single Multi-Platform Image Build with Efficient Caching + +**Feature Branch**: `[002-multiarch-image-cache]` + +**Created**: 2026-06-09 + +**Status**: Draft + +**Input**: User description: "the build process should create a single docker image suitable for running on linux/amd64 and linux/arm64 platforms. The build process should be as efficient as possible, using caching to avoid duplication of effort." + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Publish One Multi-Platform Image (Priority: P1) + +As a maintainer, I want each release build to publish one image tag that works on both linux/amd64 and linux/arm64 so users can pull a single tag regardless of architecture. + +**Why this priority**: A single tag across platforms is the primary user-facing outcome and removes manual tag selection. + +**Independent Test**: Run one release build, then pull the same tag on amd64 and arm64 environments and confirm each environment receives a runnable variant. + +**Acceptance Scenarios**: + +1. **Given** a release build is triggered, **When** it completes successfully, **Then** one image tag is available with both amd64 and arm64 variants. +2. **Given** a user pulls the release tag from an amd64 host, **When** the image is resolved, **Then** the host receives a working amd64 variant. +3. **Given** a user pulls the same release tag from an arm64 host, **When** the image is resolved, **Then** the host receives a working arm64 variant. + +--- + +### User Story 2 - Reduce Rebuild Time Through Cache Reuse (Priority: P2) + +As a release engineer, I want repeated builds to reuse prior build work so unchanged layers are not rebuilt and delivery time is reduced. + +**Why this priority**: Faster builds reduce infrastructure cost and improve release turnaround. + +**Independent Test**: Execute a cold build and then a warm build with no relevant source changes; verify the second run completes significantly faster and reports cache reuse. + +**Acceptance Scenarios**: + +1. **Given** a prior successful build exists, **When** the same commit is rebuilt, **Then** unchanged build steps are restored from cache instead of rebuilt. +2. **Given** only one component changes, **When** a build runs, **Then** cache is reused for unaffected components and only impacted steps are rebuilt. + +--- + +### User Story 3 - Preserve Runtime Verification Across Platforms (Priority: P3) + +As a maintainer, I want runtime checks to pass for both amd64 and arm64 outputs so performance optimizations do not compromise runtime expectations. + +**Why this priority**: Caching and multi-platform publishing must not break Ruby or allocator behavior. + +**Independent Test**: After publishing the multi-platform image, run runtime verification checks for both architectures and confirm all checks pass. + +**Acceptance Scenarios**: + +1. **Given** a multi-platform build is completed, **When** runtime verification is executed on amd64 and arm64 variants, **Then** required runtime checks pass on both. +2. **Given** one platform verification fails, **When** release eligibility is evaluated, **Then** the build is treated as failed and not considered release-ready. + +### Edge Cases + +- One platform build fails while the other succeeds; ensure no incomplete release is published. +- Cache entries are unavailable or expired; ensure the build still succeeds through full rebuild behavior. +- Dependency version updates invalidate cache; ensure rebuild scope is limited to impacted layers. +- Manifest publication succeeds but one platform image is missing; ensure validation detects and fails this state. + +## Requirements *(mandatory)* + +### Functional Requirements + +- **FR-001**: The system MUST produce a single release image tag that contains both linux/amd64 and linux/arm64 variants. +- **FR-002**: The system MUST allow consumers on both target platforms to pull the same image tag and receive the correct runnable variant. +- **FR-003**: The system MUST reuse cached build work across repeated runs when relevant inputs have not changed. +- **FR-004**: The system MUST minimize duplicated work by rebuilding only cache-invalidated steps. +- **FR-005**: The system MUST fail release publication if any required platform variant fails build or verification. +- **FR-006**: The system MUST provide build evidence showing cache usage behavior and platform-specific build outcomes. +- **FR-007**: The system MUST preserve required runtime verification outcomes for both target platforms after optimization changes. +- **FR-008**: The system MUST document the single-tag multi-platform workflow and cache behavior for maintainers. + +### Container & Runtime Constraints *(mandatory)* + +- Feature work MUST preserve Ruby 2.6 runtime compatibility unless an explicit migration decision is approved. +- Feature work affecting memory allocator behavior MUST define how jemalloc usage is verified. +- Feature work MUST define deterministic build input expectations (base image, package sources, and build args). +- Feature work MUST identify documentation updates required in README or operational guidance. + +### Key Entities *(include if feature involves data)* + +- **Multi-Platform Image Tag**: A single published image identifier that references both amd64 and arm64 runnable variants. +- **Platform Build Result**: Per-platform outcome record including build status, verification status, and elapsed time. +- **Build Cache Record**: Evidence describing cache hit/miss behavior and which build steps were reused versus rebuilt. + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: 100% of successful release builds publish one image tag containing both linux/amd64 and linux/arm64 variants. +- **SC-002**: In repeated builds with unchanged inputs, median warm-build duration is at least 30% faster than median cold-build duration. +- **SC-003**: At least 80% of unchanged build steps are restored from cache during warm builds. +- **SC-004**: Required runtime verification checks pass for both platform variants in 100% of release-eligible builds. + +## Assumptions + +- The target registry supports publishing and resolving one image tag across multiple platforms. +- Build infrastructure supports persistent cache storage between runs. +- Existing runtime verification checks remain the acceptance baseline for Ruby and allocator behavior. +- Platform support scope for this feature is limited to linux/amd64 and linux/arm64. diff --git a/specs/002-multiarch-image-cache/tasks.md b/specs/002-multiarch-image-cache/tasks.md new file mode 100644 index 0000000..b05d073 --- /dev/null +++ b/specs/002-multiarch-image-cache/tasks.md @@ -0,0 +1,175 @@ +# Tasks: Single Multi-Platform Image Build with Efficient Caching + +**Input**: Design documents from `/specs/002-multiarch-image-cache/` + +**Prerequisites**: plan.md (required), spec.md (required for user stories), research.md, data-model.md, contracts/, quickstart.md + +**Tests**: No standalone TDD suite was requested in the feature specification; verification tasks are included as build, cache, and runtime evidence. + +**Organization**: Tasks are grouped by user story to preserve independent implementation and validation. + +## Phase 1: Setup (Shared Infrastructure) + +**Purpose**: Prepare shared workflow, build, and contributor scaffolding for the multi-platform release flow. + +- [ ] T001 Update release workflow structure for multi-platform publication in .github/workflows/build.yml +- [ ] T002 [P] Add cache-enabled Buildx build settings for release jobs in .github/workflows/build.yml +- [ ] T003 [P] Add local multi-platform build and verify targets in Makefile +- [ ] T004 [P] Record contributor-facing single-tag release guidance in README.md + +--- + +## Phase 2: Foundational (Blocking Prerequisites) + +**Purpose**: Establish design, contract, and evidence scaffolding required before user story delivery. + +**⚠️ CRITICAL**: No user story work should begin until this phase is complete. + +- [ ] T005 Capture multi-platform manifest and cache reuse decisions in specs/002-multiarch-image-cache/research.md +- [ ] T006 [P] Finalize build result and cache record entities in specs/002-multiarch-image-cache/data-model.md +- [ ] T007 [P] Finalize single-tag release and cache reuse contract in specs/002-multiarch-image-cache/contracts/build-release-contract.md +- [ ] T008 Document cold-build and warm-build validation flow in specs/002-multiarch-image-cache/quickstart.md +- [ ] T009 Create release-manifest evidence template in specs/002-multiarch-image-cache/verification/release-manifest.md +- [ ] T010 Create cache-reuse evidence template in specs/002-multiarch-image-cache/verification/cache-reuse.md + +**Checkpoint**: Foundation ready; story work can begin. + +--- + +## Phase 3: User Story 1 - Publish One Multi-Platform Image (Priority: P1) 🎯 MVP + +**Goal**: Publish one release tag that resolves to both linux/amd64 and linux/arm64 variants. + +**Independent Test**: Trigger one release build and confirm the same tag resolves to a working variant on both amd64 and arm64 hosts. + +### Implementation for User Story 1 + +- [ ] T011 [US1] Publish one manifest-backed image tag for both amd64 and arm64 in .github/workflows/build.yml +- [ ] T012 [US1] Validate release publication includes both platform variants in .github/workflows/build.yml +- [ ] T013 [US1] Record single-tag publication evidence in specs/002-multiarch-image-cache/verification/release-manifest.md + +**Checkpoint**: User Story 1 can be demonstrated and validated independently. + +--- + +## Phase 4: User Story 2 - Reduce Rebuild Time Through Cache Reuse (Priority: P2) + +**Goal**: Reuse prior build work so unchanged layers are not rebuilt on repeated runs. + +**Independent Test**: Run a cold build and a warm build with unchanged inputs and confirm cache reuse reduces duplicated work. + +### Implementation for User Story 2 + +- [ ] T014 [US2] Enable remote cache import/export for repeated builds in .github/workflows/build.yml +- [ ] T015 [US2] Add cache-aware local build guidance and targets in Makefile +- [ ] T016 [US2] Record warm-build and cache-hit evidence in specs/002-multiarch-image-cache/verification/cache-reuse.md + +**Checkpoint**: User Story 2 is independently verifiable with cache evidence. + +--- + +## Phase 5: User Story 3 - Preserve Runtime Verification Across Platforms (Priority: P3) + +**Goal**: Keep Ruby and jemalloc verification passing for both amd64 and arm64 outputs. + +**Independent Test**: After publishing the multi-platform image, run runtime verification for both platforms and confirm all checks pass. + +### Implementation for User Story 3 + +- [ ] T017 [US3] Keep runtime verification checks aligned with both target platforms in scripts/verify-ruby-jemalloc.sh +- [ ] T018 [US3] Wire runtime verification into the release workflow for both platforms in .github/workflows/build.yml +- [ ] T019 [P] [US3] Document runtime verification and release-gating behavior in README.md +- [ ] T020 [US3] Record runtime verification evidence for both platform variants in specs/002-multiarch-image-cache/verification/runtime-consistency.md + +**Checkpoint**: User Story 3 provides standalone runtime confidence for the release flow. + +--- + +## Phase 6: Polish & Cross-Cutting Concerns + +**Purpose**: Final alignment across docs, workflow, and evidence artifacts. + +- [ ] T021 [P] Validate quickstart examples against final workflow commands in specs/002-multiarch-image-cache/quickstart.md +- [ ] T022 Verify README release and cache guidance matches workflow behavior in README.md +- [ ] T023 Verify all verification artifacts are linked from specs/002-multiarch-image-cache/quickstart.md + +--- + +## Dependencies & Execution Order + +### Phase Dependencies + +- Setup (Phase 1): Starts immediately. +- Foundational (Phase 2): Depends on Phase 1 and blocks all user stories. +- User Stories (Phases 3-5): Depend on Phase 2 completion. +- Polish (Phase 6): Depends on completion of selected user stories. + +### User Story Dependencies + +- User Story 1 (P1): Starts first after Foundational; establishes the single-tag multi-platform release path. +- User Story 2 (P2): Builds on the workflow foundation to add cache reuse and warm-build efficiency. +- User Story 3 (P3): Depends on the release workflow and cache strategy to validate runtime behavior across both platforms. + +### Parallel Opportunities + +- Setup: T002 and T003 can run in parallel once T001 is in place. +- Foundational: T006 and T007 can run in parallel while T008 progresses. +- User Story 1: T012 and T013 can run in parallel after the manifest publication step is defined. +- User Story 2: T015 can run in parallel with T014 once cache strategy is agreed. +- User Story 3: T019 can run in parallel with T017 and T018 after runtime contract details are stable. +- Polish: T021 can run in parallel with T022 before final documentation linkage is confirmed. + +--- + +## Parallel Example: User Story 1 + +```bash +Task: "Validate release publication includes both platform variants in .github/workflows/build.yml" +Task: "Record single-tag publication evidence in specs/002-multiarch-image-cache/verification/release-manifest.md" +``` + +## Parallel Example: User Story 2 + +```bash +Task: "Enable remote cache import/export for repeated builds in .github/workflows/build.yml" +Task: "Add cache-aware local build guidance and targets in Makefile" +``` + +## Parallel Example: User Story 3 + +```bash +Task: "Document runtime verification and release-gating behavior in README.md" +Task: "Record runtime verification evidence for both platform variants in specs/002-multiarch-image-cache/verification/runtime-consistency.md" +``` + +--- + +## Implementation Strategy + +### MVP First (User Story 1 Only) + +1. Complete Phase 1 (Setup). +2. Complete Phase 2 (Foundational). +3. Complete Phase 3 (User Story 1). +4. Validate that one tag publishes both platform variants before expanding scope. + +### Incremental Delivery + +1. Deliver User Story 1 for single-tag multi-platform publication. +2. Add User Story 2 to reduce warm-build time with cache reuse. +3. Add User Story 3 to preserve runtime verification across platforms. +4. Finish with polish tasks and linked evidence artifacts. + +### Parallel Team Strategy + +1. Contributor A: Release workflow and manifest publication tasks (US1). +2. Contributor B: Cache reuse workflow and build guidance tasks (US2). +3. Contributor C: Runtime verification and documentation tasks (US3). + +--- + +## Notes + +- All tasks follow the required checklist format with IDs, labels, and explicit file paths. +- Story phases are independently testable and sequenced by priority. +- Verification artifacts are first-class deliverables for CI and review gates. From f93448231a37e08837edfb66d4ec364ae99f7016 Mon Sep 17 00:00:00 2001 From: John Trammell Date: Tue, 9 Jun 2026 16:21:03 -0500 Subject: [PATCH 2/3] autoapprove more shell commands --- .vscode/settings.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.vscode/settings.json b/.vscode/settings.json index 388adce..d85be24 100644 --- a/.vscode/settings.json +++ b/.vscode/settings.json @@ -8,6 +8,7 @@ }, "chat.tools.terminal.autoApprove": { ".specify/scripts/bash/": true, - ".specify/scripts/powershell/": true + ".specify/scripts/powershell/": true, + "printf": true } } From 00d82d95506017acad6270d74b9638fd0a7b0f4f Mon Sep 17 00:00:00 2001 From: John Trammell Date: Wed, 10 Jun 2026 09:14:34 -0500 Subject: [PATCH 3/3] [Spec Kit] Implementation progress --- .github/copilot-instructions.md | 39 ++++-- .github/workflows/build.yml | 114 ++++++++++++++---- Makefile | 22 +++- README.md | 32 ++++- specs/002-multiarch-image-cache/quickstart.md | 21 +++- specs/002-multiarch-image-cache/tasks.md | 46 +++---- .../verification/cache-reuse.md | 24 ++++ .../verification/release-manifest.md | 23 ++++ .../verification/runtime-consistency.md | 24 ++++ 9 files changed, 279 insertions(+), 66 deletions(-) create mode 100644 specs/002-multiarch-image-cache/verification/cache-reuse.md create mode 100644 specs/002-multiarch-image-cache/verification/release-manifest.md create mode 100644 specs/002-multiarch-image-cache/verification/runtime-consistency.md diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index e482f08..3d82a8c 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -4,14 +4,33 @@ shell commands, and other important information, read specs/002-multiarch-image- -* The purpose of this project is to build a Docker container running the latest 2.6.x Ruby - version, with jemalloc. -* The target architecture of this a multi-architecture image supporting `linux/amd64` and - `linux/arm64` architectures. -* The image will be built using a multistage Dockerfile that compiles OpenSSL 1.1.1w, - jemalloc 5.3.0, and Ruby 2.6.10 from source. -* The resulting image will have a minimal runtime footprint, containing only the compiled - Ruby binaries and necessary runtime libraries. -* The build process will be structured with shell here-doc `RUN` blocks for clarity and - maintainability. + +# Instructions + +The purpose of this project is to build a Docker container image providing the latest version of Ruby +2.6.x, including the `jemalloc` implementation of `malloc` for improved memory performance. To support +development and testing, the resulting container will be multi-architecture, supporting both +`linux/amd64` and `linux/arm64` architectures. + +The project will use `dependabot` to keep the pinned versions of Ruby, OpenSSL, and jemalloc up to date. + +## Docker Specification + +The resulting Docker image will have a minimal runtime footprint, using a multi-stage build to limit +image contents to only the compiled Ruby binaries and necessary runtime libraries. + +The `Dockerfile` will be formatted with "here-doc" `RUN` blocks for clarity and maintainability. + +The image will provide the latest compatible versions of its dependencies: + +* Ruby 2.6.x (currently 2.6.10) +* OpenSSL 1.1.x (currently 1.1.1w) +* jemalloc 5.3.x (currently 5.3.0) + +## CICD + +* The CI workflow will be implemented with GitHub Actions, using `Buildx` for multi-architecture + builds and cache management. +* Use the `pre-commit` toolchain for local development to ensure code quality and consistency. + \ No newline at end of file diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index b627c1b..459078b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -11,18 +11,12 @@ on: env: REGISTRY: ghcr.io IMAGE_NAME: ${{ github.repository }} + MULTIARCH_SCOPE: multiarch-image-cache jobs: - build-and-push: + release: + if: github.event_name != 'pull_request' runs-on: ubuntu-latest - strategy: - fail-fast: false - matrix: - include: - - arch: amd64 - platform: linux/amd64 - - arch: arm64 - platform: linux/arm64 permissions: contents: read packages: write @@ -38,7 +32,6 @@ jobs: uses: docker/setup-qemu-action@v3 - name: Log in to GitHub Container Registry - if: github.event_name != 'pull_request' uses: docker/login-action@v3 with: registry: ${{ env.REGISTRY }} @@ -61,30 +54,99 @@ jobs: uses: docker/build-push-action@v6 with: context: . - platforms: ${{ matrix.platform }} - push: ${{ github.event_name != 'pull_request' }} + platforms: linux/amd64,linux/arm64 + push: true tags: ${{ steps.meta.outputs.tags }} labels: ${{ steps.meta.outputs.labels }} - cache-from: type=gha - cache-to: type=gha,mode=max + cache-from: type=gha,scope=${{ env.MULTIARCH_SCOPE }} + cache-to: type=gha,mode=max,scope=${{ env.MULTIARCH_SCOPE }} build-args: | RUBY_VERSION=2.6.10 OPENSSL_VERSION=1.1.1w JEMALLOC_VERSION=5.3.0 - - name: Build verification image (load) + verify-release: + if: github.event_name != 'pull_request' + needs: release + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + include: + - arch: amd64 + platform: linux/amd64 + - arch: arm64 + platform: linux/arm64 + permissions: + contents: read + packages: read + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Set up QEMU + uses: docker/setup-qemu-action@v3 + + - name: Log in to GitHub Container Registry + uses: docker/login-action@v3 + with: + registry: ${{ env.REGISTRY }} + username: ${{ github.actor }} + password: ${{ secrets.GITHUB_TOKEN }} + + - name: Resolve release image reference + run: echo "RELEASE_IMAGE=${REGISTRY}/${IMAGE_NAME}:sha-${GITHUB_SHA}" >> "$GITHUB_ENV" + + - name: Pull release image + run: docker pull "$RELEASE_IMAGE" + + - name: Verify Ruby and jemalloc runtime run: | - docker buildx build \ - --platform "${{ matrix.platform }}" \ - --build-arg RUBY_VERSION=2.6.10 \ - --build-arg OPENSSL_VERSION=1.1.1w \ - --build-arg JEMALLOC_VERSION=5.3.0 \ - --load \ - -t "ruby2.6-jemalloc:ci-${{ matrix.arch }}" \ - . + ./scripts/verify-ruby-jemalloc.sh "$RELEASE_IMAGE" "${{ matrix.platform }}" + + verify-pr: + if: github.event_name == 'pull_request' + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + include: + - arch: amd64 + platform: linux/amd64 + - arch: arm64 + platform: linux/arm64 + permissions: + contents: read + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Set up Docker Buildx + uses: docker/setup-buildx-action@v3 + + - name: Set up QEMU + uses: docker/setup-qemu-action@v3 + + - name: Build verification image (load) + uses: docker/build-push-action@v6 + with: + context: . + platforms: ${{ matrix.platform }} + load: true + push: false + tags: ruby2.6-jemalloc:ci-${{ matrix.arch }} + cache-from: type=gha,scope=${{ env.MULTIARCH_SCOPE }} + cache-to: type=gha,mode=max,scope=${{ env.MULTIARCH_SCOPE }}-pr-${{ matrix.arch }} + build-args: | + RUBY_VERSION=2.6.10 + OPENSSL_VERSION=1.1.1w + JEMALLOC_VERSION=5.3.0 - name: Verify Ruby and jemalloc runtime run: | - ./scripts/verify-ruby-jemalloc.sh \ - "ruby2.6-jemalloc:ci-${{ matrix.arch }}" \ - "${{ matrix.platform }}" + ./scripts/verify-ruby-jemalloc.sh "ruby2.6-jemalloc:ci-${{ matrix.arch }}" "${{ matrix.platform }}" diff --git a/Makefile b/Makefile index 0bd4c1c..b033f8a 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,10 @@ usage: - @echo "usage: make [usage|lint|build|build-amd64|build-arm64|verify|verify-amd64|verify-arm64]" + @echo "usage: make [usage|lint|build|build-amd64|build-arm64|build-release|verify|verify-amd64|verify-arm64|verify-release]" + +REGISTRY ?= ghcr.io/umnlibraries/ruby2.6-jemalloc-docker +MULTIARCH_TAG ?= sha-$(shell git rev-parse --short HEAD 2>/dev/null || echo local) +MULTIARCH_PLATFORMS ?= linux/amd64,linux/arm64 +MULTIARCH_CACHE_DIR ?= .buildx-cache lint: pre-commit run --all-file @@ -13,6 +18,17 @@ build-amd64: build-arm64: docker buildx build --platform linux/arm64 --load -t ruby2.6-jemalloc:arm64 . +build-release: + docker buildx build \ + --platform $(MULTIARCH_PLATFORMS) \ + --cache-from type=local,src=$(MULTIARCH_CACHE_DIR) \ + --cache-to type=local,dest=$(MULTIARCH_CACHE_DIR)-new,mode=max \ + --push \ + -t $(REGISTRY):$(MULTIARCH_TAG) \ + . + @rm -rf $(MULTIARCH_CACHE_DIR) + @mv $(MULTIARCH_CACHE_DIR)-new $(MULTIARCH_CACHE_DIR) + verify: build ./scripts/verify-ruby-jemalloc.sh ruby2.6-jemalloc:local @@ -21,3 +37,7 @@ verify-amd64: build-amd64 verify-arm64: build-arm64 ./scripts/verify-ruby-jemalloc.sh ruby2.6-jemalloc:arm64 linux/arm64 + +verify-release: build-release + ./scripts/verify-ruby-jemalloc.sh $(REGISTRY):$(MULTIARCH_TAG) linux/amd64 + ./scripts/verify-ruby-jemalloc.sh $(REGISTRY):$(MULTIARCH_TAG) linux/arm64 diff --git a/README.md b/README.md index 7e074d4..06b9752 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,22 @@ Long Docker build sequences are expressed with shell here-doc `RUN` blocks so ea is easier to review and maintain. This keeps ordering explicit while avoiding fragile line continuation chains. +## Multi-Platform Release and Caching + +The release workflow publishes one image tag that resolves to both `linux/amd64` and +`linux/arm64` variants. GitHub Actions uses Buildx cache import/export so unchanged work can be +reused across repeated builds. + +Local contributors can mirror the release workflow with: + +```sh +make build-release +make verify-release +``` + +`make build-release` publishes the multi-platform image tag and preserves cache in `.buildx-cache` +between runs. `make verify-release` pulls the published tag and verifies both architectures. + ## Usage ### Pull from GitHub Container Registry @@ -63,6 +79,13 @@ make verify-amd64 make verify-arm64 ``` +For the multi-platform release flow, use: + +```sh +make build-release +make verify-release +``` + The verification script checks: - Ruby reports `2.6.x` @@ -71,7 +94,10 @@ The verification script checks: ## CI/CD -GitHub Actions builds the image on every push to `main` and publishes it to the [GitHub Container Registry](https://ghcr.io/umnlibraries/ruby2.6-jemalloc-docker). Pull requests trigger a build-only run (no push) to validate the `Dockerfile`. +GitHub Actions publishes one multi-platform image tag on pushes to `main` and manual release +runs. Pull requests run per-platform verification builds with cache reuse enabled so changes are +validated without publishing. -CI runs a build matrix for `linux/amd64` and `linux/arm64` and executes runtime verification for -each platform. +The release job publishes a single manifest-backed tag for `linux/amd64` and `linux/arm64`, while +the verification job checks the published tag on `main` and uses cached per-platform build/load +runs on pull requests. diff --git a/specs/002-multiarch-image-cache/quickstart.md b/specs/002-multiarch-image-cache/quickstart.md index 499c404..816291c 100644 --- a/specs/002-multiarch-image-cache/quickstart.md +++ b/specs/002-multiarch-image-cache/quickstart.md @@ -13,6 +13,13 @@ make build make verify ``` +## Multi-Platform Release Workflow + +```sh +make build-release +make verify-release +``` + ## Multi-Platform Verification ```sh @@ -28,8 +35,16 @@ make verify-arm64 - Warm builds reuse cached work when inputs are unchanged. - Runtime verification passes for both platforms. +## Cache Notes + +- Local release builds store cache in `.buildx-cache`. +- CI release builds export cache to GitHub Actions cache for reuse across runs. +- Pull request verification builds import the shared cache and rebuild only platform-specific work. + ## Evidence To Capture -- Build logs showing cache reuse on repeated runs -- Release workflow logs showing one tag published for both variants -- Runtime verification output for amd64 and arm64 +Record results in the following verification artifacts: + +- [verification/release-manifest.md](./verification/release-manifest.md) — single-tag publication evidence for both platform variants +- [verification/cache-reuse.md](./verification/cache-reuse.md) — cold-build and warm-build cache reuse evidence +- [verification/runtime-consistency.md](./verification/runtime-consistency.md) — runtime verification results for amd64 and arm64 diff --git a/specs/002-multiarch-image-cache/tasks.md b/specs/002-multiarch-image-cache/tasks.md index b05d073..3e7c422 100644 --- a/specs/002-multiarch-image-cache/tasks.md +++ b/specs/002-multiarch-image-cache/tasks.md @@ -12,10 +12,10 @@ **Purpose**: Prepare shared workflow, build, and contributor scaffolding for the multi-platform release flow. -- [ ] T001 Update release workflow structure for multi-platform publication in .github/workflows/build.yml -- [ ] T002 [P] Add cache-enabled Buildx build settings for release jobs in .github/workflows/build.yml -- [ ] T003 [P] Add local multi-platform build and verify targets in Makefile -- [ ] T004 [P] Record contributor-facing single-tag release guidance in README.md +- [X] T001 Update release workflow structure for multi-platform publication in .github/workflows/build.yml +- [X] T002 [P] Add cache-enabled Buildx build settings for release jobs in .github/workflows/build.yml +- [X] T003 [P] Add local multi-platform build and verify targets in Makefile +- [X] T004 [P] Record contributor-facing single-tag release guidance in README.md --- @@ -25,12 +25,12 @@ **⚠️ CRITICAL**: No user story work should begin until this phase is complete. -- [ ] T005 Capture multi-platform manifest and cache reuse decisions in specs/002-multiarch-image-cache/research.md -- [ ] T006 [P] Finalize build result and cache record entities in specs/002-multiarch-image-cache/data-model.md -- [ ] T007 [P] Finalize single-tag release and cache reuse contract in specs/002-multiarch-image-cache/contracts/build-release-contract.md -- [ ] T008 Document cold-build and warm-build validation flow in specs/002-multiarch-image-cache/quickstart.md -- [ ] T009 Create release-manifest evidence template in specs/002-multiarch-image-cache/verification/release-manifest.md -- [ ] T010 Create cache-reuse evidence template in specs/002-multiarch-image-cache/verification/cache-reuse.md +- [X] T005 Capture multi-platform manifest and cache reuse decisions in specs/002-multiarch-image-cache/research.md +- [X] T006 [P] Finalize build result and cache record entities in specs/002-multiarch-image-cache/data-model.md +- [X] T007 [P] Finalize single-tag release and cache reuse contract in specs/002-multiarch-image-cache/contracts/build-release-contract.md +- [X] T008 Document cold-build and warm-build validation flow in specs/002-multiarch-image-cache/quickstart.md +- [X] T009 Create release-manifest evidence template in specs/002-multiarch-image-cache/verification/release-manifest.md +- [X] T010 Create cache-reuse evidence template in specs/002-multiarch-image-cache/verification/cache-reuse.md **Checkpoint**: Foundation ready; story work can begin. @@ -44,9 +44,9 @@ ### Implementation for User Story 1 -- [ ] T011 [US1] Publish one manifest-backed image tag for both amd64 and arm64 in .github/workflows/build.yml -- [ ] T012 [US1] Validate release publication includes both platform variants in .github/workflows/build.yml -- [ ] T013 [US1] Record single-tag publication evidence in specs/002-multiarch-image-cache/verification/release-manifest.md +- [X] T011 [US1] Publish one manifest-backed image tag for both amd64 and arm64 in .github/workflows/build.yml +- [X] T012 [US1] Validate release publication includes both platform variants in .github/workflows/build.yml +- [X] T013 [US1] Record single-tag publication evidence in specs/002-multiarch-image-cache/verification/release-manifest.md **Checkpoint**: User Story 1 can be demonstrated and validated independently. @@ -60,9 +60,9 @@ ### Implementation for User Story 2 -- [ ] T014 [US2] Enable remote cache import/export for repeated builds in .github/workflows/build.yml -- [ ] T015 [US2] Add cache-aware local build guidance and targets in Makefile -- [ ] T016 [US2] Record warm-build and cache-hit evidence in specs/002-multiarch-image-cache/verification/cache-reuse.md +- [X] T014 [US2] Enable remote cache import/export for repeated builds in .github/workflows/build.yml +- [X] T015 [US2] Add cache-aware local build guidance and targets in Makefile +- [X] T016 [US2] Record warm-build and cache-hit evidence in specs/002-multiarch-image-cache/verification/cache-reuse.md **Checkpoint**: User Story 2 is independently verifiable with cache evidence. @@ -76,10 +76,10 @@ ### Implementation for User Story 3 -- [ ] T017 [US3] Keep runtime verification checks aligned with both target platforms in scripts/verify-ruby-jemalloc.sh -- [ ] T018 [US3] Wire runtime verification into the release workflow for both platforms in .github/workflows/build.yml -- [ ] T019 [P] [US3] Document runtime verification and release-gating behavior in README.md -- [ ] T020 [US3] Record runtime verification evidence for both platform variants in specs/002-multiarch-image-cache/verification/runtime-consistency.md +- [X] T017 [US3] Keep runtime verification checks aligned with both target platforms in scripts/verify-ruby-jemalloc.sh +- [X] T018 [US3] Wire runtime verification into the release workflow for both platforms in .github/workflows/build.yml +- [X] T019 [P] [US3] Document runtime verification and release-gating behavior in README.md +- [X] T020 [US3] Record runtime verification evidence for both platform variants in specs/002-multiarch-image-cache/verification/runtime-consistency.md **Checkpoint**: User Story 3 provides standalone runtime confidence for the release flow. @@ -89,9 +89,9 @@ **Purpose**: Final alignment across docs, workflow, and evidence artifacts. -- [ ] T021 [P] Validate quickstart examples against final workflow commands in specs/002-multiarch-image-cache/quickstart.md -- [ ] T022 Verify README release and cache guidance matches workflow behavior in README.md -- [ ] T023 Verify all verification artifacts are linked from specs/002-multiarch-image-cache/quickstart.md +- [X] T021 [P] Validate quickstart examples against final workflow commands in specs/002-multiarch-image-cache/quickstart.md +- [X] T022 Verify README release and cache guidance matches workflow behavior in README.md +- [X] T023 Verify all verification artifacts are linked from specs/002-multiarch-image-cache/quickstart.md --- diff --git a/specs/002-multiarch-image-cache/verification/cache-reuse.md b/specs/002-multiarch-image-cache/verification/cache-reuse.md new file mode 100644 index 0000000..5519e14 --- /dev/null +++ b/specs/002-multiarch-image-cache/verification/cache-reuse.md @@ -0,0 +1,24 @@ +# Cache Reuse Evidence + +## Purpose + +Capture evidence that repeated builds reuse unchanged work through cache import/export. + +## Expected Commands + +```sh +make build-release +make build-release +``` + +## Evidence Fields + +- Cold-build duration +- Warm-build duration +- Cache hit evidence +- Cache miss evidence +- Steps rebuilt after changes + +## Notes + +- Record whether the second run reused cache layers and which steps were rebuilt. \ No newline at end of file diff --git a/specs/002-multiarch-image-cache/verification/release-manifest.md b/specs/002-multiarch-image-cache/verification/release-manifest.md new file mode 100644 index 0000000..deaa7a7 --- /dev/null +++ b/specs/002-multiarch-image-cache/verification/release-manifest.md @@ -0,0 +1,23 @@ +# Release Manifest Evidence + +## Purpose + +Capture evidence that the release workflow publishes one tag with both linux/amd64 and linux/arm64 variants. + +## Expected Commands + +```sh +make build-release +make verify-release +``` + +## Evidence Fields + +- Release tag name +- Published platform variants +- Registry reference +- Verification results for both platforms + +## Notes + +- Record the release workflow run URL or build log excerpt here after execution. \ No newline at end of file diff --git a/specs/002-multiarch-image-cache/verification/runtime-consistency.md b/specs/002-multiarch-image-cache/verification/runtime-consistency.md new file mode 100644 index 0000000..999e207 --- /dev/null +++ b/specs/002-multiarch-image-cache/verification/runtime-consistency.md @@ -0,0 +1,24 @@ +# Runtime Consistency Evidence + +## Purpose + +Capture evidence that runtime verification passes for both platform variants after the release workflow changes. + +## Expected Commands + +```sh +make verify-release +``` + +## Evidence Fields + +- Release tag reference +- amd64 verification result +- arm64 verification result +- Ruby version check result +- jemalloc linkage result +- ldconfig visibility result + +## Notes + +- Record the release run URL and verification output for both platforms here after execution. \ No newline at end of file