Skip to content

fix: run and await after-build pipelines before deploying - #235

Merged
vmelikyan merged 2 commits into
mainfrom
fix/after-build-on-image-exists
Aug 11, 2026
Merged

fix: run and await after-build pipelines before deploying#235
vmelikyan merged 2 commits into
mainfrom
fix/after-build-on-image-exists

Conversation

@vmelikyan

Copy link
Copy Markdown
Contributor

Issue

Services can declare an afterBuildPipelineId: a pipeline that must run after the image build
for the deployed image to actually work (for example, publishing build artifacts the running
service depends on). That pipeline was only ever triggered by the run that built the image, and
the image-exists check (tagsExist) assumed a present tag meant all build work was done:

  • If the building run was superseded or crashed between pushing the image and its after-build
    completing, the image was left permanently unprocessed. Every later pass — including manual
    redeploys — found the tag, skipped the pipeline, and deployed anyway. There was no recovery
    path short of deleting the image tag to force a rebuild.
  • Codefresh-builder cache hits never ran or verified the pipeline at all.
  • A superseded run could fire the pipeline without awaiting it, so its outcome was never
    observed and deploys could race it.

Fix

An image is only deployable when its after-build pipeline is proven to have run.

  • A successful, awaited pipeline run is recorded on the deploy
    (afterBuildCompletionKey = pipelineId@imageTag, migration 033). A Codefresh-engine build
    also records it on success when its embedded post-build step is non-detached, since that
    step's failure fails the parent pipeline. The record is cleared, fail-closed, before any
    rebuild of the same tag.
  • The image-exists path requires a matching record. Missing or mismatched → the pipeline is
    triggered and awaited before BUILT (any builder engine); failure records BUILD_FAILED and
    blocks rollout. Matching → skipped, so redeploys of already-verified services don't re-run it.
  • Fresh builds publish BUILT only after the pipeline succeeds, and record BUILD_FAILED when it
    doesn't.
  • Superseded runs publish and trigger nothing; the current run owns the pipeline, and currency
    is rechecked after every pipeline wait so a stale run can't publish state.
  • The pipeline receives both historical invocation contracts' variables (branch plus
    SOURCE_REVISION/SOURCE_BRANCH).

The record is only ever an optimization: a missing or stale record causes a re-run, never a
skip. This relies on two contracts already implicit in tag-level build dedupe — an image tag
(sha + env hash) identifies equivalent build outputs, and after-build pipelines are idempotent
per image tag.

After deploying this change, the first pass over each environment re-runs its after-build
services' pipelines once (no recorded completions exist yet), then goes quiet.

@vmelikyan
vmelikyan requested a review from a team as a code owner August 11, 2026 18:31
@vmelikyan
vmelikyan merged commit f1c45b1 into main Aug 11, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant