Skip to content

OCPBUGS-90633: Fix bootstrap failure in openshift-e2e-aws-disconnected#80851

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
mdbooth:disconnected-fix-gated-credentials
Jun 23, 2026
Merged

OCPBUGS-90633: Fix bootstrap failure in openshift-e2e-aws-disconnected#80851
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
mdbooth:disconnected-fix-gated-credentials

Conversation

@mdbooth

@mdbooth mdbooth commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

aws-provision-cco-manual-users-static was incorrectly filtering CredentialRequest manifests, resulting in attempting to create a CR secret in a namespace which was not created in the cluster.

oc adm release extract --included is the modern solution to this.

This latent bug was triggered by openshift/karpenter-operator#12

Unfortunately I think this is untestable without first merging #80814, as that failure occurs first, and afaik pj-rehearse can't do anything like 'testwith'.

Summary by CodeRabbit

This PR fixes a bootstrap failure affecting the openshift-e2e-aws-disconnected test job in the cluster-capi-operator CI configurations. The failure was triggered by aws-provision-cco-manual-users-static incorrectly filtering CredentialRequest manifests, causing the job to try to create a CredentialRequest secret in a namespace that had not been created in the cluster.

To address this, the CI job now enables the modern manifest extraction path by setting EXTRACT_MANIFEST_INCLUDED: "true" on the e2e-aws-capi-disconnected-techpreview test step environment. This allows the job to use oc adm release extract --included, ensuring the correct set of manifests are extracted and applied.

Updated CI configurations:

  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-main.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-4.23.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.0.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.1.yaml

The underlying latent bug was exposed by changes in the karpenter-operator repository.

CI/Jira validation note: The referenced Jira issue (OCPBUGS-90633) was re-validated after an initial invalidation, and the OpenShift CI bot confirmed it as valid following /jira refresh (including a transition to the POST state and matching the expected 5.0.0 target version for the branch).

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 22, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@mdbooth: This pull request references Jira Issue OCPBUGS-90633, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

aws-provision-cco-manual-users-static was incorrectly filtering CredentialRequest manifests, resulting in attempting to create a CR secret in a namespace which was not created in the cluster.

oc adm release extract --included is the modern solution to this.

This latent bug was triggered by openshift/karpenter-operator#12

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 38a6775a-1e8a-4dcd-b4cd-a0d929ae6bc4

📥 Commits

Reviewing files that changed from the base of the PR and between 3e3a428 and e1d4809.

📒 Files selected for processing (4)
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-main.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-4.23.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.0.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.1.yaml
✅ Files skipped from review due to trivial changes (1)
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-4.23.yaml
🚧 Files skipped from review as they are similar to previous changes (3)
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.1.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.0.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-main.yaml

Walkthrough

Adds the environment variable EXTRACT_MANIFEST_INCLUDED: "true" to the e2e-aws-capi-disconnected-techpreview job's env block in four CI operator configuration files: main, release-4.23, release-5.0, and release-5.1.

Changes

Disconnected Techpreview Job Configuration

Layer / File(s) Summary
Add env var to all branch configs
ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-main.yaml, openshift-cluster-capi-operator-release-4.23.yaml, openshift-cluster-capi-operator-release-5.0.yaml, openshift-cluster-capi-operator-release-5.1.yaml
EXTRACT_MANIFEST_INCLUDED: "true" is added to the e2e-aws-capi-disconnected-techpreview step's env block in each of the four active branch configurations.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly references the Jira issue (OCPBUGS-90633) and clearly describes the main fix: addressing a bootstrap failure in the openshift-e2e-aws-disconnected test job by adding the EXTRACT_MANIFEST_INCLUDED environment variable across four configuration files.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed PR modifies only YAML CI configuration files (4 files), adding environment variable EXTRACT_MANIFEST_INCLUDED; contains no Ginkgo test definitions or Go test files, so the check does not apply.
Test Structure And Quality ✅ Passed This PR modifies only CI operator YAML configuration files (adding environment variables), not Ginkgo test code, making the test structure quality check not applicable.
Microshift Test Compatibility ✅ Passed This PR does not add any new Ginkgo e2e tests. It only modifies CI operator YAML configuration files to add environment variables to existing test jobs, which are not subject to MicroShift compatib...
Single Node Openshift (Sno) Test Compatibility ✅ Passed PR modifies only CI/operator YAML configuration files, adding an environment variable to existing jobs. No new Ginkgo e2e tests are added, so SNO compatibility check is not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed This PR modifies only CI operator configuration files (test job parameters), not deployment manifests, operator code, or controllers. No scheduling constraints are introduced.
Ote Binary Stdout Contract ✅ Passed The PR adds CI configuration changes (environment variables to YAML) and a new utility tool (junitreport) for test output conversion, neither of which are OTE binaries that communicate with openshi...
Ipv6 And Disconnected Network Test Compatibility ✅ Passed This PR does not add any new Ginkgo e2e tests. It only modifies CI operator configuration files to add environment variables. The check for IPv6/disconnected network compatibility only applies to n...
No-Weak-Crypto ✅ Passed PR contains only CI configuration YAML changes adding an environment variable; no cryptographic code, weak algorithms, or secret comparisons present.
Container-Privileges ✅ Passed PR adds only environment variables to CI operator config files; contains no privileged containers, hostPID, hostNetwork, hostIPC, SYS_ADMIN, or allowPrivilegeEscalation settings.
No-Sensitive-Data-In-Logs ✅ Passed PR adds only EXTRACT_MANIFEST_INCLUDED: "true" env var to CI configs; variable is a boolean flag used in conditionals only, never logged, contains no sensitive data.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci openshift-ci Bot requested review from damdo and racheljpg June 22, 2026 15:55
@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 22, 2026

@tthvo tthvo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Based on #80856 rehearsal results, this change looks good to me; and it further confirms that the fix in #76835 works too. Unfortunately, this is an on-going issue affecting all other disconnected jobs 😅

In #76835, we will set EXTRACT_MANIFEST_INCLUDED: true by default.

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 22, 2026
@tthvo

tthvo commented Jun 22, 2026

Copy link
Copy Markdown
Member

/jira refresh

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jun 22, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@tthvo: This pull request references Jira Issue OCPBUGS-90633, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

aws-provision-cco-manual-users-static was incorrectly filtering
CredentialRequest manifests, resulting in attempting to create a CR
secret in a namespace which was not created in the cluster.

oc adm release extract --included is the modern solution to this.
@mdbooth mdbooth force-pushed the disconnected-fix-gated-credentials branch from 3e3a428 to e1d4809 Compare June 23, 2026 08:15
@openshift-ci openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Jun 23, 2026
@mdbooth

mdbooth commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

Rebased to pull in #80814. pj-rehearse should now succeed.

@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@mdbooth: This pull request references Jira Issue OCPBUGS-90633, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

aws-provision-cco-manual-users-static was incorrectly filtering CredentialRequest manifests, resulting in attempting to create a CR secret in a namespace which was not created in the cluster.

oc adm release extract --included is the modern solution to this.

This latent bug was triggered by openshift/karpenter-operator#12

Unfortunately I think this is untestable without first merging #80814, as that failure occurs first, and afaik pj-rehearse can't do anything like 'testwith'.

Summary by CodeRabbit

This PR fixes a bootstrap failure affecting the openshift-e2e-aws-disconnected test job in the cluster-capi-operator CI configurations. The failure was triggered by aws-provision-cco-manual-users-static incorrectly filtering CredentialRequest manifests, causing the job to try to create a CredentialRequest secret in a namespace that had not been created in the cluster.

To address this, the CI job now enables the modern manifest extraction path by setting EXTRACT_MANIFEST_INCLUDED: "true" on the e2e-aws-capi-disconnected-techpreview test step environment. This allows the job to use oc adm release extract --included, ensuring the correct set of manifests are extracted and applied.

Updated CI configurations:

  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-main.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-4.23.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.0.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.1.yaml

The underlying latent bug was exposed by changes in the karpenter-operator repository.

CI/Jira validation note: The referenced Jira issue (OCPBUGS-90633) was re-validated after an initial invalidation, and the OpenShift CI bot confirmed it as valid following /jira refresh (including a transition to the POST state and matching the expected 5.0.0 target version for the branch).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

[REHEARSALNOTIFIER]
@mdbooth: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name Repo Type Reason
pull-ci-openshift-cluster-capi-operator-release-4.23-e2e-aws-capi-disconnected-techpreview openshift/cluster-capi-operator presubmit Ci-operator config changed
pull-ci-openshift-cluster-capi-operator-release-5.1-e2e-aws-capi-disconnected-techpreview openshift/cluster-capi-operator presubmit Ci-operator config changed
pull-ci-openshift-cluster-capi-operator-release-5.0-e2e-aws-capi-disconnected-techpreview openshift/cluster-capi-operator presubmit Ci-operator config changed
pull-ci-openshift-cluster-capi-operator-main-e2e-aws-capi-disconnected-techpreview openshift/cluster-capi-operator presubmit Ci-operator config changed
Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

@mdbooth

mdbooth commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse pull-ci-openshift-cluster-capi-operator-main-e2e-aws-capi-disconnected-techpreview

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mdbooth: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@damdo damdo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 23, 2026
@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: damdo, mdbooth, tthvo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@mdbooth

mdbooth commented Jun 23, 2026

Copy link
Copy Markdown
Contributor Author

/pj-rehearse ack

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

@mdbooth: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

@openshift-merge-bot openshift-merge-bot Bot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Jun 23, 2026
@openshift-ci

openshift-ci Bot commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

@mdbooth: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot Bot merged commit bcd67c9 into openshift:main Jun 23, 2026
15 checks passed
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@mdbooth: Jira Issue OCPBUGS-90633: All pull requests linked via external trackers have merged:

Jira Issue OCPBUGS-90633 has been moved to the MODIFIED state.

Details

In response to this:

aws-provision-cco-manual-users-static was incorrectly filtering CredentialRequest manifests, resulting in attempting to create a CR secret in a namespace which was not created in the cluster.

oc adm release extract --included is the modern solution to this.

This latent bug was triggered by openshift/karpenter-operator#12

Unfortunately I think this is untestable without first merging #80814, as that failure occurs first, and afaik pj-rehearse can't do anything like 'testwith'.

Summary by CodeRabbit

This PR fixes a bootstrap failure affecting the openshift-e2e-aws-disconnected test job in the cluster-capi-operator CI configurations. The failure was triggered by aws-provision-cco-manual-users-static incorrectly filtering CredentialRequest manifests, causing the job to try to create a CredentialRequest secret in a namespace that had not been created in the cluster.

To address this, the CI job now enables the modern manifest extraction path by setting EXTRACT_MANIFEST_INCLUDED: "true" on the e2e-aws-capi-disconnected-techpreview test step environment. This allows the job to use oc adm release extract --included, ensuring the correct set of manifests are extracted and applied.

Updated CI configurations:

  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-main.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-4.23.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.0.yaml
  • ci-operator/config/openshift/cluster-capi-operator/openshift-cluster-capi-operator-release-5.1.yaml

The underlying latent bug was exposed by changes in the karpenter-operator repository.

CI/Jira validation note: The referenced Jira issue (OCPBUGS-90633) was re-validated after an initial invalidation, and the OpenShift CI bot confirmed it as valid following /jira refresh (including a transition to the POST state and matching the expected 5.0.0 target version for the branch).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@mdbooth mdbooth deleted the disconnected-fix-gated-credentials branch June 23, 2026 12:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. rehearsals-ack Signifies that rehearsal jobs have been acknowledged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants