Skip to content

OCPBUGS-99292: Re-enable Knative Cypress e2e tests - #16837

Open
cajieh wants to merge 1 commit into
openshift:mainfrom
cajieh:re-enable-knative-cypress-e2e-tests
Open

OCPBUGS-99292: Re-enable Knative Cypress e2e tests#16837
cajieh wants to merge 1 commit into
openshift:mainfrom
cajieh:re-enable-knative-cypress-e2e-tests

Conversation

@cajieh

@cajieh cajieh commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Analysis / Root cause:
Knative Cypress e2e was disabled under OCPBUGS-99226 (#16782) because knative-ci.feature was flaking in CI. Failures commonly hit the beforeEachcy.initAdmin() path when the perspective switcher was not ready yet (timeout on perspective-switcher-toggle / menu interaction), which blocked unrelated PRs.

The harness was also using fixed cy.wait() delays around perspective menu open, which is brittle under React 18 / CI load.

Solution description:
Re-enable Knative Cypress suites and harden shared perspective-switcher helpers:

  1. Re-enable Knative runs in:
    • frontend/integration-tests/test-cypress.sh (test-cypress-knative-nightly, test-cypress-knative-headless)
    • test-prow-e2e.sh (SCENARIO == knative)
  2. In initAdmin / initDeveloper, wait for perspective-switcher-toggle to exist and be visible (60s) before switching perspective.
  3. In nav.ts perspective switcher helpers:
    • Increase menu aria-expanded wait where needed
    • Replace fixed cy.wait(1000/1500) with waiting for perspective-switcher-menu-option visibility
    • Use a normal .click() (no force: true) once the option is visible

No product UI changes; test harness / CI wiring only.

Screenshots / screen recording:
N/A — CI / Cypress harness change only.

Test setup:

  • Cluster with Knative / Serverless available (same as existing knative Cypress jobs)
  • Local (optional): from frontend/, run knative headless suite via ./integration-tests/test-cypress.sh -p knative -h true against a configured bridge/cluster
  • CI: confirm knative scenario / nightly + headless jobs run and are green on this PR

Test cases:

  1. Knative prow scenario (SCENARIO=knative) executes (no longer skipped/commented out).
  2. Nightly / headless Cypress aggregators invoke knative suites again.
  3. Knative beforeEach / initAdmin no longer times out waiting for perspective switcher under CI load.
  4. Switching to Core platform and Developer perspectives succeeds without forced clicks or fixed sleeps.
  5. knative-ci.feature smoke scenarios pass (create / interact with knative service & revision as covered by the suite).
  6. Unrelated Cypress packages that use initAdmin / initDeveloper still pass (shared harness change).

Browser conformance:

  • Chrome
  • Firefox
  • Safari (or Epiphany on Linux)

N/A for merge gate (CI Cypress / Chrome-like headless). Mark browsers only if manually exercised locally.

Additional info:

Reviewers and assignees:
/assign

Summary by CodeRabbit

  • Tests
    • Enabled Knative Cypress suites for both nightly and headless runs.
    • Re-enabled Knative Cypress execution when the end-to-end scenario is set to Knative.
    • Improved reliability of perspective-switching tests by replacing fixed waits with UI readiness checks (waiting for the correct legacy toggle and menu options to become visible before interacting).

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 27, 2026
@openshift-ci-robot openshift-ci-robot added the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Jul 27, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@cajieh: This pull request references Jira Issue OCPBUGS-99292, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

<!--
 Please fill in every section below before requesting review. Filled-in descriptions are
 required for the OpenShift Console team to triage the pull request, review the code, and
 verify the behavior end-to-end.

 The pull request title must be prefixed with a Jira issue in order to be merged. For example:

 For e.g Features: https://redhat.atlassian.net/browse/CONSOLE-XXXX
 - CONSOLE-XXXX: <title>
 For e.g Jira Bug Fixes: https://redhat.atlassian.net/browse/OCPBUGS-XXXX
 - OCPBUGS-XXXX: <title>
-->

**Analysis / Root cause**:
Knative Cypress e2e was disabled under OCPBUGS-99226 (#16782) because `knative-ci.feature` was flaking in CI. Failures commonly hit the `beforeEach``cy.initAdmin()` path when the perspective switcher was not ready yet (timeout on `perspective-switcher-toggle` / menu interaction), which blocked unrelated PRs.

The harness was also using fixed `cy.wait()` delays around perspective menu open, which is brittle under React 18 / CI load.

**Solution description**:
Re-enable Knative Cypress suites and harden shared perspective-switcher helpers:

1. Re-enable Knative runs in:
  - `frontend/integration-tests/test-cypress.sh` (`test-cypress-knative-nightly`, `test-cypress-knative-headless`)
  - `test-prow-e2e.sh` (`SCENARIO == knative`)
2. In `initAdmin` / `initDeveloper`, wait for `perspective-switcher-toggle` to exist and be visible (60s) before switching perspective.
3. In `nav.ts` perspective switcher helpers:
  - Increase menu `aria-expanded` wait where needed
  - Replace fixed `cy.wait(1000/1500)` with waiting for `perspective-switcher-menu-option` visibility
  - Use a normal `.click()` (no `force: true`) once the option is visible

No product UI changes; test harness / CI wiring only.

**Screenshots / screen recording**:
N/A — CI / Cypress harness change only.

**Test setup:**
- Cluster with Knative / Serverless available (same as existing knative Cypress jobs)
- Local (optional): from `frontend/`, run knative headless suite via `./integration-tests/test-cypress.sh -p knative -h true` against a configured bridge/cluster
- CI: confirm knative scenario / nightly + headless jobs run and are green on this PR

**Test cases:**
1. Knative prow scenario (`SCENARIO=knative`) executes (no longer skipped/commented out).
2. Nightly / headless Cypress aggregators invoke knative suites again.
3. Knative `beforeEach` / `initAdmin` no longer times out waiting for perspective switcher under CI load.
4. Switching to Core platform and Developer perspectives succeeds without forced clicks or fixed sleeps.
5. `knative-ci.feature` smoke scenarios pass (create / interact with knative service & revision as covered by the suite).
6. Unrelated Cypress packages that use `initAdmin` / `initDeveloper` still pass (shared harness change).

**Browser conformance**:
- [ ] Chrome
- [ ] Firefox
- [ ] Safari (or Epiphany on Linux)

N/A for merge gate (CI Cypress / Chrome-like headless). Mark browsers only if manually exercised locally.

**Additional info:**
- Closes / tracks: https://issues.redhat.com/browse/OCPBUGS-99292
- Related disable: OCPBUGS-99226 (#16782)
- Shared harness impact: `admin.ts` / `nav.ts` changes affect all Cypress suites using `initAdmin` / `initDeveloper`, not only knative.
- Release branches disabled under OCPBUGS-99226 should be re-enabled or explicitly waived after main is stable.
- Merge recommendation: wait for green knative CI on this PR before considering the flake fixed.

**Reviewers and assignees:**
/assign 

<!-- Tag additional knative/e2e owners as needed -->

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 27, 2026
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9ff7f2e9-d95d-46ef-af5f-865d43c9bc69

📥 Commits

Reviewing files that changed from the base of the PR and between 1e46424 and 5e89083.

📒 Files selected for processing (4)
  • frontend/integration-tests/test-cypress.sh
  • frontend/packages/integration-tests/support/admin.ts
  • frontend/packages/integration-tests/views/nav.ts
  • test-prow-e2e.sh
🚧 Files skipped from review as they are similar to previous changes (4)
  • frontend/packages/integration-tests/support/admin.ts
  • test-prow-e2e.sh
  • frontend/integration-tests/test-cypress.sh
  • frontend/packages/integration-tests/views/nav.ts

Walkthrough

Knative Cypress suites are enabled in nightly, headless, and Prow execution paths. Cypress admin and developer initialization now waits for the perspective toggle, while perspective selection waits for visible matching menu options instead of fixed delays.

Changes

Knative Cypress testing

Layer / File(s) Summary
Synchronize perspective navigation
frontend/packages/integration-tests/support/admin.ts, frontend/packages/integration-tests/views/nav.ts
Initialization waits for the perspective toggle, and perspective selection waits for visible matching menu options before clicking.
Enable Knative Cypress paths
frontend/integration-tests/test-cypress.sh, test-prow-e2e.sh
Nightly, headless, and Prow Knative paths now invoke the corresponding Cypress test commands.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: do-not-merge/work-in-progress

Suggested reviewers: jhadvig, spadgett

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise, specific, and accurately reflects the re-enabled Knative Cypress e2e tests.
Description check ✅ Passed The description follows the required template and covers root cause, solution, testing, and additional info.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The PR only changes CI/harness files; no Ginkgo It/Describe/Context/When titles were added or modified, and no dynamic test names appear.
Test Structure And Quality ✅ Passed PASS: The PR only changes Cypress harness and shell scripts; no Ginkgo specs, cluster resource setup/cleanup, or Eventually/Consistently logic are present.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; the PR only changes Cypress shell/TypeScript harness files, so MicroShift-specific API compatibility is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed PASS: The PR only changes Cypress shell/TS harness files; no new Ginkgo tests were added, and no multi-node/SNO assumptions appear in the touched code.
Topology-Aware Scheduling Compatibility ✅ Passed Only Cypress/Prow test harness files changed; no manifests, operators, or controllers with scheduling constraints were modified.
Ote Binary Stdout Contract ✅ Passed Patch only changes Cypress shell/TS harness files; no Go main/TestMain/init or OTE stdout writes were added.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The PR only changes Cypress harness/scripts; no new Ginkgo tests, IPv4 literals, host/IP URL building, or public-internet dependencies were added.
No-Weak-Crypto ✅ Passed Touched files only change Cypress harness waits/Knative invocations; scans found no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret compares.
Container-Privileges ✅ Passed PR only changes Cypress/prow scripts and TS helpers; no container/K8s manifests were modified, so no privileged/root/host* settings were introduced.
No-Sensitive-Data-In-Logs ✅ Passed The PR only changes Cypress waits and re-enables Knative runs; it adds no new logging of secrets, tokens, PII, or hostnames.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci-robot openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 27, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@cajieh: This pull request references Jira Issue OCPBUGS-99292, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)
Details

In response to this:

Analysis / Root cause:
Knative Cypress e2e was disabled under OCPBUGS-99226 (#16782) because knative-ci.feature was flaking in CI. Failures commonly hit the beforeEachcy.initAdmin() path when the perspective switcher was not ready yet (timeout on perspective-switcher-toggle / menu interaction), which blocked unrelated PRs.

The harness was also using fixed cy.wait() delays around perspective menu open, which is brittle under React 18 / CI load.

Solution description:
Re-enable Knative Cypress suites and harden shared perspective-switcher helpers:

  1. Re-enable Knative runs in:
  • frontend/integration-tests/test-cypress.sh (test-cypress-knative-nightly, test-cypress-knative-headless)
  • test-prow-e2e.sh (SCENARIO == knative)
  1. In initAdmin / initDeveloper, wait for perspective-switcher-toggle to exist and be visible (60s) before switching perspective.
  2. In nav.ts perspective switcher helpers:
  • Increase menu aria-expanded wait where needed
  • Replace fixed cy.wait(1000/1500) with waiting for perspective-switcher-menu-option visibility
  • Use a normal .click() (no force: true) once the option is visible

No product UI changes; test harness / CI wiring only.

Screenshots / screen recording:
N/A — CI / Cypress harness change only.

Test setup:

  • Cluster with Knative / Serverless available (same as existing knative Cypress jobs)
  • Local (optional): from frontend/, run knative headless suite via ./integration-tests/test-cypress.sh -p knative -h true against a configured bridge/cluster
  • CI: confirm knative scenario / nightly + headless jobs run and are green on this PR

Test cases:

  1. Knative prow scenario (SCENARIO=knative) executes (no longer skipped/commented out).
  2. Nightly / headless Cypress aggregators invoke knative suites again.
  3. Knative beforeEach / initAdmin no longer times out waiting for perspective switcher under CI load.
  4. Switching to Core platform and Developer perspectives succeeds without forced clicks or fixed sleeps.
  5. knative-ci.feature smoke scenarios pass (create / interact with knative service & revision as covered by the suite).
  6. Unrelated Cypress packages that use initAdmin / initDeveloper still pass (shared harness change).

Browser conformance:

  • Chrome
  • Firefox
  • Safari (or Epiphany on Linux)

N/A for merge gate (CI Cypress / Chrome-like headless). Mark browsers only if manually exercised locally.

Additional info:

Reviewers and assignees:
/assign

Summary by CodeRabbit

  • Tests
  • Enabled Knative Cypress suites for nightly and headless test runs.
  • Re-enabled Knative end-to-end test execution in Prow.
  • Improved perspective-switching test reliability by waiting for controls and menu options to be visible before interacting.
  • Replaced fixed delays with targeted UI readiness checks.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@cajieh

cajieh commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

/retest

@openshift-ci
openshift-ci Bot requested review from jhadvig and spadgett July 27, 2026 15:19
@openshift-ci openshift-ci Bot added the kind/cypress Related to Cypress e2e integration testing label Jul 27, 2026
@openshift-ci

openshift-ci Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: cajieh
Once this PR has been reviewed and has the lgtm label, please assign spadgett for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cajieh cajieh changed the title [WIP] OCPBUGS-99292: Re-enable Knative Cypress e2e tests OCPBUGS-99292: Re-enable Knative Cypress e2e tests Jul 27, 2026
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 27, 2026
@cajieh
cajieh force-pushed the re-enable-knative-cypress-e2e-tests branch from 1e46424 to 5e89083 Compare July 27, 2026 18:41
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@cajieh: This pull request references Jira Issue OCPBUGS-99292, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (5.0.0) matches configured target version for branch (5.0.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Analysis / Root cause:
Knative Cypress e2e was disabled under OCPBUGS-99226 (#16782) because knative-ci.feature was flaking in CI. Failures commonly hit the beforeEachcy.initAdmin() path when the perspective switcher was not ready yet (timeout on perspective-switcher-toggle / menu interaction), which blocked unrelated PRs.

The harness was also using fixed cy.wait() delays around perspective menu open, which is brittle under React 18 / CI load.

Solution description:
Re-enable Knative Cypress suites and harden shared perspective-switcher helpers:

  1. Re-enable Knative runs in:
  • frontend/integration-tests/test-cypress.sh (test-cypress-knative-nightly, test-cypress-knative-headless)
  • test-prow-e2e.sh (SCENARIO == knative)
  1. In initAdmin / initDeveloper, wait for perspective-switcher-toggle to exist and be visible (60s) before switching perspective.
  2. In nav.ts perspective switcher helpers:
  • Increase menu aria-expanded wait where needed
  • Replace fixed cy.wait(1000/1500) with waiting for perspective-switcher-menu-option visibility
  • Use a normal .click() (no force: true) once the option is visible

No product UI changes; test harness / CI wiring only.

Screenshots / screen recording:
N/A — CI / Cypress harness change only.

Test setup:

  • Cluster with Knative / Serverless available (same as existing knative Cypress jobs)
  • Local (optional): from frontend/, run knative headless suite via ./integration-tests/test-cypress.sh -p knative -h true against a configured bridge/cluster
  • CI: confirm knative scenario / nightly + headless jobs run and are green on this PR

Test cases:

  1. Knative prow scenario (SCENARIO=knative) executes (no longer skipped/commented out).
  2. Nightly / headless Cypress aggregators invoke knative suites again.
  3. Knative beforeEach / initAdmin no longer times out waiting for perspective switcher under CI load.
  4. Switching to Core platform and Developer perspectives succeeds without forced clicks or fixed sleeps.
  5. knative-ci.feature smoke scenarios pass (create / interact with knative service & revision as covered by the suite).
  6. Unrelated Cypress packages that use initAdmin / initDeveloper still pass (shared harness change).

Browser conformance:

  • Chrome
  • Firefox
  • Safari (or Epiphany on Linux)

N/A for merge gate (CI Cypress / Chrome-like headless). Mark browsers only if manually exercised locally.

Additional info:

Reviewers and assignees:
/assign

Summary by CodeRabbit

  • Tests
  • Enabled Knative Cypress suites for both nightly and headless runs.
  • Re-enabled Knative Cypress execution when the end-to-end scenario is set to Knative.
  • Improved reliability of perspective-switching tests by replacing fixed waits with UI readiness checks (waiting for the correct legacy toggle and menu options to become visible before interacting).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@cajieh: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. kind/cypress Related to Cypress e2e integration testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants