splitting e2e tests between bundle and operator#1548
splitting e2e tests between bundle and operator#1548blublinsky wants to merge 1 commit intoopenshift:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
blublinsky
force-pushed
the
tests-split-between-operator-bundle
branch
2 times, most recently
from
33ee02f to
9731cba
Compare
|
/retest |
blublinsky
force-pushed
the
tests-split-between-operator-bundle
branch
15 times, most recently
from
3897154 to
053c2e8
Compare
blublinsky
changed the title
blublinsky
force-pushed
the
tests-split-between-operator-bundle
branch
3 times, most recently
from
411e277 to
499dee7
Compare
openshift-ci
Bot
added
the
needs-rebase
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
| git remote add origin https://github.com/openshift/lightspeed-operator.git | ||
| git fetch --depth 1 origin "${COMMIT}" | ||
| git checkout -q FETCH_HEAD | ||
| ./.tekton/integration-tests/scripts/run-konflux-operator-install.sh "$(params.install-mode)" |
There was a problem hiding this comment.
Any reason for us to install the operator in the service tests considering we already have that logic in place in the service tests?
There was a problem hiding this comment.
ols-install puts the operator from this SNAPSHOT on the cluster (bundle/direct via hack/install). The service step only runs lightspeed-service e2e (test-e2e-cluster.sh / pytest), which reconfigures OLS per suite (adapt_ols_config) and does not replace that snapshot install contract.
Pytest calls install_ols() only when the operator is not already there (OLM: CSV; direct: lightspeed-operator-controller-manager deployment). Otherwise it skips install and still runs adapt_ols_config() each run.
| echo "error: ${REPO_ROOT}/config/default/deployment-patch.yaml not found" >&2 | ||
| exit 1 | ||
| } | ||
| if [[ -z "$operator_img" ]]; then |
There was a problem hiding this comment.
at script entry we're verifying that $IMG is not empty. Therefore this and the following ifs will never activate.
There was a problem hiding this comment.
good catch, fixed
| echo "error: catalog dir not found: ${catalog_dir} (cwd: $(pwd))" >&2 | ||
| exit 1 | ||
| fi | ||
| oldest_ver="$(ls "${catalog_dir}"/bundle-v*.yaml | sed -n 's/.*bundle-v\(.*\)\.yaml/\1/p' | sort -V | head -n1)" |
There was a problem hiding this comment.
This will cause every upgrade test to fail since we're picking 1.0.8 and there's a known upgrade issue from 1.0.9 to 1.0.10. So if we pick 1.0.8 to any newer version, it will always fail. Thus why in my PR, I'm picking latest - 1. Assuming we're always testing it from now on, we never will run into problems and we will test all upgrades.
There was a problem hiding this comment.
Agreed — we switched the upgrade base from the oldest catalog entry to latest − 1 (second-newest semver in lightspeed-catalog-), with a hard error if there are fewer than two bundles. That avoids long broken chains from very old bases (e.g. 1.0.8) while still testing a real catalog → snapshot upgrade. Pipeline descriptions are updated to match.
Made-with: Cursor
blublinsky
force-pushed
the
tests-split-between-operator-bundle
branch
from
499dee7 to
5b143e7
Compare
|
@blublinsky: The following tests failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
2 participants
Description
Summary
Consolidates Konflux integration Tekton Pipeline YAML into one file per test family (operator e2e, operator upgrade e2e, service integration) with pipeline parameters for OpenShift minor line (openshift-version-prefix, artifact-oci-tag-prefix where needed, test-name, namespace). Removes 13 per-version pipeline files and the stale e2e-tests.yaml scenario that pointed at a non-existent pipeline.
Extracts bundle install logic into hack/install/ (install-operator-bundle.sh, _lib.sh) and reuses it from the merged pipelines and rapidast-scan.yaml (shallow clone at ols-bundle snapshot revision, then script).
Adds konflux-integration-test-scenarios.yaml with 12 IntegrationTestScenario resources (v1beta2) targeting the merged pipelines and passing the correct parameters per minor (4.16–4.19).
Upgrade pipeline: installs baseline bundle, then snapshot bundle using PRE_BUNDLE_IMAGE + BUNDLE_IMAGE and a single install-operator-bundle.sh invocation (matches script contract and make test-upgrade).
config/default/deployment-patch.yaml: intentionally unchanged (keeps REPLACE*_ placeholders; no pinned digests in this PR).
Next steps (post-merge / follow-ups)
Konflux operations
Remove or rename legacy IntegrationTestScenario objects if names conflict with the new CRs (operator-e2e-tests-*, etc.).
Update any required integration test or release policy references that still point at ols-e2e-tests-416 or the old pipeline path.
Confirm test-name semantics
Operator e2e and service integration scenarios both use ols-e2e-tests-4.xx as the test-name param while metadata.name differs. Verify Konflux/reporting does not require globally unique test-name values; adjust if needed.
Git resolver revision
Scenarios pin revision: main. If you want PRs to exercise pipeline YAML on the PR branch, adopt whatever revision substitution pattern your tenant supports (e.g. dynamic revision), or document that pipeline changes apply after merge to main.
Optional: IDMS parity
rapidast-scan.yaml sets IMAGE_DIGEST_MIRROR_SET_URL; merged operator e2e install does not. If ephemeral clusters show pull errors for registry.redhat.io content, export the same URL in the operator e2e install step.
Security / hygiene
Consider dropping cat $KUBECONFIG (or redacting) from Tekton step logs in a follow-up PR.
Direct install path (not in this PR)
install-operator-direct.sh exists but is not wired into a parallel Konflux pipeline or scenario.
Later: add a pipeline/scenario (or parameterized branch), SKIP_IDMS / Hypershift behavior as needed, and document when to use direct vs bundle.
Timeouts
If integration runs hit default timeouts, add Konflux IntegrationTestScenario annotations for pipeline/tasks/finally timeouts per Konflux docs.
Developer ergonomics
Type of change
Related Tickets & Documents
https://redhat.atlassian.net/browse/OLS-2857
Checklist before requesting a review
Testing