Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create index image from pre-built bundles #597

Merged
merged 30 commits into from
Jul 11, 2024

Conversation

vkmc
Copy link
Collaborator

@vkmc vkmc commented May 14, 2024

Deploy from bundles can now be used along with deploy from index enabled.

This way, an index image can be created with specific pre built bundles and deploy STF with those.

Deploy from bundles can now be used along with
deploy from index enabled.

This way, an index image can be created with
specific pre built bundles and deploy STF with
those.
@vkmc vkmc added the do-not-merge Code is not ready to be merged label May 14, 2024
@vkmc vkmc changed the title Create index image from pre-built bundles [WIP] Create index image from pre-built bundles May 14, 2024
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/995fade75a1c45ae8ae54db7ec3666f7

✔️ stf-crc-ocp_412-local_build SUCCESS in 35m 51s
✔️ stf-crc-ocp_414-local_build SUCCESS in 40m 52s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 32m 43s
stf-crc-ocp_414-local_build-index_deploy FAILURE in 40m 09s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/101ed9e66b254f08b569a40965ad4998

✔️ stf-crc-ocp_412-local_build SUCCESS in 34m 38s
✔️ stf-crc-ocp_414-local_build SUCCESS in 42m 53s
✔️ stf-crc-ocp_412-local_build-index_deploy SUCCESS in 44m 19s
stf-crc-ocp_414-local_build-index_deploy FAILURE in 35m 55s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/aa553a06bb894ae48ddf186ce9355e0a

stf-crc-ocp_412-local_build RETRY_LIMIT in 30m 01s
✔️ stf-crc-ocp_414-local_build SUCCESS in 40m 18s
stf-crc-ocp_412-local_build-index_deploy RETRY_LIMIT in 30m 37s
✔️ stf-crc-ocp_414-local_build-index_deploy SUCCESS in 43m 02s

vkmc added 4 commits May 17, 2024 16:57
We need to use the internal registry address
to build the index image that will serve
as base for the catalog
We only need the tag when creating the index image
Keeping this value will generate errors
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/27644a66ea214bdbad5f95bbadb2f08d

✔️ stf-crc-ocp_412-local_build SUCCESS in 36m 10s
✔️ stf-crc-ocp_414-local_build SUCCESS in 42m 13s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 41m 37s
✔️ stf-crc-ocp_414-local_build-index_deploy SUCCESS in 52m 59s

@vkmc vkmc removed the do-not-merge Code is not ready to be merged label May 17, 2024
@vkmc vkmc changed the title [WIP] Create index image from pre-built bundles Create index image from pre-built bundles May 17, 2024
@vkmc vkmc added the do-not-merge Code is not ready to be merged label May 17, 2024
@vkmc vkmc changed the title Create index image from pre-built bundles [WIP] Create index image from pre-built bundles May 17, 2024
@vkmc
Copy link
Collaborator Author

vkmc commented May 17, 2024

Still have to work on updating docs, further testing, and better handling of the bundles version (we should honor the sgo_bundle_tag and sto_bundle_tag we already have instead of manipulating the tags directly from the sto ang sto bundles path)

But... managed to get a working deployment :)

vkmc added 3 commits May 21, 2024 09:51
Add more details on how to deploy from index with
pre built bundles
Define this value to stable-1.5 when deploying
from bundles and index, unstable (the default)
if bundles have been built locally.

Also, honor the sgo and sto bundles tags. Ideally,
we would update the sgo and sto image path to be
only the path (without the tag) and define the tag
separately. In this case, it is responsibility
from the user to make sure that the tag in the
image path and the tag provided in sto and sgo
image path match when deploying from externally
built bundles and index.
@vkmc vkmc removed the do-not-merge Code is not ready to be merged label May 22, 2024
@vkmc vkmc changed the title [WIP] Create index image from pre-built bundles Create index image from pre-built bundles May 22, 2024
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/33cb40ed42bf43088782cf8d3ac2af68

stf-crc-ocp_412-local_build RETRY_LIMIT in 17m 57s
stf-crc-ocp_414-local_build RETRY_LIMIT in 23m 56s
stf-crc-ocp_412-local_build-index_deploy RETRY_LIMIT in 18m 35s
stf-crc-ocp_414-local_build-index_deploy RETRY_LIMIT in 24m 37s

@vkmc
Copy link
Collaborator Author

vkmc commented May 22, 2024

recheck

@mrunge
Copy link
Member

mrunge commented May 22, 2024

There was an earlier outage on RDO cloud, resulting in "could not download metadata" errors

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/e98a669277da449b87c0433099208925

stf-crc-ocp_412-local_build FAILURE in 22m 17s
✔️ stf-crc-ocp_414-local_build SUCCESS in 43m 11s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 27m 02s
✔️ stf-crc-ocp_414-local_build-index_deploy SUCCESS in 52m 59s

Copy link

This change depends on a change that failed to merge.

Change #608 is needed.

Nightly bundles use the "unstable" channel while
RH catalog bundles use "stable-1.5" channel

We should use nightly bundles "unstable" by default
and pass "stable-1.5" when deploying from RH catalog
Copy link

This change depends on a change that failed to merge.

Change #608 is needed.

@vkmc
Copy link
Collaborator Author

vkmc commented Jul 3, 2024

recheck

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/d49870b4dd8c4553b49b086ddde8b0cb

✔️ stf-crc-ocp_412-local_build SUCCESS in 34m 39s
✔️ stf-crc-ocp_414-local_build SUCCESS in 34m 33s
✔️ stf-crc-ocp_412-local_build-index_deploy SUCCESS in 37m 12s
✔️ stf-crc-ocp_414-local_build-index_deploy SUCCESS in 42m 20s
stf-crc-ocp_414-nightly_bundles-index_deploy FAILURE in 39m 32s

vkmc added 2 commits July 8, 2024 12:42
We need opm to get the bundle version when deploying
from index with pre-built bundles

Get opm following the same logic we use for operator-sdk
Inspect the provided bundles with opm render and
obtain the operator bundle version for each of them.
@vkmc vkmc added the do-not-merge Code is not ready to be merged label Jul 8, 2024
vkmc added 8 commits July 8, 2024 13:25
Let's stick to latest-4.14 for now, with the option
of updating to a different one if needed
oc image info command provides operator bundle
version information. Use this to get the proper
version numbers when building the index image
Parse the oc image info output to get
the STO and SGO bundle versions
@vkmc vkmc removed the do-not-merge Code is not ready to be merged label Jul 8, 2024
@vkmc vkmc requested a review from elfiesmelfie July 8, 2024 13:41
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/858e3f8d87ac445c96ee9ecb4bef98e0

✔️ stf-crc-ocp_412-local_build SUCCESS in 34m 42s
✔️ stf-crc-ocp_414-local_build SUCCESS in 33m 14s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 32m 39s
stf-crc-ocp_414-local_build-index_deploy RETRY_LIMIT in 19m 11s

Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/e107cb69228d4c8897c10fa573b56d2f

✔️ stf-crc-ocp_412-local_build SUCCESS in 35m 37s
✔️ stf-crc-ocp_414-local_build SUCCESS in 33m 56s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 30m 28s
stf-crc-ocp_414-local_build-index_deploy FAILURE in 29m 47s

sto_bundle_info and sgo_bundle_info is overriding
even though the step is skipped

Add an extra logic check to avoid variables
override
@vkmc vkmc force-pushed the vkmc-STF-1747-index_with_prebuilt_bundles branch from f57dc99 to cfc3abc Compare July 9, 2024 17:07
Copy link

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://review.rdoproject.org/zuul/buildset/ecff0c13d1ef49c69129d4dc9fb85a3a

✔️ stf-crc-ocp_412-local_build SUCCESS in 37m 27s
✔️ stf-crc-ocp_414-local_build SUCCESS in 40m 57s
stf-crc-ocp_412-local_build-index_deploy FAILURE in 20m 54s
✔️ stf-crc-ocp_414-local_build-index_deploy SUCCESS in 42m 56s

@vkmc
Copy link
Collaborator Author

vkmc commented Jul 11, 2024

recheck

@vkmc vkmc enabled auto-merge (squash) July 11, 2024 11:25
@vkmc vkmc merged commit e82ab70 into master Jul 11, 2024
10 checks passed
@vkmc vkmc deleted the vkmc-STF-1747-index_with_prebuilt_bundles branch July 11, 2024 12:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants