Adapt testing harness for updates testing

[jlebon]

See #34 (comment)

[alicefr]

Yeah, agree. Though I think there's an argument here for the bootc-operator driving the --node-image instead of using the bink default (even if it'll be the same for now). So then it's controlled here. That would allow us to check the pullspec but not the exact digest. But we could e.g. podman inspect before starting the e2e tests and pass it in as an env var.

I don't think it will quite work. The reason is because bootc reports the name of the image that was built locally by the CI and then used to create the image tagged with the disk. To bink we passed the image containing the disk.
To be clear, the bootc status looks something like:

sudo bootc status
● Booted image: localhost/bink-node:v1.35-fedora-44
        Digest: sha256:8b309d07f734fd6f12e8826f53de8ea296dc3b6bba9a3fed2a120850e9716b16 (amd64)
     Timestamp: 2026-05-21T11:55:26Z

and the image name is coming from here. Maybe we should fix the disk build an always reference the public image

[alicefr]

When starting nodes, we inject a systemd service that overrides the origin refspec so it actually thinks that it's following registry.cluster.local:5000/node:latest. This should probably be exposed as bink node add --target-imgref (similar to bootc install to-filesystem --target-imgref switch).

We are basically re-installing manually the OS at node creation. I'm not a big fan of it since it will increase the test time. A similar alternative could also that we rely on the operator updating the new nodes to the target reference image when they are created.

Otherwise, we could "re-tag" the original image at build time as registry.cluster.local:5000/node:<tag>. In this way, bootc points immediately to the local registry. To be consistent we could also push by default the node image in to the registry.

[jlebon]

I don't think it will quite work. The reason is because bootc reports the name of the image that was built locally by the CI and then used to create the image tagged with the disk. To bink we passed the image containing the disk. To be clear, the bootc status looks something like:

Ahh right. But we could easily add a label to the disk node image that points to the matching bootc image. I'm pretty sure actually bootc-image-builder does something similar too.

Anyway, to not get confused, let's dedicate this ticket to the long-term approach, and perhaps discuss the short-term approach to a separate ticket?

When starting nodes, we inject a systemd service that overrides the origin refspec so it actually thinks that it's following registry.cluster.local:5000/node:latest. This should probably be exposed as bink node add --target-imgref (similar to bootc install to-filesystem --target-imgref switch).

We are basically re-installing manually the OS at node creation.

No. The origin is just a file. We're basically changing the refspec the image thinks it's following rather than the booted image itself.

But yeah regardless of this I think it we should embed the ghcr.io refspec at disk node image build time (this is done using the bcvk install switch I referenced). In the e2e test harness we would scrub and replace this but for the image should be set up correctly on its own.

[alicefr]

I just tried with bootc switch --transport registry registry.cluster.local:5000/bink-node:v1.35-fedora-44-test and then the reboot of the node and now it points to the local registry. However, the question is if this configuration should be part of bink core or as test setup of the operator. I would be in favor of the later. In this way, we can control which image we push to the registry for the update.

This approach is a bit complicated without the tag resolution which is part of milestone 5a

[alicefr]

However, if we relay on tag, I'm wondering if we use the same image, if the tag resolution won't result always in the same tag even if the same image is tagged differently

[jlebon]

We had an OOB chat on this. The general idea:

• Add a --target-imgref switch to bink (Add --target-imgref to bink cluster start and bink node add alicefr/bink#45).
• In the Makefile here, hardcode a BINK_NODE_DISK_IMAGE pullspec to docker pull ghcr.io/alicefr/bink/node:<digest>. We can bump manually for now but later should have it be auto-bumped.
• Add a new Makefile target, e.g. node-images, which inspects the disk image to get the tag/digest of the node image, and then (1) pulls the node image and copies it to the bink registry, (2) creates a new derived node image with new content (e.g. using buildah from + buildah run + buildah commit) and copies it to the bink registry.
• Make e2e depend on node-images. Export the pullspecs of the node images pushed to the internal registry as env vars. E.g. BINK_NODE_IMAGE_BASE and BINK_NODE_IMAGE_UPDATE. The former is used by --target-imgref in env.go when starting new nodes. The latter will be available for use by any test that wants to test a real rollout (those tests should sanity-check that the new content is really there).