Skip to content

Skip the docs CI image build when that ref was already built - #70675

Merged
potiuk merged 1 commit into
mainfrom
skip-docs-image-build-when-ref-unchanged
Jul 29, 2026
Merged

Skip the docs CI image build when that ref was already built#70675
potiuk merged 1 commit into
mainfrom
skip-docs-image-build-when-ref-unchanged

Conversation

@potiuk

@potiuk potiuk commented Jul 29, 2026

Copy link
Copy Markdown
Member

Follow-up to #70650.

Docs for a ref get published more than once — an RC, then the final docs after the vote — and today the second publish rebuilds an image the first one already built from the very same commit. #70650 made that image usable as a build cache; this makes it usable as the answer, so the second publish skips the build entirely.

  • Ref shown in the step titles. image-stash-suffix becomes image-stash-ref (the caller passes the ref, not -<ref>), so every ref-dependent step names it: Restore CI docker image built for ref providers-amazon/9.0.0 …, Stash cache mount … for ref …, and the matching restores in the docs and registry jobs.
  • Reuse instead of rebuild. A tiny second stash records the commit each ref-image was built from. It is restored before every other cache, and on a match the job skips serve-as-cache, the ghcr login, the build, the export, both image stashes and the mount-cache steps — a ~40-byte download instead of a ~25-minute job.
  • The commit check is load-bearing. A ref is not a commit: a branch documented twice moves between the two publishes, so its image is a good cache and a wrong answer. Comparing against git rev-parse HEAD takes the fast path only for an identical commit and otherwise falls back to today's seed-and-build. The marker is stashed last and with the image's retention, so finding it implies the image is still there; the fast path is also off for push-image: true callers, which have an errand beyond producing the image.
  • A ref stashes under its own keys. Reusing means publishing nothing, which the shared per-branch stash could not support — it is written by every build on the branch, so whoever restored it next would get sources they never asked for. The image, its commit and the mount cache are now keyed per ref, and this run's docs and registry jobs read those keys. As a side effect the docs publish stops overwriting main's image and mount-cache stashes with a ref's.

The first publish after this merges still builds — existing per-ref stashes carry no commit marker.


Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Opus 5)

Generated-by: Claude Code (Opus 5) following the guidelines

Docs for a ref are published more than once - an RC, then the final docs
after the vote - and the second publish rebuilds an image the first one
already built from the very same commit. Seeding the build cache with that
image, as #70650 does, still pays for a docker load, a registry push, a
build that hits cache on every layer, an export and two multi-gigabyte
stash uploads, none of which can produce anything the first publish did not
already produce.

Reusing the image outright rather than as a cache turns on knowing it was
built from this commit. A ref does not say that: a branch documented twice
moves between the two publishes, so its image is a good cache and a wrong
answer. The commit each image is stashed with is what tells those apart,
and it is stashed on its own so that deciding costs a few bytes rather than
the image the decision may make unnecessary.

Reusing it also means publishing nothing, which the shared per-branch stash
could not support: it is written by every build on this branch, so whoever
restores it next would get sources they never asked for. The ref's own
stash is the one this run's docs and registry jobs now read, so a run with
nothing to build leaves them reading what the previous publish left there.
The mount cache moves with it for the same reason - it holds the dependency
set the sources resolve to, and a ref's and the branch tip's are exactly
what differ.
@potiuk
potiuk merged commit f346e64 into main Jul 29, 2026
117 of 118 checks passed
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Airflow Registry Jul 29, 2026
@potiuk
potiuk deleted the skip-docs-image-build-when-ref-unchanged branch July 29, 2026 11:04
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-3-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
v3-3-test Commit Link

You can attempt to backport this manually by running:

cherry_picker f346e64 v3-3-test

This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants