Skip to content

ext package: bake resolved version into the published avocado.yaml#161

Open
mobileoverlord wants to merge 2 commits into
mainfrom
jschneck/ext-package-bake-version
Open

ext package: bake resolved version into the published avocado.yaml#161
mobileoverlord wants to merge 2 commits into
mainfrom
jschneck/ext-package-bake-version

Conversation

@mobileoverlord

Copy link
Copy Markdown
Contributor

Problem

The in-source program extensions (avocado-ext-cli, avocado-ext-connect, avocado-ext-tunnels) keep the extension version in sync with Cargo.toml by templating it from an env var:

extensions:
  avocado-ext-connect:
    version: '{{ env.AVOCADO_EXT_VERSION }}'   # CI exports it from [package] version

That template only resolves while AVOCADO_EXT_VERSION is set — i.e. at package / PR-test time. The ext package RPM payload copied the raw on-disk avocado.yaml, so the literal template survived into the published package. Later, a runtime build consuming that extension has no AVOCADO_EXT_VERSION in scope, so it interpolated to '' and failed:

Extension 'avocado-ext-connect' has invalid version ''. Version must be in
semantic versioning format (e.g., '1.0.0', '2.1.3')

Fix

Resolve the required version field at package time and bake the concrete semver into the avocado.yaml that ships in the RPM payload. The resolved value is metadata.version — already interpolated through the normal config pipeline and semver-validated before packaging — so no new resolution logic, just persistence.

Only extensions.<ext>.version is rewritten, via a comment-preserving, indentation-scoped line edit (no serde round-trip, so comments and quoting are untouched). Every other template ({{ avocado.target }}, {{ env.AVOCADO_DISTRO_RELEASE }}, …) is intentionally left for the consumer to resolve at their build time — matching the "interpolate the required fields, defer the rest" model.

Implementation notes:

  • The bake rewrites the single small staged avocado.yaml (≈1 KB) that's already in the staging dir — it adds no copy of the (potentially large) source tree.
  • Remote/unreadable source configs (e.g. packaging a fetched remote ext) are left untouched rather than failing the package.

Tests

New unit tests for the bake_extension_version helper:

  • version-only replacement, with other templates and comments preserved
  • nested version: keys (e.g. under packages:) left untouched
  • multi-extension files bake only the named extension
  • not-found error path

Local: cargo fmt --check, cargo clippy --all-targets --all-features -- -D warnings, and cargo test all green (17 package tests, incl. 4 new).

Follow-up (not in this PR)

ext package currently copies the selected files twice (src → staging → rpmbuild buildroot). A separate refactor can have %install copy straight from the source into the buildroot (dropping the intermediate staging dir) to cut one copy for large source trees; it touches the rpmbuild flow so it should be validated against a real avocado ext package run.

In-source program extensions (avocado-ext-cli/connect/tunnels) declare
`version: '{{ env.AVOCADO_EXT_VERSION }}'` so the packaged version tracks
Cargo.toml — CI exports AVOCADO_EXT_VERSION from `[package] version`. That
template only resolves while the env var is set (package/PR-test time).

When the env template survived into the published package, a downstream
runtime build — which has no AVOCADO_EXT_VERSION in scope — interpolated it
to '' and failed:

    Extension 'avocado-ext-connect' has invalid version ''. Version must be
    in semantic versioning format (e.g., '1.0.0', '2.1.3')

Resolve the required `version` field at package time and bake the concrete
semver into the avocado.yaml that ships in the RPM payload. Only
`extensions.<ext>.version` is rewritten (a comment-preserving, indentation-
scoped edit); every other template (`{{ avocado.target }}`,
`{{ env.AVOCADO_DISTRO_RELEASE }}`, ...) is left for the consumer to resolve
at their build time. The bake rewrites just the one small staged file, so it
adds no source-tree copy.

Adds unit tests covering: version-only replacement with other templates and
comments preserved, nested `version:` keys left untouched, multi-extension
files baking only the named extension, and the not-found error path.
@mobileoverlord mobileoverlord requested a review from jetm June 26, 2026 16:39
@jetm

jetm commented Jun 26, 2026

Copy link
Copy Markdown

@mobileoverlord before the review below: I've now gone through every PR you asked me to look at, so I'd appreciate the same in return. Mine have been sitting without a review and they're the foundation the CI/CD lab work depends on. Fair's fair - here are mine, roughly in priority order:

Now to #161. The direction is right and the base64 transport is a clean way to dodge shell-escaping the YAML, but I think a few things need addressing before this merges - two of them turn a currently-working ext package into a hard failure, and one is a command-injection path.

Critical

  1. Version sourced from a target/kernel override block hard-fails. metadata.version comes from the merged config, so the resolved version can originate from a target-<name>.version sub-block. bake_extension_version only matches a version: that's a direct child of the extension, so in that case it bail!s and the ? aborts a package that built fine before. It can also half-bake: a base version: template gets resolved while a target-*.version template survives into the payload and still interpolates to '' downstream - the exact failure this is meant to prevent.

  2. Command injection through the version string. validate_semver only checks the three numeric components; pre-release/build metadata after -/+ isn't validated. metadata.version is then interpolated into the double-quoted echo "Baked resolved version '...'" in the rpmbuild script, so a version like 1.0.0-$(...) executes in the build container. The baked YAML rides safely through base64 already; the version shouldn't be interpolated into the shell at all.

  3. Templated extension keys hard-fail. Extension keys can be templates (avocado-bsp-{{ avocado.target }}), and find_ext_in_mapping resolves them by interpolating. bake_extension_version compares the key literally, so any template-keyed extension never matches and bail!s.

Warning

  1. .yml vs .yaml silently skips the bake. cfg_basename follows the on-disk name, but staging copies the hardcoded avocado.yaml. If the config is avocado.yml, the if [ -f "$STAGING_DIR/avocado.yml" ] guard is false and the bake is skipped with no error, shipping the unresolved template.

  2. Remote extensions never get baked. The host can't read the container-volume config path, so the bake section is empty and the template ships. It's noted as intentional, but it's the same downstream breakage, just for remote ext.

Suggestion

  1. Reuse src/utils/config_edit.rs. We already have a comment-preserving avocado.yaml line-editor there, with quoted-key handling and the compact-indent fix from ENG-1868, plus tests. A set_extension_field built on find_top_level_key would replace the hand-rolled state machine and inherit those fixes instead of drifting from them. Resolving against the merged config (which fixes 1 and 3) fits naturally there too.

The tests are good - the nested-version: and multi-extension cases are exactly the right edges. Happy to pair on the config_edit.rs rework if useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants