Skip to content

feat: E2E and release testing improvements #2527

Description

@drew

Problem Statement

Today it is difficult to assemble reproducible E2E environments with the correct operating-system dependencies and configuration, such as Fedora with Podman, Rocky Linux with Docker, or an SELinux-enabled host. We want to make it easy to provision these environments, deploy OpenShell into them, and run E2E and release tests against the resulting deployment across documented beta platforms.

Proposed Design

  • 1. Common test guest foundation (test(e2e): add reusable QEMU infrastructure for E2E tests #2471). Land reusable Nix/QEMU guests with an uncached correctness path and optional OCI-backed QCOW2 caching. Restrict cache publication to trusted workflows and fall back safely when caching fails.

  • 2. VM-backed E2E runner (test(e2e): add VM-backed E2E suite runner #2473). Build OpenShell from the checkout, run it inside a test guest, execute host-side E2E scenarios, and retain useful diagnostics. Clean up reliably after successful, failed, and interrupted runs.

  • 3. Release canaries on test guests (ci(release): run Linux release canaries in QEMU VMs #2472). Run release canaries inside representative Ubuntu and Fedora-family guests using the same artifacts and installation paths users consume. Exercise cache hits and forced misses while reporting boot, installation, and test timing separately.

  • 4. Release upgrade coverage. Test clean installation and previous-release-to-candidate upgrades for Debian and RPM packages, verifying versions, gateway health, persisted state, and sandbox operation. Cover Debian/Ubuntu amd64 and arm64, and macOS Apple Silicon.

  • 5. Test guests as development sandboxes. Make prepared test-guest VM images consumable through the OpenShell sandbox lifecycle so developers can provision a selected distro, runtime, and security configuration and build or test OpenShell inside it. Define image provenance, writable workspace, persistence, resource, lifecycle, and cache invalidation boundaries.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions