diff --git a/.agents/skills/build-openshell-mxc-windows/SKILL.md b/.agents/skills/build-openshell-mxc-windows/SKILL.md new file mode 100644 index 0000000000..a2db0a635c --- /dev/null +++ b/.agents/skills/build-openshell-mxc-windows/SKILL.md @@ -0,0 +1,339 @@ +--- +name: build-openshell-mxc-windows +description: Maintain and validate OpenShell's build-only Windows MSVC lane for x64 and ARM64. Use when working on Windows compilation, `windows:*` mise tasks, unsupported Windows compute-driver contracts, or Windows build reports. This skill does not implement Docker, Kubernetes, Podman, VM, MXC driver, policy translation, MSI, service, or supervisor runtime support on Windows. +--- + +# Build OpenShell-MXC for Windows + +This skill maintains the existing native Windows MSVC build lane in the +OpenShell repository. The Windows lane is already present in `main`; do not +treat this skill as a first-time porting recipe unless the user explicitly asks +for a new fork or a from-scratch bring-up. + +The lane is build-only. It validates that OpenShell can compile and test on +Windows MSVC for the supported deliverables: + +- `openshell-gateway.exe` +- `openshell.exe` + +It intentionally does not make Windows a Docker, Kubernetes, Podman, or VM +runtime host. + +## Current Repository Shape + +The Windows build lane is implemented by these tracked files: + +| Path | Purpose | +|---|---| +| `tasks/windows.toml` | Mise task entry points for `windows:*` commands. | +| `tasks/rust.toml`, `tasks/test.toml`, and `tasks/markdown.toml` | Windows routing for compiler-bearing checks, explicit Unix-only test skips, and Markdown dependency setup. | +| `tasks/scripts/windows-msvc.ps1` | PowerShell wrapper that enters the Visual Studio developer environment and invokes Cargo. | +| `.github/workflows/windows-msvc.yml` | GitHub Actions scaffold for x64 and future ARM64 Windows validation. | +| `architecture/windows-msvc-build.md` | Design notes and validation contract. | +| `.agents/skills/build-openshell-mxc-windows/` | This skill and companion reference material. | + +Use the code that is already in the repo. Do not generate a parallel Windows +build system, duplicate the wrapper, or add repository automation that the user +did not request. + +## Scope + +In scope: + +- Refreshing a local checkout to the latest GitLab `main`. +- Maintaining `tasks/windows.toml` and `tasks/scripts/windows-msvc.ps1`. +- Running x64 and ARM64 MSVC checks. +- Building x64 and ARM64 release binaries for `openshell-gateway` and + `openshell`. +- Running workspace tests on a native x64 or ARM64 host. +- Running focused unsupported-driver contract tests. +- Reporting test counts, skipped/gated areas, warnings, artifacts, and logs. +- Keeping Linux and macOS build paths unchanged. +- Keeping unsupported Windows compute drivers explicit and testable. + +Out of scope: + +- Docker Desktop support on Windows. +- Kubernetes support on Windows. +- Podman, Podman machine, or Podman Desktop support on Windows. +- VM, Hyper-V, WSL, libkrun, or VM-backed sandbox execution on Windows. +- New MXC compute driver crate. +- OpenShell to MXC policy translation. +- Windows named-pipe driver IPC. +- Windows Credential Manager or DPAPI integration. +- MSI, WinGet, Windows service registration, or installer work. +- Windows supervisor runtime port. + +## Hard Rules + +- Do not enable Docker, Kubernetes, Podman, or VM runtimes on Windows. +- Do not build, package, ship, or smoke-test standalone Windows binaries for + unsupported compute drivers. +- Keep unsupported drivers in the Windows build graph only as library/config + stubs when needed by the gateway. +- Unsupported Windows runtime entry points must return a clear unsupported + error. +- Keep Windows-specific code behind `#[cfg(target_os = "windows")]`. +- Keep Unix/Linux-only code behind `#[cfg(unix)]` or + `#[cfg(target_os = "linux")]`. +- Do not modify the default Linux `mise run ci` path unless the user explicitly + asks for it. +- Use `mise run --skip-tools windows:*` for Windows validation. The Windows + toolchain is rustup plus Visual Studio Build Tools, not mise-provisioned Rust. +- Keep Unix `run` bodies unchanged when adding `run_windows` behavior to shared + pre-commit tasks. + +## Recommended Checkout Flow + +From the OpenShell checkout root, use: + +```powershell +git fetch gitlab main +git switch main +git merge --ff-only gitlab/main +git branch --set-upstream-to=gitlab/main main +git status --short --branch +``` + +If there are local changes, preserve or resolve them before refreshing. Do not +discard user work unless the user explicitly asks to clean the checkout. + +For automated GitHub-to-GitLab sync work, use a dedicated sync checkout outside +the user's active working repo. Accept the checkout path from the user or an +environment variable instead of hardcoding a local machine path, for example: + +```text + +``` + +## Prerequisites + +The lane targets a Windows host with Visual Studio Build Tools and rustup. + +| Requirement | Check | Notes | +|---|---|---| +| Windows 11 | `[System.Environment]::OSVersion.Version` | Build 26100+ is recommended for MXC-adjacent validation, but compilation can still surface useful errors on older hosts. | +| Visual Studio 2022 or newer | `where.exe cl.exe` from a Developer PowerShell | Build Tools, Community, Professional, and Enterprise editions work when the target C++ components are installed. The wrapper discovers `VsDevCmd.bat` through `OPENSHELL_VSDEVCMD`, `vswhere`, or installed release directories such as `18` and `2022`. | +| Visual C++ ARM64 tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Tools.ARM64 -property installationPath` | Required for native ARM64 check, build, and tests and for x64-to-ARM64 check/build. Tests always require a native runner. | +| Visual C++ ARM64 Spectre-mitigated libraries | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Runtimes.ARM64.Spectre -property installationPath` | Required by `regorus` through `msvc_spectre_libs`; the build fails when the selected MSVC toolset lacks `lib\spectre\arm64`. | +| Visual C++ Clang tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Llvm.Clang -property installationPath` | Provides host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for ARM64 crypto dependencies such as `ring` and `aws-lc-sys`. On ARM64, the wrapper uses `VC\Tools\Llvm\Arm64\bin`. | +| Visual C++ CMake tools | `vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.CMake.Project -property installationPath` | Provides CMake and Ninja. The x64-to-ARM64 path adds Ninja to `PATH` for native dependencies but keeps bundled Z3 on CMake's Visual Studio ARM64 generator with native MSVC `cl.exe`. | +| Windows SDK | `where.exe rc.exe` from a Developer PowerShell | Install an SDK containing target libraries and ARM64 tools. | +| Rust via rustup | `rustc --version` | Add each target being validated: `x86_64-pc-windows-msvc` and/or `aarch64-pc-windows-msvc`. The wrapper also adds the selected target. | +| mise | `mise --version` | Used as a task runner only. | +| Git | `git --version` | Needed for checkout and sync work. | +| PowerShell | `$PSVersionTable.PSVersion` | Windows PowerShell 5.1 works; PowerShell 7 is quieter with mise shell hooks. | + +Do not install Visual Studio, Rust, Docker, Kubernetes, Podman, WSL, or Hyper-V +from this skill. + +## Environment Variables + +| Variable | Default | Purpose | +|---|---|---| +| `OPENSHELL_VSDEVCMD` | unset | Optional explicit path to `VsDevCmd.bat`. | +| `OPENSHELL_MXC_SKIP_ARM64` | `0` | Set to `1` to skip ARM64 when using `all` tasks. | +| `OPENSHELL_WINDOWS_BUILD_JOBS` | `CARGO_BUILD_JOBS`, then `4` | Positive Cargo job limit used by the wrapper. | +| `CARGO_TARGET_DIR` | `target` under repo root | Override Cargo output location. Use a short absolute path when x64-to-ARM64 builds approach Windows path-length limits. | +| `Z3_LIBRARY_PATH_OVERRIDE` | unset | Directory containing an x64 system `libz3.lib`; not valid for ARM64. | +| `Z3_SYS_Z3_HEADER` | unset | Full `z3.h` path required with a system Z3 library. | +| `Z3_SYS_BUNDLED_DIR_OVERRIDE` | pinned source cached under `CARGO_TARGET_DIR` when explicit, otherwise `%LOCALAPPDATA%\OpenShell\cache\z3` | Use an existing Z3 source tree containing `src/api/z3.h`; otherwise the wrapper fetches the pinned revision through Git and sets this automatically. | +| `RUSTC_WRAPPER` | cleared by wrapper | The wrapper clears inherited values because `--skip-tools` does not provision `sccache`. | + +Legacy fork variables such as `OPENSHELL_UPSTREAM`, +`OPENSHELL_MXC_FORK_DIR`, and `OPENSHELL_MXC_FORK_BRANCH` are no longer part +of the normal maintenance workflow. Use them only if the user explicitly asks +for a new disposable fork. + +## Validation Workflow + +Run the smallest useful slice first, then broaden: + +```powershell +mise run --skip-tools windows:check:x64 +mise run --skip-tools windows:check:arm64 +mise run --skip-tools windows:build:x64 +mise run --skip-tools windows:build:arm64 +mise run --skip-tools windows:test:x64 +mise run --skip-tools windows:test:unsupported:x64 +``` + +For full validation, detect the Windows host architecture first and choose the +native lane dynamically: + +```powershell +$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture +switch ($arch.ToString()) { + "X64" { + mise run --skip-tools windows:ci + } + "Arm64" { + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:build:arm64 + mise run --skip-tools windows:test:arm64 + mise run --skip-tools windows:test:unsupported:arm64 + mise run --skip-tools windows:artifacts + } + default { + throw "Unsupported Windows host architecture for OpenShell MSVC validation: $arch" + } +} +``` + +On x64 hosts, `windows:ci` is the full current CI contract and runs in this +order: + +1. x64 check. +2. ARM64 check, unless `OPENSHELL_MXC_SKIP_ARM64=1`. +3. x64 release build. +4. ARM64 release build, unless skipped. +5. Native x64 workspace tests. +6. Focused unsupported-driver contract tests. +7. Artifact reporting. + +The ARM64 check/build steps in this x64-host contract are cross-builds. The +wrapper discovers and adds host-native LLVM and Ninja to `PATH`, requires the +ARM64 compiler and Spectre-mitigated libraries, lets ARM64 crypto crates select +`clang-cl`, and keeps bundled Z3 on native MSVC `cl.exe` with CMake's Visual +Studio ARM64 generator. Z3 does not use Ninja because `z3-sys 0.10.9` passes +the MSBuild-only `-m` argument. + +On ARM64 hosts, validate the native ARM64 check, build, and test path. The +wrapper rejects test targets that do not match the host architecture, so x64 +compatibility under emulation is not part of these tasks. The aggregate +`windows:ci` task remains the x64-host CI contract; run the explicit ARM64 +commands above on an ARM64 host. + +The repository-wide `mise run pre-commit` task is also supported on Windows. +Its Rust check, Clippy, and test dependencies enter the same MSVC environment +for the native host target and clear inherited `RUSTC_WRAPPER`. Linux glibc +installer tests and Linux service/RPM packaging-asset tests skip explicitly; +the Linux build-environment shell-helper test also skips; cross-platform checks +continue to run. The blocking Windows Clippy pass excludes unsupported +Windows runtime packages as top-level targets. It allows only unused imports, +dead code, and unused async functions that result from cfg-gated Windows stubs; +other warnings remain errors. + +The wrapper limits Cargo to four jobs by default and serializes wrapper-owned +Cargo commands with a host-local mutex. It deliberately does not set `CL` or +`_CL_`: those variables are also consumed by `clang-cl`, where a global MSVC +option such as `/MP4` can be interpreted as an input file and break ARM64 +crypto dependency builds. + +## Expected Task Behavior + +| Task | Expected behavior | +|---|---| +| `windows:check:x64` | `cargo check --workspace` for `x86_64-pc-windows-msvc`, excluding unsupported Windows packages as top-level workspace targets. | +| `windows:check:arm64` | `cargo check --workspace` for `aarch64-pc-windows-msvc`, with the same top-level exclusions. | +| `windows:build:x64` | Release-builds `openshell-gateway.exe` and `openshell.exe` for x64. | +| `windows:build:arm64` | Release-builds `openshell-gateway.exe` and `openshell.exe` for ARM64. | +| `windows:test:x64` | Runs native x64 workspace tests with `--no-fail-fast`, excluding unsupported Windows packages as top-level workspace targets. | +| `windows:test:arm64` | Runs native ARM64 workspace tests with `--no-fail-fast` and the same package exclusions. Rejects non-ARM64 hosts. | +| `windows:test:unsupported:x64` | Re-runs focused `openshell-server` tests for unsupported Windows driver behavior. | +| `windows:test:unsupported:arm64` | Re-runs the same focused contracts natively on ARM64. Rejects non-ARM64 hosts. | +| `windows:artifacts` | Reports size and SHA256 for release artifacts that exist. | +| `windows:ci` | Runs the full ordered x64-host Windows CI lane, plus ARM64 check/build when not skipped. | + +The unsupported driver package excludes are intentional. They prevent standalone +driver crates from being top-level Windows check/test targets while still +allowing Windows stubs to compile through gateway dependencies. + +## Unsupported Driver Contract + +Windows must continue to reject unsupported compute drivers clearly. + +| Driver | Windows build behavior | Runtime behavior | +|---|---|---| +| Docker | Config/library stub may compile as a gateway dependency. | Gateway construction returns unsupported. | +| Kubernetes | Config/library stub may compile as a gateway dependency. | Gateway construction returns unsupported. | +| Podman | Config/library stub may compile as a gateway dependency. | Gateway construction returns unsupported. | +| VM | Server-side VM path compiles. | VM spawn returns unsupported. | + +The focused contract tasks for either native architecture run: + +```text +windows_compute_driver_stubs_report_unsupported +windows_spawn_reports_unsupported +``` + +These tests are also included in the full x64 workspace test run; the focused +task intentionally re-runs them so unsupported Windows behavior is visible in +the CI report. + +## Test Accounting Guidance + +When reporting `windows:ci`, distinguish these categories: + +- Passed tests from the full x64 workspace test log. +- Passed tests from the full ARM64 workspace test log when run on a native + ARM64 host. +- The two focused unsupported-contract re-runs. +- Explicit Cargo ignored tests, usually ignored doc examples. +- Tests hidden by `#[cfg(not(target_os = "windows"))]`; these often appear as + `running 0 tests`, not as ignored tests. +- Test-name `filtered out` counts from focused `cargo test` invocations. +- Package-level exclusions for unsupported Windows crates; Cargo does not report + those as ignored tests. + +Useful log files: + +| Log | Meaning | +|---|---| +| `build-x86_64-pc-windows-msvc-check.log` | x64 check output. | +| `build-aarch64-pc-windows-msvc-check.log` | ARM64 check output. | +| `build-x86_64-pc-windows-msvc-release.log` | x64 release build output. | +| `build-aarch64-pc-windows-msvc-release.log` | ARM64 release build output. | +| `test-x86_64-pc-windows-msvc.log` | Full native x64 workspace test output. | +| `test-aarch64-pc-windows-msvc.log` | Full native ARM64 workspace test output. | +| `test-x86_64-pc-windows-msvc-unsupported-*.log` | Focused unsupported-driver contract output. | +| `test-aarch64-pc-windows-msvc-unsupported-*.log` | Focused native ARM64 contract output. | + +The first bundled-Z3 check or test can spend several minutes in CMake/MSBuild +without much console output because Cargo output is redirected to the log. Look +for native `MSBuild.exe` workers before treating the process as stalled. The +wrapper fetches the pinned Z3 source through Git before Cargo starts. It caches +under an explicitly configured `CARGO_TARGET_DIR`, or under the current user's +local application data directory when Cargo uses its default target tree. +Concurrent commands publish the validated source through an atomic directory +rename, so x64 and ARM64 validation can share the cache safely. The wrapper does +not rely on the rate-limited GitHub Contents API used by `z3-sys`. A failed +fetch reports the partial checkout path for diagnosis. The artifact report +computes SHA256 through .NET directly and does not rely on the +`Get-FileHash` module being available inside the mise-launched Windows +PowerShell process. + +## Common Fix Patterns + +When Windows validation fails: + +1. Identify whether the error is from a top-level Windows deliverable, a + gateway dependency stub, or a Unix-only module leaking into the Windows build. +2. Prefer existing local patterns in the same crate. +3. Gate Unix imports and modules with `#[cfg(unix)]` or + `#[cfg(target_os = "linux")]`. +4. Add or preserve Windows stubs that return unsupported errors. +5. Keep Linux behavior unchanged. +6. Run `cargo fmt --all`, `git diff --check`, and the relevant `windows:*` + tasks after changes. + +Do not add broad abstractions or new Windows runtime support to satisfy a build +error. If a missing runtime feature is required, stop and propose a follow-on +skill or design doc. + +## Final Report Checklist + +Every substantial Windows build run should report: + +| Item | Required detail | +|---|---| +| Git state | Branch, latest GitLab commit, and whether local changes existed. | +| Host preconditions | OS, Rust, MSVC discovery, and notable warnings. | +| Commands run | Exact `mise run --skip-tools windows:*` commands. | +| x64 check/build | Pass/fail and log path. | +| ARM64 check/build | Pass/fail/skipped and log path. | +| Native tests | Passed/failed/ignored/filtered counts and log path for the host architecture. | +| Unsupported contracts | Which focused tests ran and their result. | +| Artifacts | Binary paths, size, and SHA256 when available. | +| Skips | Explicitly explain tests not run for a non-native architecture, unsupported driver package exclusions, and Windows cfg-gated tests. | +| Follow-ups | Only concrete follow-ups tied to failures or requested scope. | diff --git a/.agents/skills/build-openshell-mxc-windows/reference.md b/.agents/skills/build-openshell-mxc-windows/reference.md new file mode 100644 index 0000000000..38121609c2 --- /dev/null +++ b/.agents/skills/build-openshell-mxc-windows/reference.md @@ -0,0 +1,225 @@ +# Reference: Windows MSVC maintenance lane + +Companion to [SKILL.md](SKILL.md). Use this file for quick lookup while +maintaining the existing build-only Windows MSVC lane. + +## Lane Files + +| File | Purpose | +|---|---| +| `tasks/windows.toml` | Mise task definitions for `windows:*`. | +| `tasks/scripts/windows-msvc.ps1` | Visual Studio environment discovery, rustup target setup, Cargo invocation, logs, artifact report. | +| `.github/workflows/windows-msvc.yml` | GitHub Actions x64 job and disabled ARM64 scaffold. | +| `architecture/windows-msvc-build.md` | Human-readable design contract. | + +## Commands + +Use `--skip-tools` for all Windows mise tasks: + +```powershell +mise run --skip-tools windows:check:x64 +mise run --skip-tools windows:check:arm64 +mise run --skip-tools windows:build:x64 +mise run --skip-tools windows:build:arm64 +mise run --skip-tools windows:test:x64 +mise run --skip-tools windows:test:arm64 +mise run --skip-tools windows:test:unsupported:x64 +mise run --skip-tools windows:test:unsupported:arm64 +mise run --skip-tools windows:ci +``` + +For host-native full validation, detect architecture first: + +```powershell +$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture +if ($arch -eq [System.Runtime.InteropServices.Architecture]::Arm64) { + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:build:arm64 + mise run --skip-tools windows:test:arm64 + mise run --skip-tools windows:test:unsupported:arm64 + mise run --skip-tools windows:artifacts +} else { + mise run --skip-tools windows:ci +} +``` + +The native test tasks reject a target that does not match the host architecture. +Do not report x64 compatibility-under-emulation coverage from an ARM64 run. + +The wrapper adds missing rustup targets and clears inherited +`RUSTC_WRAPPER`. It does not install Visual Studio, Rust, Docker, Kubernetes, +Podman, WSL, Hyper-V, or VM tooling. + +On Windows, `mise run pre-commit` routes `rust:check`, `rust:lint`, and +`test:rust` through this wrapper for the host-native target. The shared task +definitions retain their existing Unix commands. Only tests for Linux glibc +installer behavior, Linux build-environment shell helpers, and Linux +service/RPM packaging assets skip on Windows. The Windows Clippy command +excludes unsupported runtime packages as top-level targets and allows only +unused imports, dead code, and unused async functions caused by cfg-gated +Windows stubs; other warnings remain errors. + +The wrapper limits Cargo to four jobs by default and serializes wrapper-owned +Cargo commands with a host-local mutex. It does not set `CL` or `_CL_` because +`clang-cl` also consumes them and can parse a global `/MP4` option as an input +file. + +For ARM64, verify the Visual Studio instance contains the ARM64 MSVC tools, +ARM64 Spectre-mitigated libraries, Clang tools, CMake tools, and a Windows SDK. +Clang supplies host-native `libclang.dll` for `bindgen` and `clang-cl.exe` for +ARM64 crypto dependencies such as `ring` and `aws-lc-sys`. Native ARM64 uses +the normal bundled-Z3 CMake path. An x64-to-ARM64 check/build discovers and +adds host-native Ninja to `PATH`, while the crypto crates select `clang-cl`. +Bundled Z3 uses CMake's Visual Studio ARM64 generator with native MSVC `cl.exe` +because `z3-sys 0.10.9` passes the MSBuild-only `-m` argument. Use a short +`CARGO_TARGET_DIR` if Windows path-length limits are reached. + +## Unsupported Driver Rules + +Windows is a build target only. These runtimes remain unsupported: + +- Docker +- Kubernetes +- Podman +- VM + +Rules: + +- Keep config/library stubs where the gateway needs them. +- Return clear unsupported errors at runtime. +- Do not build standalone Windows driver binaries. +- Do not add Docker Desktop, WSL, Hyper-V, Podman machine, Podman Desktop, or + VM-backed execution as part of this skill. + +Current focused unsupported-contract tests: + +```text +windows_compute_driver_stubs_report_unsupported +windows_spawn_reports_unsupported +``` + +Run them with the architecture-specific focused task on the native host. + +## Cargo Excludes + +The Windows wrapper intentionally excludes unsupported driver packages as +top-level workspace targets for check/test: + +```text +--exclude openshell-driver-docker +--exclude openshell-driver-kubernetes +--exclude openshell-driver-podman +--exclude openshell-driver-vm +--exclude openshell-supervisor-process +``` + +This does not mean all driver code disappears from the build graph. Docker, +Kubernetes, and Podman stubs can still compile as gateway dependencies. The +process supervisor is also excluded because its runtime is not supported on +Windows. + +## Common Errors + +### Unix imports leak into Windows builds + +Symptoms: + +```text +unresolved import std::os::unix +unresolved import tokio::net::UnixListener +unresolved import nix::... +``` + +Fix pattern: + +```rust +#[cfg(unix)] +use tokio::net::{UnixListener, UnixStream}; +``` + +Move Unix-only functions into Unix-only modules, or add a Windows stub that +returns an unsupported error. + +### Linux-only dependency reaches Windows + +Symptoms: + +```text +failed to run custom build command for libseccomp-sys +pkg-config could not find libsecret +``` + +Fix pattern: + +```toml +[target.'cfg(target_os = "linux")'.dependencies] +libseccomp = "..." +``` + +Only gate the dependency if no Windows path should use it. + +### ARM64 check fails but x64 passes + +Likely causes: + +- Native dependency does not support `aarch64-pc-windows-msvc`. +- ARM64 MSVC or Spectre-mitigated libraries are missing. +- Host-native `clang-cl`, Ninja, or CMake is missing during an x64-to-ARM64 build. +- `CL` or `_CL_` injects a global MSVC option such as `/MP4` into `clang-cl`. +- Build script assumes x64 tools. +- Inline assembly or prebuilt artifact lacks ARM64 handling. + +Do not skip ARM64 silently. Either fix the target handling or report the exact +blocked dependency. + +### Focused tests report many filtered-out tests + +This is expected for `windows:test:unsupported:x64`. Cargo runs one named test +and filters the other `openshell-server` tests. Report these as filtered, not +ignored. + +## Reporting Counts + +Use the log summaries from: + +| Log | Count source | +|---|---| +| `test-x86_64-pc-windows-msvc.log` | Full x64 workspace test pass. | +| `test-aarch64-pc-windows-msvc.log` | Full native ARM64 workspace test pass. | +| `test-x86_64-pc-windows-msvc-unsupported-*.log` | Focused unsupported-contract re-runs and filtered counts. | +| `test-aarch64-pc-windows-msvc-unsupported-*.log` | Focused native ARM64 re-runs and filtered counts. | + +Separate: + +- passed +- failed +- ignored +- filtered out +- cfg-gated zero-test targets +- package-level excludes + +Package-level excludes are not printed as ignored tests by Cargo. + +## Final Sanity Checks + +Before committing Windows-lane changes, choose checks based on the host +architecture: + +```powershell +cargo fmt --all +git diff --check +$arch = [System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture +if ($arch -eq [System.Runtime.InteropServices.Architecture]::Arm64) { + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:build:arm64 + mise run --skip-tools windows:test:arm64 + mise run --skip-tools windows:test:unsupported:arm64 +} else { + mise run --skip-tools windows:check:x64 + mise run --skip-tools windows:check:arm64 + mise run --skip-tools windows:test:unsupported:x64 +} +``` + +Run the full x64-host `windows:ci` lane when build or test behavior changed and +the host can run that lane natively. diff --git a/.github/workflows/windows-msvc.yml b/.github/workflows/windows-msvc.yml new file mode 100644 index 0000000000..076af08a82 --- /dev/null +++ b/.github/workflows/windows-msvc.yml @@ -0,0 +1,36 @@ +name: Windows MSVC (build-only) +on: + push: + branches: [main] + pull_request: +jobs: + x64: + runs-on: windows-2025 + steps: + - uses: actions/checkout@v4 + - uses: jdx/mise-action@v3 + with: + install: false + experimental: true + - uses: dtolnay/rust-toolchain@stable + with: + targets: x86_64-pc-windows-msvc + - run: mise run --skip-tools windows:check:x64 + - run: mise run --skip-tools windows:build:x64 + - run: mise run --skip-tools windows:test:x64 + - run: mise run --skip-tools windows:test:unsupported:x64 + arm64: + # TODO: provision a windows-arm64 self-hosted runner + runs-on: [self-hosted, windows-arm64] + if: false # flip to true once the runner is online + steps: + - uses: actions/checkout@v4 + - uses: jdx/mise-action@v3 + with: + install: false + experimental: true + - uses: dtolnay/rust-toolchain@stable + with: + targets: aarch64-pc-windows-msvc + - run: mise run --skip-tools windows:check:arm64 + - run: mise run --skip-tools windows:build:arm64 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0465f88f7d..874e9610b5 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -84,6 +84,7 @@ Skills live in `.agents/skills/`. Your agent's harness can discover and load the | Platform | `generate-sandbox-policy` | Generate YAML sandbox policies from requirements or API docs | | Platform | `helm-dev-environment` | Start and manage the local Kubernetes development environment | | Platform | `tui-development` | Development guide for the ratatui-based terminal UI | +| Platform | `build-openshell-mxc-windows` | Maintain and validate the build-only x64 and ARM64 Windows MSVC lane | | Documentation | `update-docs` | Scan recent commits and draft doc updates for user-facing changes | | Maintenance | `sync-agent-infra` | Detect and fix drift across agent-first infrastructure files | | Reference | `sbom` | Generate SBOMs and resolve dependency licenses | @@ -130,30 +131,18 @@ Project requirements: - Rust 1.90+ - Python 3.11+ - Docker (running) -- Z3 solver library (for the policy prover crate) -### macOS build tools - -Install Apple Command Line Tools before building locally: - -```bash -xcode-select --install -``` +The common local development workflow runs on macOS and Linux. The +`openshell-cli` crate also supports x86-64 Windows builds with the MSVC toolchain +(`x86_64-pc-windows-msvc`). Windows builds require the Visual Studio C++ build +tools and LLVM/libclang for crates that generate C bindings. -If Cargo fails while building `protobuf-src` with an error such as -`fatal error: 'utility' file not found`, `fatal error: 'cstdlib' file not -found`, or `A compiler with support for C++11 language features is required`, -your Command Line Tools install may not expose the libc++ headers on the -compiler's default include path. Reinstall Command Line Tools to correct the error: - -```bash -sudo rm -rf /Library/Developer/CommandLineTools -xcode-select --install -``` +The policy prover uses Z3 on supported targets. ### Z3 installation -The `openshell-prover` crate links against the system Z3 library via pkg-config. +The `openshell-prover` crate links against Z3. On macOS and Linux, install the +system Z3 development package; `z3-sys` discovers it through `pkg-config`. ```bash # macOS @@ -166,12 +155,46 @@ sudo apt install libz3-dev sudo dnf install z3-devel ``` -If you prefer not to install Z3 system-wide, you can compile it from source as a one-time step: +If you prefer not to install Z3 system-wide, use the bundled Z3 feature. This +compiles Z3 from source during the Rust build: ```bash cargo build -p openshell-prover --features bundled-z3 ``` +For x86-64 Windows MSVC builds, use one of these Z3 paths: + +- System Z3: point `Z3_LIBRARY_PATH_OVERRIDE` at the directory containing the + 64-bit MSVC Z3 library and `Z3_SYS_Z3_HEADER` at the full path to `z3.h`. + The `windows:*` tasks use this path automatically when `Z3_LIBRARY_PATH_OVERRIDE` + is set. +- Bundled Z3: pass `--features bundled-z3` so `z3-sys` builds Z3 from source. + +Both Windows paths still require `libclang.dll` for `bindgen`. If LLVM is not on +the default search path, set `LIBCLANG_PATH` to the directory containing +`libclang.dll`. + +```powershell +$env:LIBCLANG_PATH='C:\Program Files\Microsoft Visual Studio\2022\\VC\Tools\Llvm\x64\bin' +cargo build -p openshell-cli --target x86_64-pc-windows-msvc --features bundled-z3 +``` + +To use a local x64 Z3 release with the Windows task wrapper: + +```powershell +$env:Z3_LIBRARY_PATH_OVERRIDE='C:\path\to\z3-4.16.0-x64-win\bin' +$env:Z3_SYS_Z3_HEADER='C:\path\to\z3-4.16.0-x64-win\include\z3.h' +mise run --skip-tools windows:build:x64 +``` + +### macOS build tools + +Install Apple Command Line Tools before building locally: + +```bash +xcode-select --install +``` + ## Getting Started ```bash diff --git a/Cargo.lock b/Cargo.lock index 31e2104987..e2ee2687dd 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -258,15 +258,6 @@ version = "1.5.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8" -[[package]] -name = "autotools" -version = "0.2.7" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "ef941527c41b0fc0dd48511a8154cd5fc7e29200a0ff8b7203c5d777dbc795cf" -dependencies = [ - "cc", -] - [[package]] name = "aws-config" version = "1.8.15" @@ -3893,7 +3884,7 @@ dependencies = [ "nix", "prost", "prost-types", - "protobuf-src", + "protoc-bin-vendored", "reqwest 0.12.28", "serde", "serde_json", @@ -4106,8 +4097,11 @@ dependencies = [ name = "openshell-sandbox" version = "0.0.0" dependencies = [ + "anyhow", "clap", "futures", + "hex", + "hmac", "miette", "nix", "openshell-core", @@ -4119,11 +4113,15 @@ dependencies = [ "openshell-supervisor-process", "prost", "prost-types", + "rand_core 0.6.4", + "russh", "rustls 0.23.38", "serde", "serde_json", + "sha2 0.10.9", "temp-env", "tempfile", + "thiserror 2.0.18", "tokio", "tokio-tungstenite 0.26.2", "tonic", @@ -4939,14 +4937,69 @@ dependencies = [ ] [[package]] -name = "protobuf-src" -version = "1.1.0+21.5" +name = "protoc-bin-vendored" +version = "3.2.0" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "c7ac8852baeb3cc6fb83b93646fb93c0ffe5d14bf138c945ceb4b9948ee0e3c1" +checksum = "d1c381df33c98266b5f08186583660090a4ffa0889e76c7e9a5e175f645a67fa" dependencies = [ - "autotools", + "protoc-bin-vendored-linux-aarch_64", + "protoc-bin-vendored-linux-ppcle_64", + "protoc-bin-vendored-linux-s390_64", + "protoc-bin-vendored-linux-x86_32", + "protoc-bin-vendored-linux-x86_64", + "protoc-bin-vendored-macos-aarch_64", + "protoc-bin-vendored-macos-x86_64", + "protoc-bin-vendored-win32", ] +[[package]] +name = "protoc-bin-vendored-linux-aarch_64" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c350df4d49b5b9e3ca79f7e646fde2377b199e13cfa87320308397e1f37e1a4c" + +[[package]] +name = "protoc-bin-vendored-linux-ppcle_64" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a55a63e6c7244f19b5c6393f025017eb5d793fd5467823a099740a7a4222440c" + +[[package]] +name = "protoc-bin-vendored-linux-s390_64" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1dba5565db4288e935d5330a07c264a4ee8e4a5b4a4e6f4e83fad824cc32f3b0" + +[[package]] +name = "protoc-bin-vendored-linux-x86_32" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8854774b24ee28b7868cd71dccaae8e02a2365e67a4a87a6cd11ee6cdbdf9cf5" + +[[package]] +name = "protoc-bin-vendored-linux-x86_64" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b38b07546580df720fa464ce124c4b03630a6fb83e05c336fea2a241df7e5d78" + +[[package]] +name = "protoc-bin-vendored-macos-aarch_64" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "89278a9926ce312e51f1d999fee8825d324d603213344a9a706daa009f1d8092" + +[[package]] +name = "protoc-bin-vendored-macos-x86_64" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "81745feda7ccfb9471d7a4de888f0652e806d5795b61480605d4943176299756" + +[[package]] +name = "protoc-bin-vendored-win32" +version = "3.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "95067976aca6421a523e491fce939a3e65249bac4b977adee0ee9771568e8aa3" + [[package]] name = "pulldown-cmark" version = "0.13.3" @@ -5450,6 +5503,7 @@ dependencies = [ "pkcs8 0.10.2", "rand 0.9.4", "rand_core 0.10.0-rc-3", + "ring", "rsa 0.10.0-rc.12", "russh-cryptovec", "russh-util", diff --git a/Cargo.toml b/Cargo.toml index 4ec6a0d44f..139896ecfc 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -108,7 +108,7 @@ futures = "0.3" bytes = "1" pin-project-lite = "0.2" tokio-stream = "0.1" -protobuf-src = "1.1.0" +protoc-bin-vendored = "3.2.0" url = "2" indexmap = "2" diff --git a/architecture/README.md b/architecture/README.md index 3fc72afd25..9453d59826 100644 --- a/architecture/README.md +++ b/architecture/README.md @@ -167,6 +167,7 @@ that crate's `README.md`. | [Compute Runtimes](compute-runtimes.md) | Docker, Podman, Kubernetes, VM, sandbox images, and runtime-specific responsibilities. | | [Build](build.md) | Build artifacts, CI/E2E, docs site validation, and release packaging. | | [Google Vertex AI Provider](google-vertex-ai-provider.md) | Implementation reference for the `google-vertex-ai` provider, from CLI through gateway to sandbox. | +| [Windows MSVC Build](windows-msvc-build.md) | Build-only native Windows MSVC lane (x64/ARM64) and unsupported-runtime behavior on Windows. | ## `rfc/` vs `architecture/` diff --git a/architecture/windows-msvc-build.md b/architecture/windows-msvc-build.md new file mode 100644 index 0000000000..9c619788ec --- /dev/null +++ b/architecture/windows-msvc-build.md @@ -0,0 +1,145 @@ +# Windows MSVC Build Design + +This page records the design decisions for the native Windows MSVC build lane. +It is intentionally build-only. It does not make Windows a Docker, Kubernetes, +Podman, or VM runtime host. + +## Goals + +- Compile the OpenShell gateway and CLI for `x86_64-pc-windows-msvc` and `aarch64-pc-windows-msvc`. +- Keep the Linux and macOS build paths unchanged. +- Preserve gateway configuration parsing for all existing compute driver names. +- Return clear unsupported errors when a Windows gateway is configured to use Docker, Kubernetes, Podman, or VM. +- Keep dedicated `windows:*` validation tasks while allowing the repository-wide + `pre-commit` task to delegate compiler-bearing Rust checks to the native + Windows MSVC environment. + +## Non-Goals + +- Do not support Docker Desktop, WSL, Hyper-V, Podman machine, Podman Desktop, Kubernetes, or VM-backed sandbox execution on Windows. +- Do not ship Windows standalone binaries for Docker, Kubernetes, Podman, or VM drivers. +- Do not implement named-pipe driver IPC, Windows services, MSI packaging, Credential Manager integration, DPAPI integration, or MXC policy translation in this build lane. + +## Unsupported Driver Strategy + +Unsupported compute drivers use contract stubs on Windows. The stubs preserve +configuration structs and public library entry points so the gateway can parse +existing config files and reject unsupported driver selection with a clear error. + +The Windows lane does not build, release, package, or smoke-test standalone +driver binaries for Docker, Kubernetes, Podman, or VM. Those binaries are Linux +or macOS deliverables only. + +| Driver | Windows build behavior | Runtime behavior | +|---|---|---| +| Docker | Library config stub compiles as a gateway dependency. | Gateway construction returns unsupported. | +| Kubernetes | Library config stub compiles as a gateway dependency. | Gateway construction returns unsupported. | +| Podman | Library config stub compiles as a gateway dependency. | Gateway construction returns unsupported. | +| VM | Library config stub compiles as a gateway dependency. | VM spawn returns unsupported. | + +This keeps Windows behavior explicit without carrying runtime dependencies or +creating misleading Windows driver artifacts. + +## Mise Lane + +Windows validation is exposed through `tasks/windows.toml`: + +| Task | Purpose | +|---|---| +| `windows:check:x64` | Check the x64 MSVC gateway/CLI build graph. | +| `windows:check:arm64` | Check the ARM64 MSVC gateway/CLI build graph. | +| `windows:build:x64` | Build release x64 `openshell-gateway.exe` and `openshell.exe`. | +| `windows:build:arm64` | Build release ARM64 `openshell-gateway.exe` and `openshell.exe`. | +| `windows:test:x64` | Run native x64 workspace tests, excluding unsupported Windows packages as top-level test targets. | +| `windows:test:arm64` | Run native ARM64 workspace tests with the same package exclusions. | +| `windows:test:unsupported:x64` | Run focused server/runtime tests for unsupported driver contracts. | +| `windows:test:unsupported:arm64` | Run the same focused contracts natively on ARM64. | +| `windows:ci` | Run check, build, test, unsupported-contract tests, and artifact reporting. | + +The Windows tasks call `tasks/scripts/windows-msvc.ps1`. The wrapper discovers +Visual Studio's `VsDevCmd.bat` with `vswhere` or by enumerating installed +release directories, validates the requested compiler and ARM64 Spectre +libraries, adds rustup MSVC targets, clears inherited `RUSTC_WRAPPER`, and +keeps build artifacts under the normal Cargo target tree. +On Windows, the generic `rust:check`, `rust:lint`, and `test:rust` tasks call +the same wrapper with the host-native MSVC target. The wrapper preserves the +Unix Cargo commands on Linux and macOS, excludes unsupported Windows runtime +packages, and runs the server test-support suite separately. Windows Clippy +continues to deny all warnings except unused imports, dead code, and unused +async functions caused by cfg-gated Windows stubs. Repository-wide pre-commit +skips only Linux-specific installer, build-environment shell-helper, and +packaging-asset tests; its +cross-platform Python, Markdown, license, and documentation checks still run. +Test tasks require the Rust target architecture to match the Windows host, so +an ARM64 test result is native coverage rather than x64 emulation coverage. +By default it enables bundled Z3 for reproducible Windows builds. When +`Z3_LIBRARY_PATH_OVERRIDE` points at a directory containing `libz3.lib`, the +wrapper uses that system Z3 instead and requires `Z3_SYS_Z3_HEADER` to point at +the full path to `z3.h`. For bundled builds, the wrapper fetches the Z3 source +revision pinned by `z3-sys` through Git and sets +`Z3_SYS_BUNDLED_DIR_OVERRIDE`. When `CARGO_TARGET_DIR` is explicit, the wrapper +uses it for the source cache. Otherwise, it caches under the current user's +local application data directory, outside the checkout. Publishing uses an +atomic directory rename so concurrent x64 and ARM64 commands can share the +cache safely. This keeps downloaded sources outside the checkout by default and +avoids the unauthenticated GitHub API lookup in the `z3-sys` build script, which +can fail with HTTP 403 when a shared runner or developer network exhausts its +API rate limit. An explicitly set `Z3_SYS_BUNDLED_DIR_OVERRIDE` remains +supported and must contain `src/api/z3.h`. + +The lane uses `mise run --skip-tools windows:*` because Windows Rust comes from +rustup and linking comes from Visual Studio Build Tools. Mise orchestrates the +tasks; it does not own the Windows toolchain. + +ARM64 validation requires the Visual Studio ARM64 MSVC tools, ARM64 +Spectre-mitigated libraries, host-native Clang tools, CMake tools, and an +ARM64-capable Windows SDK. Clang provides `libclang.dll` for `bindgen` and +`clang-cl.exe` for ARM64 crypto dependencies. During x64-to-ARM64 check/build, +the wrapper discovers and adds the Visual Studio-bundled Ninja to `PATH` for +native dependencies. It lets `cmake-rs` select the Visual Studio ARM64 +generator with native MSVC `cl.exe` for bundled Z3 so the Z3 build does not +inherit the crypto crates' compiler requirement. Z3 stays on the Visual Studio +generator because `z3-sys` emits an MSBuild-only `-m` argument that Ninja +rejects. Artifact hashing uses .NET SHA256 directly because module autoloading +in the mise-launched Windows PowerShell process is not guaranteed. + +The wrapper defaults Cargo compilation to four jobs. Set +`OPENSHELL_WINDOWS_BUILD_JOBS` to a positive integer to override that limit. +A host-local mutex serializes wrapper-owned Cargo commands so concurrent +pre-commit tasks do not multiply the process count while bundled Z3 compiles. +The wrapper does not set `CL` or `_CL_`: those variables are also consumed by +`clang-cl`, where MSVC's `/MP` option can be interpreted as an input file and +break ARM64 crypto dependency builds. + +## CI Shape + +The x64 GitHub Actions job runs on `windows-2025` and executes: + +```powershell +mise run --skip-tools windows:check:x64 +mise run --skip-tools windows:build:x64 +mise run --skip-tools windows:test:x64 +mise run --skip-tools windows:test:unsupported:x64 +``` + +The ARM64 check and release build in this x64 job are cross-builds. Native +ARM64 tests remain exclusive to an ARM64 runner. + +The ARM64 job is scaffolded but disabled until a Windows ARM64 runner is +available. Once enabled, it should run check, release build, native workspace +tests, and the focused unsupported-driver contracts for +`aarch64-pc-windows-msvc`. + +## Validation Contract + +A successful Windows build report should include: + +- x64 and ARM64 `cargo check` status. +- x64 and ARM64 release build status for `openshell-gateway.exe` and `openshell.exe`. +- x64 test summary. +- Native ARM64 test summary when validation runs on an ARM64 host. +- Focused unsupported-driver contract test status. +- Artifact size and SHA256 for each Windows binary. + +Warnings from Linux-only dead code are acceptable in this build-only phase when +they come from code paths intentionally disabled on Windows. diff --git a/crates/openshell-bootstrap/Cargo.toml b/crates/openshell-bootstrap/Cargo.toml index c860cb1384..96a7985085 100644 --- a/crates/openshell-bootstrap/Cargo.toml +++ b/crates/openshell-bootstrap/Cargo.toml @@ -11,7 +11,6 @@ rust-version.workspace = true [dependencies] openshell-core = { path = "../openshell-core", default-features = false } -bollard = "0.20" bytes = { workspace = true } futures = { workspace = true } miette = { workspace = true } @@ -24,6 +23,9 @@ tempfile = "3" tokio = { workspace = true } tracing = { workspace = true } +[target.'cfg(not(target_os = "windows"))'.dependencies] +bollard = "0.20" + [dev-dependencies] [lints] diff --git a/crates/openshell-bootstrap/src/build_windows.rs b/crates/openshell-bootstrap/src/build_windows.rs new file mode 100644 index 0000000000..93af1345d3 --- /dev/null +++ b/crates/openshell-bootstrap/src/build_windows.rs @@ -0,0 +1,23 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +//! Windows stub for local Dockerfile image builds. + +use std::collections::HashMap; +use std::path::Path; + +use miette::Result; + +// Keep this stub's signature aligned with the supported-platform implementation. +#[allow(clippy::implicit_hasher)] +pub async fn build_local_image( + _dockerfile_path: &Path, + _tag: &str, + _context_dir: &Path, + _build_args: &HashMap, + _on_log: &mut impl FnMut(String), +) -> Result<()> { + Err(miette::miette!( + "local Dockerfile sandbox sources are unsupported on Windows" + )) +} diff --git a/crates/openshell-bootstrap/src/lib.rs b/crates/openshell-bootstrap/src/lib.rs index aa0532260e..5598d524ee 100644 --- a/crates/openshell-bootstrap/src/lib.rs +++ b/crates/openshell-bootstrap/src/lib.rs @@ -1,6 +1,10 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +#[cfg(not(target_os = "windows"))] +pub mod build; +#[cfg(target_os = "windows")] +#[path = "build_windows.rs"] pub mod build; pub mod edge_token; pub mod jwt; diff --git a/crates/openshell-cli/Cargo.toml b/crates/openshell-cli/Cargo.toml index d7b8fd502c..0d1cdb531f 100644 --- a/crates/openshell-cli/Cargo.toml +++ b/crates/openshell-cli/Cargo.toml @@ -72,7 +72,6 @@ tokio-tungstenite = { workspace = true } # Streams futures = { workspace = true } tokio-stream = { workspace = true } -nix = { workspace = true } # URL parsing url = { workspace = true } @@ -84,6 +83,9 @@ tracing-subscriber = { workspace = true } [lints] workspace = true +[target.'cfg(unix)'.dependencies] +nix = { workspace = true } + [dev-dependencies] futures = { workspace = true } rcgen = { version = "0.13", features = ["crypto", "pem"] } diff --git a/crates/openshell-cli/src/main.rs b/crates/openshell-cli/src/main.rs index 00942d79ec..9f117f7407 100644 --- a/crates/openshell-cli/src/main.rs +++ b/crates/openshell-cli/src/main.rs @@ -2131,9 +2131,32 @@ enum WorkspaceMemberCommands { }, } -#[tokio::main] -#[allow(clippy::large_stack_frames)] // CLI dispatch holds many futures; OK at top level. -async fn main() -> Result<()> { +#[cfg(windows)] +fn main() -> Result<()> { + std::thread::Builder::new() + .name("openshell-main".to_string()) + .stack_size(8 * 1024 * 1024) + .spawn(run_main) + .map_err(|err| miette::miette!("failed to start OpenShell main thread: {err}"))? + .join() + .map_err(|_| miette::miette!("OpenShell main thread panicked"))? +} + +#[cfg(not(windows))] +fn main() -> Result<()> { + run_main() +} + +fn run_main() -> Result<()> { + tokio::runtime::Builder::new_multi_thread() + .enable_all() + .build() + .map_err(|err| miette::miette!("failed to build Tokio runtime: {err}"))? + .block_on(run_async()) +} + +#[allow(clippy::large_stack_frames)] // CLI dispatch holds many futures; run on an expanded Windows stack. +async fn run_async() -> Result<()> { // Install the rustls crypto provider before completion runs — completers may // establish TLS connections to the gateway. rustls::crypto::ring::default_provider() diff --git a/crates/openshell-cli/src/run.rs b/crates/openshell-cli/src/run.rs index 4843475e5d..8cdb22e5fc 100644 --- a/crates/openshell-cli/src/run.rs +++ b/crates/openshell-cli/src/run.rs @@ -1752,8 +1752,10 @@ impl Drop for RawModeGuard { } } +#[cfg(unix)] struct TaskGuard(tokio::task::JoinHandle<()>); +#[cfg(unix)] impl Drop for TaskGuard { fn drop(&mut self) { self.0.abort(); @@ -1768,7 +1770,9 @@ async fn sandbox_exec_interactive_grpc( timeout_seconds: u32, environment: &HashMap, ) -> Result { - use openshell_core::proto::{ExecSandboxInput, ExecSandboxWindowResize, exec_sandbox_input}; + #[cfg(unix)] + use openshell_core::proto::ExecSandboxWindowResize; + use openshell_core::proto::{ExecSandboxInput, exec_sandbox_input}; use tokio_stream::wrappers::ReceiverStream; let (cols, rows) = crossterm::terminal::size().unwrap_or((80, 24)); @@ -1807,31 +1811,26 @@ async fn sandbox_exec_interactive_grpc( // spawn_blocking) so the tokio runtime shutdown doesn't wait for a // thread blocked on stdin.read(). The thread exits when the channel // closes (blocking_send returns Err) or stdin hits EOF. - #[cfg(unix)] - { - let stdin_tx = input_tx.clone(); - std::thread::spawn(move || { - let mut stdin = std::io::stdin().lock(); - let mut buf = [0u8; 4096]; - loop { - match stdin.read(&mut buf) { - Ok(0) | Err(_) => break, - Ok(n) => { - if stdin_tx - .blocking_send(ExecSandboxInput { - payload: Some(exec_sandbox_input::Payload::Stdin( - buf[..n].to_vec(), - )), - }) - .is_err() - { - break; - } + let stdin_tx = input_tx.clone(); + std::thread::spawn(move || { + let mut stdin = std::io::stdin().lock(); + let mut buf = [0u8; 4096]; + loop { + match stdin.read(&mut buf) { + Ok(0) | Err(_) => break, + Ok(n) => { + if stdin_tx + .blocking_send(ExecSandboxInput { + payload: Some(exec_sandbox_input::Payload::Stdin(buf[..n].to_vec())), + }) + .is_err() + { + break; } } } - }); - } + } + }); // SIGWINCH handler: forward terminal resize events. #[cfg(unix)] @@ -6926,6 +6925,8 @@ mod tests { refresh_status_row, resolve_from, sandbox_should_persist, sandbox_upload_plan, service_expose_status_error, service_url_for_gateway, }; + #[cfg(unix)] + use super::{local_upload_path_exists, local_upload_path_is_symlink}; use crate::TEST_ENV_LOCK; use crate::commands::common::progress_step_from_metadata; use crate::test_utils::EnvVarGuard; diff --git a/crates/openshell-cli/src/ssh.rs b/crates/openshell-cli/src/ssh.rs index f8de99a692..4b16240bcb 100644 --- a/crates/openshell-cli/src/ssh.rs +++ b/crates/openshell-cli/src/ssh.rs @@ -2233,7 +2233,9 @@ mod tests { #[derive(Debug)] struct UploadArchiveEntry { path: String, + #[cfg_attr(not(unix), allow(dead_code))] entry_type: tar::EntryType, + #[cfg_attr(not(unix), allow(dead_code))] link_name: Option, } diff --git a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs index 9bac31da4d..be5717496d 100644 --- a/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs +++ b/crates/openshell-cli/tests/sandbox_create_lifecycle_integration.rs @@ -1,6 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +#![cfg(not(target_os = "windows"))] + mod helpers; use helpers::{ diff --git a/crates/openshell-core/Cargo.toml b/crates/openshell-core/Cargo.toml index 35a3732cf9..2fd737508b 100644 --- a/crates/openshell-core/Cargo.toml +++ b/crates/openshell-core/Cargo.toml @@ -40,7 +40,7 @@ telemetry = ["dep:reqwest", "dep:chrono"] [build-dependencies] tonic-prost-build = { workspace = true } -protobuf-src = { workspace = true } +protoc-bin-vendored = { workspace = true } [dev-dependencies] tempfile = "3" diff --git a/crates/openshell-core/build.rs b/crates/openshell-core/build.rs index 7955772a67..86fce73ef2 100644 --- a/crates/openshell-core/build.rs +++ b/crates/openshell-core/build.rs @@ -22,19 +22,23 @@ fn main() -> Result<(), Box> { // --- Protobuf compilation --- // Re-run when anything under proto/ changes (including newly added .proto files). println!("cargo:rerun-if-changed={PROTO_REL}"); - // Use bundled protoc from protobuf-src. The system protoc (from apt-get) - // does not bundle the well-known type includes (google/protobuf/struct.proto - // etc.), so we must use protobuf-src which ships both the binary and the - // include tree. + // Use a vendored protoc binary and include tree. System protoc installs + // often omit the well-known type includes (google/protobuf/struct.proto, + // etc.), and protobuf-src requires autotools/sh which breaks MSVC builds. // SAFETY: This is run at build time in a single-threaded build script context. // No other threads are reading environment variables concurrently. #[allow(unsafe_code)] unsafe { - env::set_var("PROTOC", protobuf_src::protoc()); + env::set_var("PROTOC", protoc_bin_vendored::protoc_bin_path()?); + env::set_var("PROTOC_INCLUDE", protoc_bin_vendored::include_path()?); } let manifest_dir = PathBuf::from(env::var("CARGO_MANIFEST_DIR")?); let proto_root = manifest_dir.join(PROTO_REL); + #[cfg(target_os = "windows")] + let proto_includes = proto_include_dirs(&proto_root)?; + #[cfg(not(target_os = "windows"))] + let proto_includes = vec![proto_root.clone()]; let mut proto_files = Vec::new(); collect_proto_files(&proto_root, &mut proto_files)?; @@ -50,7 +54,7 @@ fn main() -> Result<(), Box> { // Emit a binary FileDescriptorSet so the server can enumerate every // RPC at runtime (used by the per-handler auth exhaustiveness test). .file_descriptor_set_path(&descriptor_path) - .compile_protos(&proto_files, &[proto_root])?; + .compile_protos(&proto_files, &proto_includes)?; println!( "cargo:rustc-env=OPENSHELL_DESCRIPTOR_PATH={}", @@ -60,6 +64,13 @@ fn main() -> Result<(), Box> { Ok(()) } +#[cfg(target_os = "windows")] +fn proto_include_dirs(proto_root: &Path) -> Result, Box> { + let mut includes = vec![proto_root.to_path_buf()]; + includes.push(protoc_bin_vendored::include_path()?); + Ok(includes) +} + fn collect_proto_files(dir: &Path, out: &mut Vec) -> std::io::Result<()> { for entry in std::fs::read_dir(dir)? { let path = entry?.path(); diff --git a/crates/openshell-core/src/driver_mounts.rs b/crates/openshell-core/src/driver_mounts.rs index 086d992c37..6860be7eb7 100644 --- a/crates/openshell-core/src/driver_mounts.rs +++ b/crates/openshell-core/src/driver_mounts.rs @@ -67,11 +67,14 @@ pub fn validate_mount_subpath(subpath: &str) -> Result<(), String> { return Err("mount subpath must not contain NUL bytes".to_string()); } let path = Path::new(subpath); - if path.is_absolute() - || path - .components() - .any(|component| matches!(component, std::path::Component::ParentDir)) - { + if path.components().any(|component| { + matches!( + component, + std::path::Component::Prefix(_) + | std::path::Component::RootDir + | std::path::Component::ParentDir + ) + }) { return Err("mount subpath must be relative and must not contain '..'".to_string()); } Ok(()) diff --git a/crates/openshell-core/src/forward.rs b/crates/openshell-core/src/forward.rs index 70ab74edd0..b32dc305d4 100644 --- a/crates/openshell-core/src/forward.rs +++ b/crates/openshell-core/src/forward.rs @@ -1341,6 +1341,7 @@ mod tests { ); } + #[cfg(not(target_os = "windows"))] #[test] fn check_port_available_occupied_ipv6_wildcard() { // Bind on [::]:0 (IPv6 wildcard) — this simulates a server like diff --git a/crates/openshell-core/src/paths.rs b/crates/openshell-core/src/paths.rs index 9445347c79..df9deca8b7 100644 --- a/crates/openshell-core/src/paths.rs +++ b/crates/openshell-core/src/paths.rs @@ -20,6 +20,10 @@ pub fn xdg_config_dir() -> Result { if let Ok(path) = std::env::var("XDG_CONFIG_HOME") { return Ok(PathBuf::from(path)); } + #[cfg(target_os = "windows")] + if let Ok(path) = std::env::var("APPDATA") { + return Ok(PathBuf::from(path)); + } let home = std::env::var("HOME") .into_diagnostic() .wrap_err("HOME is not set")?; @@ -38,6 +42,10 @@ pub fn xdg_state_dir() -> Result { if let Ok(path) = std::env::var("XDG_STATE_HOME") { return Ok(PathBuf::from(path)); } + #[cfg(target_os = "windows")] + if let Ok(path) = std::env::var("LOCALAPPDATA") { + return Ok(PathBuf::from(path)); + } let home = std::env::var("HOME") .into_diagnostic() .wrap_err("HOME is not set")?; @@ -56,6 +64,10 @@ pub fn xdg_data_dir() -> Result { if let Ok(path) = std::env::var("XDG_DATA_HOME") { return Ok(PathBuf::from(path)); } + #[cfg(target_os = "windows")] + if let Ok(path) = std::env::var("LOCALAPPDATA") { + return Ok(PathBuf::from(path)); + } let home = std::env::var("HOME") .into_diagnostic() .wrap_err("HOME is not set")?; @@ -133,7 +145,8 @@ pub fn is_file_permissions_too_open(path: &Path) -> bool { /// /// This is a lexical normalization only — it does NOT resolve symlinks or /// check the filesystem. `..` components are preserved verbatim; callers that -/// need to reject parent traversal must validate separately. +/// need to reject parent traversal must validate separately. The normalized +/// representation always uses `/` so sandbox policy paths are host-independent. pub fn normalize_path(path: &str) -> String { use std::path::Component; @@ -152,7 +165,15 @@ pub fn normalize_path(path: &str) -> String { Component::Normal(c) => normalized.push(c), } } - normalized.to_string_lossy().to_string() + let normalized = normalized.to_string_lossy(); + #[cfg(windows)] + { + normalized.replace('\\', "/") + } + #[cfg(not(windows))] + { + normalized.into_owned() + } } #[cfg(test)] diff --git a/crates/openshell-driver-docker/Cargo.toml b/crates/openshell-driver-docker/Cargo.toml index 7e1bc069cb..9ee0a3a388 100644 --- a/crates/openshell-driver-docker/Cargo.toml +++ b/crates/openshell-driver-docker/Cargo.toml @@ -10,16 +10,20 @@ rust-version.workspace = true license.workspace = true repository.workspace = true +[lib] +path = "src/lib_entry.rs" + [dependencies] openshell-core = { path = "../openshell-core", default-features = false } +serde = { workspace = true } +[target.'cfg(not(target_os = "windows"))'.dependencies] tokio = { workspace = true } tonic = { workspace = true } futures = { workspace = true } tokio-stream = { workspace = true } tracing = { workspace = true } bytes = { workspace = true } -serde = { workspace = true } serde_json = { workspace = true } prost-types = { workspace = true } bollard = { version = "0.20" } diff --git a/crates/openshell-driver-docker/src/lib_entry.rs b/crates/openshell-driver-docker/src/lib_entry.rs new file mode 100644 index 0000000000..4272e7c537 --- /dev/null +++ b/crates/openshell-driver-docker/src/lib_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("lib.rs"); + +#[cfg(target_os = "windows")] +include!("lib_win.rs"); diff --git a/crates/openshell-driver-docker/src/lib_win.rs b/crates/openshell-driver-docker/src/lib_win.rs new file mode 100644 index 0000000000..343ce7c585 --- /dev/null +++ b/crates/openshell-driver-docker/src/lib_win.rs @@ -0,0 +1,6 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +mod windows; + +pub use windows::{DockerComputeConfig, default_docker_supervisor_image}; diff --git a/crates/openshell-driver-docker/src/windows.rs b/crates/openshell-driver-docker/src/windows.rs new file mode 100644 index 0000000000..87313dc9c2 --- /dev/null +++ b/crates/openshell-driver-docker/src/windows.rs @@ -0,0 +1,77 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +use openshell_core::config::DEFAULT_DOCKER_NETWORK_NAME; +use std::path::PathBuf; + +const DEFAULT_DOCKER_SUPERVISOR_IMAGE_REPO: &str = "ghcr.io/nvidia/openshell/supervisor"; +const DEFAULT_SANDBOX_PIDS_LIMIT: i64 = 2048; + +#[must_use] +pub fn default_docker_supervisor_image() -> String { + format!( + "{DEFAULT_DOCKER_SUPERVISOR_IMAGE_REPO}:{}", + default_docker_supervisor_image_tag() + ) +} + +fn default_docker_supervisor_image_tag() -> String { + let tag = option_env!("OPENSHELL_IMAGE_TAG") + .filter(|tag| !tag.is_empty()) + .or_else(|| option_env!("IMAGE_TAG").filter(|tag| !tag.is_empty())) + .unwrap_or_else(|| { + if env!("CARGO_PKG_VERSION").is_empty() || env!("CARGO_PKG_VERSION") == "0.0.0" { + "dev" + } else { + env!("CARGO_PKG_VERSION") + } + }); + + tag.replace('+', "-") +} + +/// Gateway-local configuration for the Docker compute driver. +/// +/// Windows builds keep this type so existing config files continue to parse, +/// but Docker runtime support is intentionally unavailable on Windows. +#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct DockerComputeConfig { + pub socket_path: Option, + pub default_image: String, + pub image_pull_policy: String, + pub sandbox_namespace: String, + pub grpc_endpoint: String, + pub supervisor_bin: Option, + pub supervisor_image: Option, + pub guest_tls_ca: Option, + pub guest_tls_cert: Option, + pub guest_tls_key: Option, + pub network_name: String, + pub host_gateway_ip: String, + pub ssh_socket_path: String, + pub sandbox_pids_limit: i64, + pub enable_bind_mounts: bool, +} + +impl Default for DockerComputeConfig { + fn default() -> Self { + Self { + socket_path: None, + default_image: openshell_core::image::default_sandbox_image(), + image_pull_policy: String::new(), + sandbox_namespace: "default".to_string(), + grpc_endpoint: String::new(), + supervisor_bin: None, + supervisor_image: None, + guest_tls_ca: None, + guest_tls_cert: None, + guest_tls_key: None, + network_name: DEFAULT_DOCKER_NETWORK_NAME.to_string(), + host_gateway_ip: String::new(), + ssh_socket_path: "/run/openshell/ssh.sock".to_string(), + sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT, + enable_bind_mounts: false, + } + } +} diff --git a/crates/openshell-driver-kubernetes/Cargo.toml b/crates/openshell-driver-kubernetes/Cargo.toml index 2c02f864ab..541f526198 100644 --- a/crates/openshell-driver-kubernetes/Cargo.toml +++ b/crates/openshell-driver-kubernetes/Cargo.toml @@ -3,6 +3,7 @@ [package] name = "openshell-driver-kubernetes" +autobins = false description = "Kubernetes compute driver for OpenShell" version.workspace = true edition.workspace = true @@ -12,12 +13,14 @@ repository.workspace = true [[bin]] name = "openshell-driver-kubernetes" -path = "src/main.rs" +path = "src/main_entry.rs" [dependencies] openshell-core = { path = "../openshell-core", default-features = false } openshell-policy = { path = "../openshell-policy" } +serde = { workspace = true } +[target.'cfg(not(target_os = "windows"))'.dependencies] tokio = { workspace = true } tonic = { workspace = true, features = ["transport"] } prost = { workspace = true } @@ -27,7 +30,6 @@ tokio-stream = { workspace = true } kube = { workspace = true } kube-runtime = { workspace = true } k8s-openapi = { workspace = true } -serde = { workspace = true } serde_json = { workspace = true } clap = { workspace = true } tracing = { workspace = true } @@ -37,6 +39,7 @@ miette = { workspace = true } [dev-dependencies] temp-env = "0.3" +serde_json = { workspace = true } [lints] workspace = true diff --git a/crates/openshell-driver-kubernetes/src/lib.rs b/crates/openshell-driver-kubernetes/src/lib.rs index 7c56c8de5b..0bb796a37b 100644 --- a/crates/openshell-driver-kubernetes/src/lib.rs +++ b/crates/openshell-driver-kubernetes/src/lib.rs @@ -2,7 +2,10 @@ // SPDX-License-Identifier: Apache-2.0 pub mod config; + +#[cfg(not(target_os = "windows"))] pub mod driver; +#[cfg(not(target_os = "windows"))] pub mod grpc; pub use config::{ @@ -10,5 +13,8 @@ pub use config::{ DEFAULT_WORKSPACE_STORAGE_SIZE, KubernetesComputeConfig, KubernetesSidecarConfig, SupervisorSideloadMethod, SupervisorTopology, }; + +#[cfg(not(target_os = "windows"))] pub use driver::{KubernetesComputeDriver, KubernetesDriverError}; +#[cfg(not(target_os = "windows"))] pub use grpc::ComputeDriverService; diff --git a/crates/openshell-driver-kubernetes/src/main_entry.rs b/crates/openshell-driver-kubernetes/src/main_entry.rs new file mode 100644 index 0000000000..432402121a --- /dev/null +++ b/crates/openshell-driver-kubernetes/src/main_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("main.rs"); + +#[cfg(target_os = "windows")] +include!("main_win.rs"); diff --git a/crates/openshell-driver-kubernetes/src/main_win.rs b/crates/openshell-driver-kubernetes/src/main_win.rs new file mode 100644 index 0000000000..56cc95ac04 --- /dev/null +++ b/crates/openshell-driver-kubernetes/src/main_win.rs @@ -0,0 +1,7 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +fn main() { + eprintln!("openshell-driver-kubernetes is unsupported on Windows"); + std::process::exit(1); +} diff --git a/crates/openshell-driver-podman/Cargo.toml b/crates/openshell-driver-podman/Cargo.toml index ed798c0ab2..9d490d4e9b 100644 --- a/crates/openshell-driver-podman/Cargo.toml +++ b/crates/openshell-driver-podman/Cargo.toml @@ -3,6 +3,7 @@ [package] name = "openshell-driver-podman" +autobins = false description = "Podman compute driver for OpenShell" version.workspace = true edition.workspace = true @@ -10,11 +11,17 @@ rust-version.workspace = true license.workspace = true repository.workspace = true +[lib] +path = "src/lib_entry.rs" + [[bin]] name = "openshell-driver-podman" -path = "src/main.rs" +path = "src/main_entry.rs" + +[target.'cfg(target_os = "windows")'.dependencies] +serde = { workspace = true } -[dependencies] +[target.'cfg(not(target_os = "windows"))'.dependencies] openshell-core = { path = "../openshell-core", default-features = false } tokio = { workspace = true } diff --git a/crates/openshell-driver-podman/src/lib_entry.rs b/crates/openshell-driver-podman/src/lib_entry.rs new file mode 100644 index 0000000000..4272e7c537 --- /dev/null +++ b/crates/openshell-driver-podman/src/lib_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("lib.rs"); + +#[cfg(target_os = "windows")] +include!("lib_win.rs"); diff --git a/crates/openshell-driver-podman/src/lib_win.rs b/crates/openshell-driver-podman/src/lib_win.rs new file mode 100644 index 0000000000..514eed1bdd --- /dev/null +++ b/crates/openshell-driver-podman/src/lib_win.rs @@ -0,0 +1,6 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +mod windows; + +pub use windows::PodmanComputeConfig; diff --git a/crates/openshell-driver-podman/src/main_entry.rs b/crates/openshell-driver-podman/src/main_entry.rs new file mode 100644 index 0000000000..432402121a --- /dev/null +++ b/crates/openshell-driver-podman/src/main_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("main.rs"); + +#[cfg(target_os = "windows")] +include!("main_win.rs"); diff --git a/crates/openshell-driver-podman/src/main_win.rs b/crates/openshell-driver-podman/src/main_win.rs new file mode 100644 index 0000000000..527dacccb2 --- /dev/null +++ b/crates/openshell-driver-podman/src/main_win.rs @@ -0,0 +1,7 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +fn main() { + eprintln!("openshell-driver-podman is unsupported on Windows"); + std::process::exit(1); +} diff --git a/crates/openshell-driver-podman/src/windows.rs b/crates/openshell-driver-podman/src/windows.rs new file mode 100644 index 0000000000..8bfe03136a --- /dev/null +++ b/crates/openshell-driver-podman/src/windows.rs @@ -0,0 +1,66 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +use std::path::PathBuf; + +const DEFAULT_SANDBOX_IMAGE: &str = "ghcr.io/nvidia/openshell/sandbox:latest"; +const DEFAULT_SUPERVISOR_IMAGE: &str = "ghcr.io/nvidia/openshell/supervisor:latest"; +const DEFAULT_SERVER_PORT: u16 = 8080; +const DEFAULT_STOP_TIMEOUT_SECS: u32 = 10; +const DEFAULT_NETWORK_NAME: &str = "openshell"; +const DEFAULT_SANDBOX_PIDS_LIMIT: i64 = 2048; +const DEFAULT_HEALTH_CHECK_INTERVAL_SECS: u64 = 10; + +#[derive(Debug, Clone, Default, serde::Serialize, serde::Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum ImagePullPolicy { + Always, + #[default] + Missing, + Never, + Newer, +} + +#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)] +#[serde(default, deny_unknown_fields)] +pub struct PodmanComputeConfig { + pub socket_path: Option, + pub default_image: String, + pub image_pull_policy: ImagePullPolicy, + pub grpc_endpoint: String, + pub gateway_port: u16, + pub host_gateway_ip: String, + pub sandbox_ssh_socket_path: String, + pub network_name: String, + pub stop_timeout_secs: u32, + pub supervisor_image: String, + pub guest_tls_ca: Option, + pub guest_tls_cert: Option, + pub guest_tls_key: Option, + pub sandbox_pids_limit: i64, + pub enable_bind_mounts: bool, + pub health_check_interval_secs: u64, +} + +impl Default for PodmanComputeConfig { + fn default() -> Self { + Self { + socket_path: None, + default_image: DEFAULT_SANDBOX_IMAGE.to_string(), + image_pull_policy: ImagePullPolicy::default(), + grpc_endpoint: String::new(), + gateway_port: DEFAULT_SERVER_PORT, + host_gateway_ip: String::new(), + sandbox_ssh_socket_path: String::new(), + network_name: DEFAULT_NETWORK_NAME.to_string(), + stop_timeout_secs: DEFAULT_STOP_TIMEOUT_SECS, + supervisor_image: DEFAULT_SUPERVISOR_IMAGE.to_string(), + guest_tls_ca: None, + guest_tls_cert: None, + guest_tls_key: None, + sandbox_pids_limit: DEFAULT_SANDBOX_PIDS_LIMIT, + enable_bind_mounts: false, + health_check_interval_secs: DEFAULT_HEALTH_CHECK_INTERVAL_SECS, + } + } +} diff --git a/crates/openshell-driver-vm/Cargo.toml b/crates/openshell-driver-vm/Cargo.toml index cef3e67f88..ba1b118340 100644 --- a/crates/openshell-driver-vm/Cargo.toml +++ b/crates/openshell-driver-vm/Cargo.toml @@ -3,6 +3,7 @@ [package] name = "openshell-driver-vm" +autobins = false description = "MicroVM compute driver for OpenShell" version.workspace = true edition.workspace = true @@ -12,13 +13,13 @@ repository.workspace = true [lib] name = "openshell_driver_vm" -path = "src/lib.rs" +path = "src/lib_entry.rs" [[bin]] name = "openshell-driver-vm" -path = "src/main.rs" +path = "src/main_entry.rs" -[dependencies] +[target.'cfg(not(target_os = "windows"))'.dependencies] openshell-core = { path = "../openshell-core", default-features = false } openshell-policy = { path = "../openshell-policy" } openshell-vfio = { path = "../openshell-vfio" } diff --git a/crates/openshell-driver-vm/src/lib_entry.rs b/crates/openshell-driver-vm/src/lib_entry.rs new file mode 100644 index 0000000000..4272e7c537 --- /dev/null +++ b/crates/openshell-driver-vm/src/lib_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("lib.rs"); + +#[cfg(target_os = "windows")] +include!("lib_win.rs"); diff --git a/crates/openshell-driver-vm/src/lib_win.rs b/crates/openshell-driver-vm/src/lib_win.rs new file mode 100644 index 0000000000..55f725ceec --- /dev/null +++ b/crates/openshell-driver-vm/src/lib_win.rs @@ -0,0 +1,6 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +pub fn windows_unsupported() -> &'static str { + "openshell-driver-vm is unsupported on Windows" +} diff --git a/crates/openshell-driver-vm/src/main_entry.rs b/crates/openshell-driver-vm/src/main_entry.rs new file mode 100644 index 0000000000..432402121a --- /dev/null +++ b/crates/openshell-driver-vm/src/main_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("main.rs"); + +#[cfg(target_os = "windows")] +include!("main_win.rs"); diff --git a/crates/openshell-driver-vm/src/main_win.rs b/crates/openshell-driver-vm/src/main_win.rs new file mode 100644 index 0000000000..8463707caf --- /dev/null +++ b/crates/openshell-driver-vm/src/main_win.rs @@ -0,0 +1,7 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +fn main() { + eprintln!("openshell-driver-vm is unsupported on Windows"); + std::process::exit(1); +} diff --git a/crates/openshell-gateway-interceptors/src/plan.rs b/crates/openshell-gateway-interceptors/src/plan.rs index b65d5612fd..36e7aa3115 100644 --- a/crates/openshell-gateway-interceptors/src/plan.rs +++ b/crates/openshell-gateway-interceptors/src/plan.rs @@ -7,6 +7,7 @@ use std::collections::{BTreeMap, BTreeSet, HashMap}; use std::path::PathBuf; use std::time::Duration; +#[cfg(unix)] use hyper_util::rt::TokioIo; use openshell_core::config::{ GatewayInterceptorBindingOverride, GatewayInterceptorBindingPolicy, GatewayInterceptorConfig, @@ -16,10 +17,13 @@ use openshell_core::proto::gateway_interceptor::v1::{ DescribeRequest, GatewayInterceptorPhase, InterceptorBinding, InterceptorSelector, gateway_interceptor_client::GatewayInterceptorClient, }; +#[cfg(unix)] use tokio::net::UnixStream; use tonic::Request; +#[cfg(unix)] use tonic::codegen::http::Uri; use tonic::transport::{Channel, Endpoint}; +#[cfg(unix)] use tower::service_fn; use tracing::{info, warn}; diff --git a/crates/openshell-prover/src/lib.rs b/crates/openshell-prover/src/lib.rs index 0fb8757577..2267052048 100644 --- a/crates/openshell-prover/src/lib.rs +++ b/crates/openshell-prover/src/lib.rs @@ -26,11 +26,11 @@ use policy::parse_policy; use queries::run_all_queries; use report::{render_compact, render_report}; -/// Run the prover end-to-end and return a result containing an exit code. +/// Run the prover end-to-end and return an exit code. /// -/// - `Ok(0)` — pass (no findings, or all accepted) -/// - `Ok(1)` — fail (one or more unaccepted findings present) -/// - `Err(_)` — input or registry loading error +/// - `0` — pass (no critical/high findings, or all accepted) +/// - `1` — fail (critical or high findings present) +/// - `2` — input error /// /// Binary and API capability registries are embedded at compile time. /// Pass `registry_dir` to override with a custom filesystem registry. diff --git a/crates/openshell-sandbox/Cargo.toml b/crates/openshell-sandbox/Cargo.toml index 6a51635b14..dc716d0e58 100644 --- a/crates/openshell-sandbox/Cargo.toml +++ b/crates/openshell-sandbox/Cargo.toml @@ -3,6 +3,7 @@ [package] name = "openshell-sandbox" +autobins = false description = "OpenShell process sandbox and monitor" version.workspace = true edition.workspace = true @@ -10,11 +11,14 @@ rust-version.workspace = true license.workspace = true repository.workspace = true +[lib] +path = "src/lib_entry.rs" + [[bin]] name = "openshell-sandbox" -path = "src/main.rs" +path = "src/main_entry.rs" -[dependencies] +[target.'cfg(not(target_os = "windows"))'.dependencies] openshell-core = { path = "../openshell-core", default-features = false } openshell-ocsf = { path = "../openshell-ocsf" } openshell-policy = { path = "../openshell-policy" } @@ -35,6 +39,13 @@ clap = { workspace = true } # Error handling miette = { workspace = true } +thiserror = { workspace = true } +anyhow = { workspace = true } +hmac = "0.12" +sha2 = { workspace = true } +hex = "0.4" +russh = { version = "0.57", default-features = false, features = ["flate2", "ring", "rsa"] } +rand_core = "0.6" # Unix ownership for Kubernetes sidecar init setup nix = { workspace = true } diff --git a/crates/openshell-sandbox/src/lib_entry.rs b/crates/openshell-sandbox/src/lib_entry.rs new file mode 100644 index 0000000000..4272e7c537 --- /dev/null +++ b/crates/openshell-sandbox/src/lib_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("lib.rs"); + +#[cfg(target_os = "windows")] +include!("lib_win.rs"); diff --git a/crates/openshell-sandbox/src/lib_win.rs b/crates/openshell-sandbox/src/lib_win.rs new file mode 100644 index 0000000000..394226e1a7 --- /dev/null +++ b/crates/openshell-sandbox/src/lib_win.rs @@ -0,0 +1,6 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +pub fn windows_unsupported() -> &'static str { + "openshell-sandbox supervisor is not available on Windows" +} diff --git a/crates/openshell-sandbox/src/main_entry.rs b/crates/openshell-sandbox/src/main_entry.rs new file mode 100644 index 0000000000..432402121a --- /dev/null +++ b/crates/openshell-sandbox/src/main_entry.rs @@ -0,0 +1,8 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +#[cfg(not(target_os = "windows"))] +include!("main.rs"); + +#[cfg(target_os = "windows")] +include!("main_win.rs"); diff --git a/crates/openshell-sandbox/src/main_win.rs b/crates/openshell-sandbox/src/main_win.rs new file mode 100644 index 0000000000..460b379c4c --- /dev/null +++ b/crates/openshell-sandbox/src/main_win.rs @@ -0,0 +1,7 @@ +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +fn main() { + eprintln!("openshell-sandbox supervisor is not available on Windows"); + std::process::exit(1); +} diff --git a/crates/openshell-sandbox/tests/stdout_logging.rs b/crates/openshell-sandbox/tests/stdout_logging.rs index c4f5213de8..6feb91f869 100644 --- a/crates/openshell-sandbox/tests/stdout_logging.rs +++ b/crates/openshell-sandbox/tests/stdout_logging.rs @@ -1,6 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +#![cfg(not(target_os = "windows"))] + use std::process::Command; #[test] diff --git a/crates/openshell-server/Cargo.toml b/crates/openshell-server/Cargo.toml index 4c4f289ed6..7f660a06df 100644 --- a/crates/openshell-server/Cargo.toml +++ b/crates/openshell-server/Cargo.toml @@ -93,7 +93,7 @@ async-trait = "0.1" url = { workspace = true } glob = { workspace = true } hex = "0.4" -russh = "0.57" +russh = { version = "0.57", default-features = false, features = ["flate2", "ring", "rsa"] } rand = { workspace = true } petname = "2" ipnet = "2" diff --git a/crates/openshell-server/src/cli.rs b/crates/openshell-server/src/cli.rs index 269b9f40e9..e9caac5de4 100644 --- a/crates/openshell-server/src/cli.rs +++ b/crates/openshell-server/src/cli.rs @@ -1171,11 +1171,15 @@ mod tests { let expected = format!( "sqlite:{}", - tmp.path().join("openshell/gateway/openshell.db").display() + tmp.path() + .join("openshell") + .join("gateway") + .join("openshell.db") + .display() ); assert!(local_tls.is_none()); assert_eq!(args.db_url.as_deref(), Some(expected.as_str())); - assert!(tmp.path().join("openshell/gateway").is_dir()); + assert!(tmp.path().join("openshell").join("gateway").is_dir()); } #[test] diff --git a/crates/openshell-server/src/compute/mod.rs b/crates/openshell-server/src/compute/mod.rs index b87f1b5fa6..4597ad8d26 100644 --- a/crates/openshell-server/src/compute/mod.rs +++ b/crates/openshell-server/src/compute/mod.rs @@ -22,6 +22,7 @@ use crate::sandbox_watch::SandboxWatchBus; use crate::supervisor_session::SupervisorSessionRegistry; use crate::tracing_bus::TracingLogBus; use futures::{Stream, StreamExt}; +#[cfg(unix)] use hyper_util::rt::TokioIo; use openshell_core::ComputeDriverKind; use openshell_core::proto::compute::v1::{ @@ -38,10 +39,13 @@ use openshell_core::proto::{ SandboxTemplate, ServiceEndpoint, SshSession, }; use openshell_core::{ObjectLabels, ObjectWorkspace}; +#[cfg(not(target_os = "windows"))] use openshell_driver_docker::DockerComputeDriver; +#[cfg(not(target_os = "windows"))] use openshell_driver_kubernetes::{ ComputeDriverService as KubernetesDriverService, KubernetesComputeDriver, }; +#[cfg(not(target_os = "windows"))] use openshell_driver_podman::{ComputeDriverService as PodmanDriverService, PodmanComputeDriver}; use prost::Message; use std::collections::HashMap; @@ -54,8 +58,11 @@ use std::time::Duration; #[cfg(unix)] use tokio::net::UnixStream; use tokio::sync::{Mutex, watch}; -use tonic::transport::{Channel, Endpoint}; +use tonic::transport::Channel; +#[cfg(unix)] +use tonic::transport::Endpoint; use tonic::{Code, Request, Status}; +#[cfg(unix)] use tower::service_fn; use tracing::{debug, info, warn}; @@ -165,6 +172,7 @@ trait ShutdownCleanup: Send + Sync { } #[tonic::async_trait] +#[cfg(not(target_os = "windows"))] impl ShutdownCleanup for DockerComputeDriver { async fn cleanup_on_shutdown(&self) -> Result<(), String> { let stopped = self @@ -190,6 +198,7 @@ trait StartupResume: Send + Sync { } #[tonic::async_trait] +#[cfg(not(target_os = "windows"))] impl StartupResume for DockerComputeDriver { async fn resume_sandbox(&self, sandbox_id: &str, sandbox_name: &str) -> Result { Self::resume_sandbox(self, sandbox_id, sandbox_name) @@ -207,6 +216,11 @@ const ORPHAN_GRACE_PERIOD: Duration = Duration::from_secs(300); // Re-export the shared error type under the name used by this module. pub use openshell_core::ComputeDriverError as ComputeError; +#[cfg(target_os = "windows")] +fn unsupported_compute_driver_error(driver: &str) -> ComputeError { + ComputeError::Message(format!("{driver} compute driver is unsupported on Windows")) +} + #[derive(Debug)] pub struct ManagedDriverProcess { child: std::sync::Mutex>, @@ -214,6 +228,7 @@ pub struct ManagedDriverProcess { } impl ManagedDriverProcess { + #[cfg(unix)] pub(crate) fn new(child: tokio::process::Child, socket_path: PathBuf) -> Self { Self { child: std::sync::Mutex::new(Some(child)), @@ -457,6 +472,7 @@ impl ComputeRuntime { self.delete_gates.entry_count() } + #[cfg(not(target_os = "windows"))] pub async fn new_docker( config: openshell_core::Config, docker_config: DockerComputeConfig, @@ -491,6 +507,20 @@ impl ComputeRuntime { .await } + #[cfg(target_os = "windows")] + pub async fn new_docker( + _config: openshell_core::Config, + _docker_config: DockerComputeConfig, + _store: Arc, + _sandbox_index: SandboxIndex, + _sandbox_watch_bus: SandboxWatchBus, + _tracing_log_bus: TracingLogBus, + _supervisor_sessions: Arc, + ) -> Result { + Err(unsupported_compute_driver_error("Docker")) + } + + #[cfg(not(target_os = "windows"))] pub async fn new_kubernetes( config: KubernetesComputeConfig, store: Arc, @@ -519,6 +549,18 @@ impl ComputeRuntime { .await } + #[cfg(target_os = "windows")] + pub async fn new_kubernetes( + _config: KubernetesComputeConfig, + _store: Arc, + _sandbox_index: SandboxIndex, + _sandbox_watch_bus: SandboxWatchBus, + _tracing_log_bus: TracingLogBus, + _supervisor_sessions: Arc, + ) -> Result { + Err(unsupported_compute_driver_error("Kubernetes")) + } + pub(crate) async fn new_remote_driver( endpoint: AcquiredRemoteDriverEndpoint, store: Arc, @@ -544,6 +586,7 @@ impl ComputeRuntime { .await } + #[cfg(not(target_os = "windows"))] pub async fn new_podman( config: PodmanComputeConfig, store: Arc, @@ -572,6 +615,18 @@ impl ComputeRuntime { .await } + #[cfg(target_os = "windows")] + pub async fn new_podman( + _config: PodmanComputeConfig, + _store: Arc, + _sandbox_index: SandboxIndex, + _sandbox_watch_bus: SandboxWatchBus, + _tracing_log_bus: TracingLogBus, + _supervisor_sessions: Arc, + ) -> Result { + Err(unsupported_compute_driver_error("Podman")) + } + #[must_use] pub fn default_image(&self) -> &str { &self.default_image @@ -6374,4 +6429,16 @@ mod tests { .unwrap(); assert_eq!(stored.object_workspace(), "alpha"); } + + #[cfg(target_os = "windows")] + #[test] + fn windows_compute_driver_stubs_report_unsupported() { + for driver in ["Docker", "Kubernetes", "Podman"] { + let message = unsupported_compute_driver_error(driver).to_string(); + assert!( + message.contains("unsupported on Windows"), + "{driver} stub message should be explicit, got: {message}" + ); + } + } } diff --git a/crates/openshell-server/src/compute/vm.rs b/crates/openshell-server/src/compute/vm.rs index be88047f33..3afb8003fd 100644 --- a/crates/openshell-server/src/compute/vm.rs +++ b/crates/openshell-server/src/compute/vm.rs @@ -35,10 +35,12 @@ use super::ManagedDriverProcess; #[cfg(unix)] use hyper_util::rt::TokioIo; #[cfg(unix)] +use openshell_core::ComputeDriverKind; +#[cfg(unix)] use openshell_core::proto::compute::v1::{ GetCapabilitiesRequest, compute_driver_client::ComputeDriverClient, }; -use openshell_core::{ComputeDriverKind, Config, Error, Result}; +use openshell_core::{Config, Error, Result}; #[cfg(unix)] use std::os::unix::fs::{FileTypeExt, MetadataExt, PermissionsExt}; #[cfg(unix)] @@ -50,14 +52,18 @@ use std::{io::ErrorKind, process::Stdio, sync::Arc, time::Duration}; use tokio::net::UnixStream; #[cfg(unix)] use tokio::process::Command; +#[cfg(unix)] use tonic::transport::Channel; #[cfg(unix)] use tonic::transport::Endpoint; #[cfg(unix)] use tower::service_fn; +#[cfg(unix)] const DRIVER_BIN_NAME: &str = "openshell-driver-vm"; +#[cfg(unix)] const COMPUTE_DRIVER_SOCKET_RUN_DIR: &str = "run"; +#[cfg(unix)] const COMPUTE_DRIVER_SOCKET_NAME: &str = "compute-driver.sock"; /// Configuration for launching and talking to the VM compute driver. @@ -136,6 +142,7 @@ impl VmComputeConfig { 4096 } + #[cfg(unix)] #[must_use] fn default_driver_search_dirs(home: Option) -> Vec { let mut dirs = Vec::new(); @@ -186,6 +193,7 @@ pub struct VmGuestTlsPaths { /// `/usr/local/libexec/openshell`, `/usr/local/libexec`. /// 3. Sibling of the gateway's own executable (last-resort fallback so /// local development builds still work out of the box). +#[cfg(unix)] pub fn resolve_compute_driver_bin(vm_config: &VmComputeConfig) -> Result { let mut searched: Vec = Vec::new(); @@ -224,6 +232,7 @@ pub fn resolve_compute_driver_bin(vm_config: &VmComputeConfig) -> Result Vec { vm_config.driver_dir.clone().map_or_else( || { @@ -245,6 +254,7 @@ fn resolve_driver_search_dirs(vm_config: &VmComputeConfig) -> Vec { ) } +#[cfg(unix)] fn push_unique_path(paths: &mut Vec, path: PathBuf) { if !paths.iter().any(|existing| existing == &path) { paths.push(path); @@ -252,6 +262,7 @@ fn push_unique_path(paths: &mut Vec, path: PathBuf) { } /// Path of the Unix domain socket the driver will listen on. +#[cfg(unix)] pub fn compute_driver_socket_path(vm_config: &VmComputeConfig) -> PathBuf { vm_config .state_dir @@ -515,7 +526,20 @@ pub async fn spawn( )) } -#[cfg(not(unix))] +#[cfg(target_os = "windows")] +fn unsupported_vm_message() -> &'static str { + "VM compute driver is unsupported on Windows" +} + +#[cfg(target_os = "windows")] +pub async fn spawn( + _config: &Config, + _vm_config: &VmComputeConfig, +) -> Result { + Err(Error::config(unsupported_vm_message())) +} + +#[cfg(all(not(unix), not(target_os = "windows")))] pub async fn spawn( _config: &Config, _vm_config: &VmComputeConfig, @@ -866,3 +890,11 @@ mod tests { assert!(!socket_path.exists()); } } + +#[cfg(all(test, target_os = "windows"))] +mod windows_tests { + #[test] + fn windows_spawn_reports_unsupported() { + assert!(super::unsupported_vm_message().contains("unsupported on Windows")); + } +} diff --git a/crates/openshell-server/src/config_file.rs b/crates/openshell-server/src/config_file.rs index c4e0cbc959..8d3b346141 100644 --- a/crates/openshell-server/src/config_file.rs +++ b/crates/openshell-server/src/config_file.rs @@ -239,6 +239,7 @@ pub enum ConfigFileError { /// /// Returns `Ok(ConfigFile::default())` for an empty file (the gateway then /// falls back entirely to CLI/env/built-in defaults). +#[cfg_attr(target_os = "windows", allow(clippy::result_large_err))] pub fn load(path: &Path) -> Result { let contents = std::fs::read_to_string(path).map_err(|source| ConfigFileError::Io { path: path.to_path_buf(), diff --git a/crates/openshell-server/src/grpc/policy.rs b/crates/openshell-server/src/grpc/policy.rs index e99ea1a30d..616a05b266 100644 --- a/crates/openshell-server/src/grpc/policy.rs +++ b/crates/openshell-server/src/grpc/policy.rs @@ -53,10 +53,12 @@ use openshell_core::{ use openshell_ocsf::{ ConfigStateChangeBuilder, OCSF_TARGET, OcsfEvent, SandboxContext, SeverityId, StateId, StatusId, }; +#[cfg(not(target_os = "windows"))] +use openshell_policy::serialize_sandbox_policy; use openshell_policy::{ PolicyMergeOp, ProviderPolicyLayer, compose_effective_policy, merge_policy, - serialize_sandbox_policy, }; +#[cfg(not(target_os = "windows"))] use openshell_prover::{ credentials::{Credential, CredentialSet}, finding::{Finding, FindingPath}, @@ -69,7 +71,9 @@ use openshell_prover::{ use openshell_providers::normalize_provider_type; use prost::Message; use sha2::{Digest, Sha256}; -use std::collections::{BTreeMap, HashMap, HashSet}; +#[cfg(not(target_os = "windows"))] +use std::collections::HashSet; +use std::collections::{BTreeMap, HashMap}; use std::net::{IpAddr, Ipv4Addr}; use std::sync::Arc; use tonic::{Request, Response, Status}; @@ -441,6 +445,7 @@ fn summarize_draft_chunk_rule(chunk: &DraftChunkRecord) -> Result String { + let merge_op = PolicyMergeOp::AddRule { + rule_name: rule_name.to_string(), + rule: proposed_rule.clone(), + }; + let merged = match merge_policy(current_policy, &[merge_op]) { + Ok(result) => result.policy, + Err(error) => return format!("merge failed: {}", one_line(&error.to_string())), + }; + if let Err(error) = validate_policy_safety(&merged) { + return format!("policy invalid: {}", one_line(&error.to_string())); + } + "validation unavailable".to_string() +} + /// Run the prover end-to-end against a single policy with the given /// credential set. Returns the raw finding list, or a short error string /// identifying which infrastructure step failed. /// /// The credential set is passed in because it's stable across all chunks in /// one `SubmitPolicyAnalysis` batch — the caller builds it once and shares. +#[cfg(not(target_os = "windows"))] fn run_prover_findings( policy: &ProtoSandboxPolicy, credentials: &CredentialSet, @@ -523,6 +549,7 @@ fn run_prover_findings( /// a `warn!` — the merged policy already excludes them at compose time, so /// silently treating them as absent here keeps the credential set consistent /// with the merged policy the prover validates against. +#[cfg(not(target_os = "windows"))] async fn build_credential_set_for_sandbox_with_catalog( store: &Store, catalog: &EffectiveProviderProfileCatalog, @@ -587,6 +614,7 @@ async fn build_credential_set_for_sandbox_with_catalog( /// the same security gap whether they live in rule `foo` or rule `bar`. This /// keeps the delta from spuriously surfacing baseline gaps just because the /// proposal added a new rule name that produces the same gap shape. +#[cfg(not(target_os = "windows"))] fn finding_path_key(path: &FindingPath) -> String { let FindingPath::Exfil(p) = path; // Include the category and (for capability_expansion) the method so @@ -608,6 +636,7 @@ fn finding_path_key(path: &FindingPath) -> String { /// are suppressed. A brand-new credentialed reach is described by the /// reach-expansion finding alone; we don't double-report by also /// flagging every method as a separate `capability_expansion`. +#[cfg(not(target_os = "windows"))] fn finding_delta(base: &[Finding], merged: &[Finding]) -> Vec { use openshell_prover::finding::category; @@ -913,6 +942,7 @@ struct AutoApproveChunkContext<'a> { source: &'a str, resolved_from: &'a str, current_policy: &'a ProtoSandboxPolicy, + #[cfg(not(target_os = "windows"))] credential_set: &'a CredentialSet, } @@ -951,12 +981,19 @@ async fn auto_approve_chunk( // stored rule instead of trusting the incoming proposal's verdict. let rule = decode_draft_chunk_rule(&chunk)? .ok_or_else(|| Status::failed_precondition("draft chunk has no proposed rule"))?; + #[cfg(not(target_os = "windows"))] let validation_result = validation_result_for_agent_proposal( context.current_policy.clone(), &chunk.rule_name, &rule, context.credential_set, ); + #[cfg(target_os = "windows")] + let validation_result = validation_result_for_agent_proposal( + context.current_policy.clone(), + &chunk.rule_name, + &rule, + ); if validation_result != "prover: no new findings" { info!( sandbox_id = %sandbox_id, @@ -1098,7 +1135,7 @@ fn truncate_for_log(input: &str, max_chars: usize) -> String { } } -#[cfg(test)] +#[cfg(all(test, not(target_os = "windows")))] fn is_sandbox_caller(request: &Request) -> bool { matches!( request.extensions().get::(), @@ -2667,18 +2704,21 @@ pub(super) async fn handle_submit_policy_analysis( // it once. v1 captures presence only — no scope modeling — so the prover // can answer "is there a credential in scope for this host?" but not // "what action class does that credential authorize?" - let provider_names_for_creds: Vec = sandbox - .spec - .as_ref() - .map(|spec| spec.providers.clone()) - .unwrap_or_default(); - let credential_set = build_credential_set_for_sandbox_with_catalog( - state.store.as_ref(), - &provider_profile_catalog, - &workspace, - &provider_names_for_creds, - ) - .await?; + #[cfg(not(target_os = "windows"))] + let credential_set = { + let provider_names_for_creds: Vec = sandbox + .spec + .as_ref() + .map(|spec| spec.providers.clone()) + .unwrap_or_default(); + build_credential_set_for_sandbox_with_catalog( + state.store.as_ref(), + &provider_profile_catalog, + &workspace, + &provider_names_for_creds, + ) + .await? + }; let current_version = state .store @@ -2742,12 +2782,19 @@ pub(super) async fn handle_submit_policy_analysis( // Source provenance (mechanistic vs agent_authored) is preserved in // OCSF audit fields, but the safety decision is grounded in the // merged-policy consequence, not the author — proposer-agnostic. + #[cfg(not(target_os = "windows"))] let validation_result = validation_result_for_agent_proposal( current_policy.clone(), &chunk.rule_name, rule_ref, &credential_set, ); + #[cfg(target_os = "windows")] + let validation_result = validation_result_for_agent_proposal( + current_policy.clone(), + &chunk.rule_name, + chunk.proposed_rule.as_ref().expect("checked above"), + ); let record = DraftChunkRecord { // The handler proposes an id; the store may swap it for an @@ -2849,6 +2896,7 @@ pub(super) async fn handle_submit_policy_analysis( source: &req.analysis_mode, resolved_from, current_policy: ¤t_policy, + #[cfg(not(target_os = "windows"))] credential_set: &credential_set, }, ) @@ -4514,7 +4562,7 @@ fn materialize_global_settings( // Tests // --------------------------------------------------------------------------- -#[cfg(test)] +#[cfg(all(test, not(target_os = "windows")))] mod tests { use super::*; use crate::auth::identity::{Identity, IdentityProvider}; diff --git a/crates/openshell-supervisor-network/src/identity.rs b/crates/openshell-supervisor-network/src/identity.rs index 5e89c35031..4f90888378 100644 --- a/crates/openshell-supervisor-network/src/identity.rs +++ b/crates/openshell-supervisor-network/src/identity.rs @@ -21,10 +21,8 @@ use tracing::debug; #[derive(Clone)] struct FileFingerprint { len: u64, - mtime_sec: i64, - mtime_nsec: i64, - ctime_sec: i64, - ctime_nsec: i64, + mtime: Option<(i64, i64)>, + ctime: Option<(i64, i64)>, #[cfg(unix)] dev: u64, #[cfg(unix)] @@ -33,12 +31,20 @@ struct FileFingerprint { impl FileFingerprint { fn from_metadata(metadata: &Metadata) -> Self { + #[cfg(unix)] + let (mtime, ctime) = ( + Some((metadata.mtime(), metadata.mtime_nsec())), + Some((metadata.ctime(), metadata.ctime_nsec())), + ); + #[cfg(not(unix))] + let (mtime, ctime) = ( + metadata.modified().ok().and_then(system_time_parts), + metadata.created().ok().and_then(system_time_parts), + ); Self { len: metadata.len(), - mtime_sec: metadata.mtime(), - mtime_nsec: metadata.mtime_nsec(), - ctime_sec: metadata.ctime(), - ctime_nsec: metadata.ctime_nsec(), + mtime, + ctime, #[cfg(unix)] dev: metadata.dev(), #[cfg(unix)] @@ -47,13 +53,22 @@ impl FileFingerprint { } } +#[cfg(not(unix))] +fn system_time_parts(time: std::time::SystemTime) -> Option<(i64, i64)> { + let duration = time.duration_since(std::time::UNIX_EPOCH).ok()?; + let seconds = i64::try_from(duration.as_secs()).ok()?; + Some((seconds, i64::from(duration.subsec_nanos()))) +} + impl PartialEq for FileFingerprint { fn eq(&self, other: &Self) -> bool { self.len == other.len - && self.mtime_sec == other.mtime_sec - && self.mtime_nsec == other.mtime_nsec - && self.ctime_sec == other.ctime_sec - && self.ctime_nsec == other.ctime_nsec + && self.mtime.is_some() + && other.mtime.is_some() + && self.mtime == other.mtime + && self.ctime.is_some() + && other.ctime.is_some() + && self.ctime == other.ctime && { #[cfg(unix)] { diff --git a/crates/openshell-supervisor-network/tests/system_inference.rs b/crates/openshell-supervisor-network/tests/system_inference.rs index 3c8e6ee8fc..189cbb6d53 100644 --- a/crates/openshell-supervisor-network/tests/system_inference.rs +++ b/crates/openshell-supervisor-network/tests/system_inference.rs @@ -1,6 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +#![cfg(not(target_os = "windows"))] + //! Integration test for the in-process system inference API. //! //! Uses the router's built-in `mock://` route support to verify the full diff --git a/crates/openshell-supervisor-network/tests/websocket_upgrade.rs b/crates/openshell-supervisor-network/tests/websocket_upgrade.rs index 322d6709cd..f9586b10e0 100644 --- a/crates/openshell-supervisor-network/tests/websocket_upgrade.rs +++ b/crates/openshell-supervisor-network/tests/websocket_upgrade.rs @@ -1,6 +1,7 @@ // SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 +#![cfg(not(target_os = "windows"))] #![allow( clippy::match_same_arms, clippy::cast_possible_truncation, diff --git a/crates/openshell-vfio/src/bind.rs b/crates/openshell-vfio/src/bind.rs index 606f56c978..8aa44ffd16 100644 --- a/crates/openshell-vfio/src/bind.rs +++ b/crates/openshell-vfio/src/bind.rs @@ -310,13 +310,13 @@ impl VfioIdRegistry { self.refcounts.clear(); } - #[cfg(test)] + #[cfg(all(test, unix))] fn remove(&mut self, id: &str) { self.refcounts.remove(id); } } -#[cfg(test)] +#[cfg(all(test, unix))] pub(crate) mod test_refcounts { use super::VFIO_ID_REGISTRY; use std::sync::{Mutex, MutexGuard, PoisonError}; @@ -337,7 +337,7 @@ pub(crate) mod test_refcounts { } } -#[cfg(test)] +#[cfg(all(test, unix))] mod tests { use super::*; use crate::test_support::{ diff --git a/crates/openshell-vfio/src/gpu.rs b/crates/openshell-vfio/src/gpu.rs index 6c421fdeeb..eb356f046b 100644 --- a/crates/openshell-vfio/src/gpu.rs +++ b/crates/openshell-vfio/src/gpu.rs @@ -162,7 +162,7 @@ pub fn prepare_gpu_for_passthrough( )) } -#[cfg(test)] +#[cfg(all(test, unix))] mod tests { use super::*; use crate::bind::test_refcounts; diff --git a/crates/openshell-vfio/src/lib.rs b/crates/openshell-vfio/src/lib.rs index 9174c78932..c773f18d3b 100644 --- a/crates/openshell-vfio/src/lib.rs +++ b/crates/openshell-vfio/src/lib.rs @@ -22,7 +22,7 @@ pub mod pci; pub mod reconcile; pub mod sysfs; -#[cfg(test)] +#[cfg(all(test, unix))] mod test_support; pub use error::VfioError; diff --git a/crates/openshell-vfio/src/pci.rs b/crates/openshell-vfio/src/pci.rs index 2db6dd0ece..83e3eba54e 100644 --- a/crates/openshell-vfio/src/pci.rs +++ b/crates/openshell-vfio/src/pci.rs @@ -523,7 +523,7 @@ pub fn release_pci_group_from_passthrough( first_err.map_or(Ok(()), Err) } -#[cfg(test)] +#[cfg(all(test, unix))] mod tests { use super::*; use crate::bind::test_refcounts; diff --git a/crates/openshell-vfio/src/reconcile.rs b/crates/openshell-vfio/src/reconcile.rs index 6ab2d8b3bb..be0b959bfa 100644 --- a/crates/openshell-vfio/src/reconcile.rs +++ b/crates/openshell-vfio/src/reconcile.rs @@ -100,7 +100,7 @@ pub fn reconcile_stale_bindings(sysfs: &SysfsRoot, state_path: &Path) -> Vec &Path { &self.base } @@ -223,7 +223,7 @@ pub(crate) fn write_sysfs(path: &Path, value: &str) -> Result<(), VfioError> { }) } -#[cfg(test)] +#[cfg(all(test, unix))] mod tests { use super::*; use crate::test_support::{create_pci_device, setup_mock_sysfs}; diff --git a/mise.lock b/mise.lock index a3864976c7..eea215b82a 100644 --- a/mise.lock +++ b/mise.lock @@ -22,6 +22,11 @@ checksum = "sha256:d5255ead3ac861a11c785bf19d4f70f16e59ac8b9519c312da61213d3d573 url = "https://github.com/EmbarkStudios/cargo-about/releases/download/0.8.4/cargo-about-0.8.4-aarch64-apple-darwin.tar.gz" url_api = "https://api.github.com/repos/EmbarkStudios/cargo-about/releases/assets/324268695" +[tools."github:EmbarkStudios/cargo-about"."platforms.windows-x64"] +checksum = "sha256:d5c38fb914bbad57c6a7d58c4847315bc3fe11efbe4fb51b3c515f997a56ceb7" +url = "https://github.com/EmbarkStudios/cargo-about/releases/download/0.8.4/cargo-about-0.8.4-x86_64-pc-windows-msvc.tar.gz" +url_api = "https://api.github.com/repos/EmbarkStudios/cargo-about/releases/assets/324269449" + [[tools."github:anchore/syft"]] version = "1.44.0" backend = "github:anchore/syft" @@ -44,6 +49,12 @@ url = "https://github.com/anchore/syft/releases/download/v1.44.0/syft_1.44.0_dar url_api = "https://api.github.com/repos/anchore/syft/releases/assets/410001187" provenance = "github-attestations" +[tools."github:anchore/syft"."platforms.windows-x64"] +checksum = "sha256:195e786eb84ec145854f20528992e86637c77d1968731dfe6ce850c90e28f47a" +url = "https://github.com/anchore/syft/releases/download/v1.44.0/syft_1.44.0_windows_amd64.zip" +url_api = "https://api.github.com/repos/anchore/syft/releases/assets/410001172" +provenance = "github-attestations" + [[tools."github:mozilla/sccache"]] version = "0.14.0" backend = "github:mozilla/sccache" @@ -63,6 +74,11 @@ checksum = "sha256:a781e8018260ab128e7690d8497736fa231b6ca895d57131d5b5b966ca987 url = "https://github.com/mozilla/sccache/releases/download/v0.14.0/sccache-v0.14.0-aarch64-apple-darwin.tar.gz" url_api = "https://api.github.com/repos/mozilla/sccache/releases/assets/353135984" +[tools."github:mozilla/sccache"."platforms.windows-x64"] +checksum = "sha256:74a3ffd4207e8e0e62af7747bd03b42deab0f6dabc7ef0a8cdd950f83f1037c8" +url = "https://github.com/mozilla/sccache/releases/download/v0.14.0/sccache-v0.14.0-x86_64-pc-windows-msvc.zip" +url_api = "https://api.github.com/repos/mozilla/sccache/releases/assets/353136140" + [[tools."github:mozilla/sccache"]] version = "0.14.0" backend = "github:mozilla/sccache" @@ -94,6 +110,11 @@ checksum = "sha256:29caf036bdbb4e6f07afea31706b6f386cb5a4db9a46a3a8b462b9b78157e url = "https://github.com/rust-cross/cargo-zigbuild/releases/download/v0.22.3/cargo-zigbuild-aarch64-apple-darwin.tar.xz" url_api = "https://api.github.com/repos/rust-cross/cargo-zigbuild/releases/assets/405676922" +[tools."github:rust-cross/cargo-zigbuild"."platforms.windows-x64"] +checksum = "sha256:675804464634cf068dc206e9fafc1bd4557ffcc0b1638a1ff246d899b776fe35" +url = "https://github.com/rust-cross/cargo-zigbuild/releases/download/v0.22.3/cargo-zigbuild-x86_64-pc-windows-msvc.zip" +url_api = "https://api.github.com/repos/rust-cross/cargo-zigbuild/releases/assets/405676944" + [[tools.helm]] version = "4.2.0" backend = "aqua:helm/helm" @@ -110,6 +131,10 @@ url = "https://get.helm.sh/helm-v4.2.0-linux-amd64.tar.gz" checksum = "sha256:f13f959015447b6bc309f9fd506509926543988a39035c088b52522ec95e2acb" url = "https://get.helm.sh/helm-v4.2.0-darwin-arm64.tar.gz" +[tools.helm."platforms.windows-x64"] +checksum = "sha256:34bf9659f8f04f3841a60131183b8e2acc44e260db4c93b889ff09718cacca6f" +url = "https://get.helm.sh/helm-v4.2.0-windows-amd64.tar.gz" + [[tools.helm-docs]] version = "1.14.2" backend = "aqua:norwoodj/helm-docs" @@ -126,6 +151,10 @@ url = "https://github.com/norwoodj/helm-docs/releases/download/v1.14.2/helm-docs checksum = "sha256:2d8399db5b33d240d5f8985241bcf5483563150b968e3229823822979f3e4b8b" url = "https://github.com/norwoodj/helm-docs/releases/download/v1.14.2/helm-docs_1.14.2_Darwin_arm64.tar.gz" +[tools.helm-docs."platforms.windows-x64"] +checksum = "sha256:3c54fd78d99e2769cf83a0faf12cf6cd4de1cac6ed7bee8d908a2c8fc23f538c" +url = "https://github.com/norwoodj/helm-docs/releases/download/v1.14.2/helm-docs_1.14.2_Windows_x86_64.tar.gz" + [[tools.k3d]] version = "5.8.3" backend = "aqua:k3d-io/k3d" @@ -158,6 +187,10 @@ url = "https://dl.k8s.io/v1.36.1/bin/linux/amd64/kubectl" checksum = "sha256:9092778abaef3079449da4cd70ded0e4be112480c93efcdeace3155968d1d133" url = "https://dl.k8s.io/v1.36.1/bin/darwin/arm64/kubectl" +[tools.kubectl."platforms.windows-x64"] +checksum = "sha256:538f4229eee91a17b34724da7daade7687393d6988e33b723c6c306572c13900" +url = "https://dl.k8s.io/v1.36.1/bin/windows/amd64/kubectl.exe" + [[tools.node]] version = "24.15.0" backend = "core:node" @@ -174,6 +207,10 @@ url = "https://nodejs.org/dist/v24.15.0/node-v24.15.0-linux-x64.tar.gz" checksum = "sha256:372331b969779ab5d15b949884fc6eaf88d5afe87bde8ba881d6400b9100ffc4" url = "https://nodejs.org/dist/v24.15.0/node-v24.15.0-darwin-arm64.tar.gz" +[tools.node."platforms.windows-x64"] +checksum = "sha256:cc5149eabd53779ce1e7bdc5401643622d0c7e6800ade18928a767e940bb0e62" +url = "https://nodejs.org/dist/v24.15.0/node-v24.15.0-win-x64.zip" + [[tools."npm:markdownlint-cli2"]] version = "0.22.0" backend = "npm:markdownlint-cli2" @@ -194,6 +231,10 @@ url = "https://github.com/protocolbuffers/protobuf/releases/download/v29.6/proto checksum = "sha256:b9576b5fa1a1ef3fe13a8c91d9d8204b46545759bea5ae155cd6ba2ea4cdaeed" url = "https://github.com/protocolbuffers/protobuf/releases/download/v29.6/protoc-29.6-osx-aarch_64.zip" +[tools.protoc."platforms.windows-x64"] +checksum = "sha256:1ebd7c87baffb9f1c47169b640872bf5fb1e4408079c691af527be9561d8f6f7" +url = "https://github.com/protocolbuffers/protobuf/releases/download/v29.6/protoc-29.6-win64.zip" + [[tools.python]] version = "3.14.5" backend = "core:python" @@ -216,6 +257,11 @@ checksum = "sha256:3a0373cc39fefd494754ef555267f245c720cddbaaabf63a7c9a4269f1e56 url = "https://github.com/astral-sh/python-build-standalone/releases/download/20260602/cpython-3.14.5+20260602-aarch64-apple-darwin-install_only_stripped.tar.gz" provenance = "github-attestations" +[tools.python."platforms.windows-x64"] +checksum = "blake3:1af86eafd814227465499d996373e89f56f750b8044eb6188621615d8ff4f8fb" +url = "https://github.com/astral-sh/python-build-standalone/releases/download/20260602/cpython-3.14.5+20260602-x86_64-pc-windows-msvc-install_only_stripped.tar.gz" +provenance = "github-attestations" + [[tools.rust]] version = "1.95.0" backend = "core:rust" @@ -234,6 +280,10 @@ url = "https://storage.googleapis.com/skaffold/releases/v2.20.0/skaffold-linux-a [tools.skaffold."platforms.macos-arm64"] url = "https://storage.googleapis.com/skaffold/releases/v2.20.0/skaffold-darwin-arm64" +[tools.skaffold."platforms.windows-x64"] +checksum = "blake3:f6db036671e29118d0fdc4457dc563e99d7bff8a39f0f2f72e3e5d857d9013a3" +url = "https://storage.googleapis.com/skaffold/releases/v2.20.0/skaffold-windows-amd64.exe" + [[tools.uv]] version = "0.10.12" backend = "aqua:astral-sh/uv" @@ -253,6 +303,11 @@ checksum = "sha256:ae738b5661a900579ec621d3918c0ef17bdec0da2a8a6d8b161137cd15f25 url = "https://github.com/astral-sh/uv/releases/download/0.10.12/uv-aarch64-apple-darwin.tar.gz" provenance = "github-attestations" +[tools.uv."platforms.windows-x64"] +checksum = "sha256:4c1d55501869b3330d4aabf45ad6024ce2367e0f3af83344395702d272c22e88" +url = "https://github.com/astral-sh/uv/releases/download/0.10.12/uv-x86_64-pc-windows-msvc.zip" +provenance = "github-attestations" + [[tools.zig]] version = "0.14.1" backend = "core:zig" @@ -268,3 +323,8 @@ url = "https://ziglang.org/download/0.14.1/zig-x86_64-linux-0.14.1.tar.xz" [tools.zig."platforms.macos-arm64"] checksum = "sha256:39f3dc5e79c22088ce878edc821dedb4ca5a1cd9f5ef915e9b3cc3053e8faefa" url = "https://ziglang.org/download/0.14.1/zig-aarch64-macos-0.14.1.tar.xz" + +[tools.zig."platforms.windows-x64"] +checksum = "sha256:554f5378228923ffd558eac35e21af020c73789d87afeabf4bfd16f2e6feed2c" +url = "https://ziglang.org/download/0.14.1/zig-x86_64-windows-0.14.1.zip" +provenance = "minisign" diff --git a/mise.toml b/mise.toml index a5d9682108..b4c00e1d8e 100644 --- a/mise.toml +++ b/mise.toml @@ -27,7 +27,7 @@ uv = "0.10.12" protoc = "29.6" helm = "4.2.0" helm-docs = "1.14.2" -skaffold = "2.20.0" +skaffold = { version = "2.20.0", os = ["linux", "macos"] } # Keep k3d out of Linux CI images until upstream ships a release rebuilt with # patched Go/container dependencies. Linux Kubernetes E2E uses kind or an # externally provided cluster context. diff --git a/python/openshell/sandbox.py b/python/openshell/sandbox.py index b62a9c8855..db0db85d85 100644 --- a/python/openshell/sandbox.py +++ b/python/openshell/sandbox.py @@ -1531,7 +1531,17 @@ def _write_to_disk(self, bundle: dict) -> None: f.write(payload) with contextlib.suppress(OSError): tmp_path.chmod(0o600) - tmp_path.replace(path) + for attempt in range(20): + try: + tmp_path.replace(path) + break + except PermissionError: + # Concurrent MoveFileExW calls can briefly lock the + # destination on Windows. Retry that transient sharing + # violation while preserving POSIX rename behavior. + if os.name != "nt" or attempt == 19: + raise + time.sleep(0.01) except BaseException: # Clean up our tmp on failure so we don't leave orphaned # `.oidc_token..tmp` files lying around. The replace diff --git a/scripts/update_license_headers.py b/scripts/update_license_headers.py index f56dbc2935..0f2d87ddd5 100755 --- a/scripts/update_license_headers.py +++ b/scripts/update_license_headers.py @@ -101,7 +101,7 @@ def find_repo_root() -> Path: def is_excluded(rel: Path) -> bool: """Return True if a path should be skipped.""" - rel_str = str(rel) + rel_str = rel.as_posix() # Exact filename exclusions. if rel.name in EXCLUDE_FILES: diff --git a/tasks/helm.toml b/tasks/helm.toml index 24b6667b1d..dd5128ef64 100644 --- a/tasks/helm.toml +++ b/tasks/helm.toml @@ -20,6 +20,7 @@ run = """ exit 1 fi """ +run_windows = "echo Skipping helm:docs:check: Helm validation is not part of the native Windows lane." hide = true ["helm:lint"] @@ -38,6 +39,7 @@ run = """ done echo "All variants passed." """ +run_windows = "echo Skipping helm:lint: Helm validation is not part of the native Windows lane." ["helm:test"] description = "Run Helm chart unit tests" @@ -49,6 +51,7 @@ run = """ helm dependency build deploy/helm/openshell helm unittest deploy/helm/openshell """ +run_windows = "echo Skipping helm:test: Helm validation is not part of the native Windows lane." ["helm:skaffold:dev"] description = "Run skaffold dev for deploy/helm/openshell (iterative deploy)" diff --git a/tasks/markdown.toml b/tasks/markdown.toml index fb070fd683..d0cba9d492 100644 --- a/tasks/markdown.toml +++ b/tasks/markdown.toml @@ -9,6 +9,7 @@ run = """ cd scripts/lint-mermaid npm ci --no-audit --no-fund --silent """ +run_windows = "npm --prefix scripts/lint-mermaid ci --no-audit --no-fund --silent" hide = true sources = ["scripts/lint-mermaid/package.json", "scripts/lint-mermaid/package-lock.json"] outputs = ["scripts/lint-mermaid/node_modules/.package-lock.json"] diff --git a/tasks/rust.toml b/tasks/rust.toml index f035193fd8..37099675b8 100644 --- a/tasks/rust.toml +++ b/tasks/rust.toml @@ -6,6 +6,7 @@ ["rust:check"] description = "Check all Rust crates for errors" run = "cargo check --workspace" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 check native" hide = true ["rust:lint"] @@ -14,6 +15,7 @@ run = [ "cargo clippy --workspace --all-targets -- -D warnings", "cargo clippy --manifest-path e2e/rust/Cargo.toml --all-targets -- -D warnings", ] +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 lint native" hide = true ["rust:format"] diff --git a/tasks/scripts/windows-msvc.ps1 b/tasks/scripts/windows-msvc.ps1 new file mode 100644 index 0000000000..aa02d66e77 --- /dev/null +++ b/tasks/scripts/windows-msvc.ps1 @@ -0,0 +1,740 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Windows MSVC build wrapper used by the `windows:*` mise tasks. + +[CmdletBinding()] +param( + [Parameter(Mandatory = $true, Position = 0)] + [ValidateSet("check", "lint", "build", "test", "test-precommit", "test-unsupported", "artifacts", "ci")] + [string] $Action, + + [Parameter(Position = 1)] + [ValidateSet("x86_64-pc-windows-msvc", "aarch64-pc-windows-msvc", "native", "all")] + [string] $Target = "all", + + [string] $LogDir +) + +Set-StrictMode -Version Latest +$ErrorActionPreference = "Stop" + +if (-not [System.Runtime.InteropServices.RuntimeInformation]::IsOSPlatform([System.Runtime.InteropServices.OSPlatform]::Windows)) { + throw "windows-msvc.ps1 requires a Windows MSVC host." +} + +$RepoRoot = (Resolve-Path (Join-Path $PSScriptRoot "..\..")).Path +if (-not $LogDir) { + $LogDir = $RepoRoot +} +if (-not (Test-Path $LogDir)) { + New-Item -ItemType Directory -Force -Path $LogDir | Out-Null +} +$LogDir = (Resolve-Path $LogDir).Path + +$TargetDirWasConfigured = -not [string]::IsNullOrWhiteSpace($env:CARGO_TARGET_DIR) +$TargetDir = $env:CARGO_TARGET_DIR +if (-not $TargetDirWasConfigured) { + $TargetDir = Join-Path $RepoRoot "target" +} + +$BundledZ3CacheRoot = $TargetDir +if (-not $TargetDirWasConfigured) { + $userCacheRoot = [Environment]::GetFolderPath([Environment+SpecialFolder]::LocalApplicationData) + if ([string]::IsNullOrWhiteSpace($userCacheRoot)) { + $userCacheRoot = $env:LOCALAPPDATA + } + if ([string]::IsNullOrWhiteSpace($userCacheRoot)) { + $userCacheRoot = [IO.Path]::GetTempPath() + } + $BundledZ3CacheRoot = Join-Path $userCacheRoot "OpenShell\cache\z3" +} + +$BuildJobsValue = $env:OPENSHELL_WINDOWS_BUILD_JOBS +if ([string]::IsNullOrWhiteSpace($BuildJobsValue)) { + $BuildJobsValue = $env:CARGO_BUILD_JOBS +} +if ([string]::IsNullOrWhiteSpace($BuildJobsValue)) { + $BuildJobsValue = "4" +} +[int] $WindowsBuildJobs = 0 +if (-not [int]::TryParse($BuildJobsValue, [ref] $WindowsBuildJobs) -or $WindowsBuildJobs -lt 1) { + throw "OPENSHELL_WINDOWS_BUILD_JOBS or CARGO_BUILD_JOBS must be a positive integer." +} +$WindowsCargoMutex = [System.Threading.Mutex]::new($false, "Local\OpenShellWindowsMsvcCargo") + +$UnsupportedDriverPackageExcludes = "--exclude openshell-driver-docker --exclude openshell-driver-kubernetes --exclude openshell-driver-podman --exclude openshell-driver-vm --exclude openshell-supervisor-process" +$WindowsClippyPackageExcludes = $UnsupportedDriverPackageExcludes +$WindowsClippyLintArgs = "-D warnings -A dead-code -A unused-imports -A clippy::unused-async" +$BundledZ3WorkspaceFeatures = "--features openshell-prover/bundled-z3" +$BundledZ3ServerFeatures = "--features openshell-server/bundled-z3,openshell-prover/bundled-z3" +$BundledZ3Repository = "https://github.com/Z3Prover/z3.git" +$BundledZ3SysVersion = "0.10.9" +# This is the matching z3-sys submodule revision. Update both pins together. +$BundledZ3Revision = "ddb49568d3520e99799e364fb22f35fc67d887b1" +$Z3WorkspaceFeatures = $BundledZ3WorkspaceFeatures +$Z3ServerFeatures = $BundledZ3ServerFeatures + +function Get-VsInstallRoots { + $programFiles = @( + [Environment]::GetEnvironmentVariable("ProgramFiles"), + [Environment]::GetEnvironmentVariable("ProgramFiles(x86)") + ) | Where-Object { $_ } + $editions = @("Enterprise", "Professional", "Community", "BuildTools") + $candidates = @() + + foreach ($programFilesRoot in $programFiles) { + $vsRoot = Join-Path $programFilesRoot "Microsoft Visual Studio" + if (-not (Test-Path $vsRoot -PathType Container)) { + continue + } + foreach ($releaseDir in Get-ChildItem $vsRoot -Directory) { + foreach ($edition in $editions) { + $installRoot = Join-Path $releaseDir.FullName $edition + $vsDevCmd = Join-Path $installRoot "Common7\Tools\VsDevCmd.bat" + if (-not (Test-Path $vsDevCmd -PathType Leaf)) { + continue + } + + $toolsetVersion = [version] "0.0" + $versionFile = Join-Path $installRoot "VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt" + if (Test-Path $versionFile -PathType Leaf) { + try { + $toolsetVersion = [version] ((Get-Content $versionFile -Raw).Trim()) + } catch { + $toolsetVersion = [version] "0.0" + } + } + $candidates += [pscustomobject]@{ + Root = $installRoot + ToolsetVersion = $toolsetVersion + } + } + } + } + + return @($candidates | Sort-Object ToolsetVersion -Descending | Select-Object -ExpandProperty Root -Unique) +} + +function Get-DefaultMsvcToolsetRoot([string] $VsInstallRoot) { + $versionFile = Join-Path $VsInstallRoot "VC\Auxiliary\Build\Microsoft.VCToolsVersion.default.txt" + if (Test-Path $versionFile -PathType Leaf) { + $version = (Get-Content $versionFile -Raw).Trim() + $toolsetRoot = Join-Path $VsInstallRoot "VC\Tools\MSVC\$version" + if (Test-Path $toolsetRoot -PathType Container) { + return (Resolve-Path $toolsetRoot).Path + } + } + + $toolsetsRoot = Join-Path $VsInstallRoot "VC\Tools\MSVC" + if (Test-Path $toolsetsRoot -PathType Container) { + $toolset = Get-ChildItem $toolsetsRoot -Directory | + Sort-Object { try { [version] $_.Name } catch { [version] "0.0" } } -Descending | + Select-Object -First 1 + if ($toolset) { + return $toolset.FullName + } + } + + return $null +} + +function Test-VsInstanceSupportsTarget([string] $VsInstallRoot, [string] $RustTarget) { + $toolsetRoot = Get-DefaultMsvcToolsetRoot $VsInstallRoot + if (-not $toolsetRoot) { + return $false + } + + $hostToolsDir = switch (Get-HostArch) { + "arm64" { "Hostarm64" } + default { "Hostx64" } + } + $targetToolsDir = switch (Get-VsTargetArch $RustTarget) { + "arm64" { "arm64" } + default { "x64" } + } + $compiler = Join-Path $toolsetRoot "bin\$hostToolsDir\$targetToolsDir\cl.exe" + if (-not (Test-Path $compiler -PathType Leaf)) { + return $false + } + + if ($RustTarget -eq "aarch64-pc-windows-msvc") { + $spectreLibs = Join-Path $toolsetRoot "lib\spectre\arm64" + if (-not (Test-Path $spectreLibs -PathType Container)) { + return $false + } + } + + return $true +} + +function Resolve-VsDevCmd([string] $RustTarget) { + if ($env:OPENSHELL_VSDEVCMD -and (Test-Path $env:OPENSHELL_VSDEVCMD)) { + return (Resolve-Path $env:OPENSHELL_VSDEVCMD).Path + } + + $programFilesX86 = [Environment]::GetEnvironmentVariable("ProgramFiles(x86)") + if ($programFilesX86) { + $vswhere = Join-Path $programFilesX86 "Microsoft Visual Studio\Installer\vswhere.exe" + } else { + $vswhere = $null + } + if ($vswhere -and (Test-Path $vswhere)) { + $requiredComponents = switch ($RustTarget) { + "x86_64-pc-windows-msvc" { @("Microsoft.VisualStudio.Component.VC.Tools.x86.x64") } + "aarch64-pc-windows-msvc" { + @( + "Microsoft.VisualStudio.Component.VC.Tools.ARM64", + "Microsoft.VisualStudio.Component.VC.Runtimes.ARM64.Spectre" + ) + } + default { throw "Unsupported target: $RustTarget" } + } + $found = & $vswhere -latest -products * -requires $requiredComponents -find "Common7\Tools\VsDevCmd.bat" | Select-Object -First 1 + if ($found -and (Test-Path $found)) { + $resolved = (Resolve-Path $found).Path + $installRoot = (Resolve-Path (Join-Path (Split-Path -Parent $resolved) "..\..")).Path + if (Test-VsInstanceSupportsTarget $installRoot $RustTarget) { + return $resolved + } + } + } + + foreach ($installRoot in Get-VsInstallRoots) { + if (Test-VsInstanceSupportsTarget $installRoot $RustTarget) { + $candidate = Join-Path $installRoot "Common7\Tools\VsDevCmd.bat" + return (Resolve-Path $candidate).Path + } + } + + if ($RustTarget -eq "aarch64-pc-windows-msvc") { + throw "Could not find a Visual Studio instance with the ARM64 compiler and ARM64 Spectre-mitigated libraries. Install Microsoft.VisualStudio.Component.VC.Tools.ARM64 and Microsoft.VisualStudio.Component.VC.Runtimes.ARM64.Spectre, or set OPENSHELL_VSDEVCMD." + } + throw "Could not find a Visual Studio instance with the x64 compiler. Install Microsoft.VisualStudio.Component.VC.Tools.x86.x64, or set OPENSHELL_VSDEVCMD." +} + +function Get-LibclangBinSubdir { + return ([System.Runtime.InteropServices.RuntimeInformation, mscorlib]::OSArchitecture.ToString()) +} + +function Resolve-LibclangPath { + $subdir = Get-LibclangBinSubdir + + if ($env:LIBCLANG_PATH) { + $candidate = Join-Path $env:LIBCLANG_PATH "libclang.dll" + if (Test-Path $candidate) { + return (Resolve-Path $env:LIBCLANG_PATH).Path + } + throw "LIBCLANG_PATH is set but libclang.dll was not found at: $candidate" + } + + $programFilesX86 = [Environment]::GetEnvironmentVariable("ProgramFiles(x86)") + if ($programFilesX86) { + $vswhere = Join-Path $programFilesX86 "Microsoft Visual Studio\Installer\vswhere.exe" + } else { + $vswhere = $null + } + if ($vswhere -and (Test-Path $vswhere)) { + $found = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.Llvm.Clang -find "VC\Tools\Llvm\$subdir\bin\libclang.dll" | Select-Object -First 1 + if ($found -and (Test-Path $found)) { + return (Split-Path -Parent (Resolve-Path $found).Path) + } + } + + foreach ($installRoot in Get-VsInstallRoots) { + $candidateDir = Join-Path $installRoot "VC\Tools\Llvm\$subdir\bin" + $candidate = Join-Path $candidateDir "libclang.dll" + if (Test-Path $candidate -PathType Leaf) { + return (Resolve-Path $candidateDir).Path + } + } + + $llvmDir = "C:\Program Files\LLVM\bin" + if (Test-Path (Join-Path $llvmDir "libclang.dll")) { + return (Resolve-Path $llvmDir).Path + } + + throw "Could not find libclang.dll. Install Visual Studio C++ Clang tools, or set LIBCLANG_PATH to the directory containing libclang.dll." +} + +function Resolve-NinjaPath { + $fromPath = Get-Command ninja.exe -ErrorAction SilentlyContinue + if ($fromPath) { + return $fromPath.Source + } + + $programFilesX86 = [Environment]::GetEnvironmentVariable("ProgramFiles(x86)") + if ($programFilesX86) { + $vswhere = Join-Path $programFilesX86 "Microsoft Visual Studio\Installer\vswhere.exe" + } else { + $vswhere = $null + } + if ($vswhere -and (Test-Path $vswhere)) { + $found = & $vswhere -latest -products * -requires Microsoft.VisualStudio.Component.VC.CMake.Project -find "Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.exe" | Select-Object -First 1 + if ($found -and (Test-Path $found -PathType Leaf)) { + return (Resolve-Path $found).Path + } + } + + foreach ($installRoot in Get-VsInstallRoots) { + $candidate = Join-Path $installRoot "Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja\ninja.exe" + if (Test-Path $candidate -PathType Leaf) { + return (Resolve-Path $candidate).Path + } + } + + throw "Could not find ninja.exe. Install Microsoft.VisualStudio.Component.VC.CMake.Project." +} + +function Add-PathEntry([string] $Directory) { + if (($env:PATH -split ";") -notcontains $Directory) { + $env:PATH = "$Directory;$env:PATH" + } +} + +function Configure-Arm64CrossBuild([string[]] $RustTargets) { + if ((Get-HostArch) -ne "amd64" -or $RustTargets -notcontains "aarch64-pc-windows-msvc") { + return + } + + $clangCl = Join-Path $env:LIBCLANG_PATH "clang-cl.exe" + if (-not (Test-Path $clangCl -PathType Leaf)) { + throw "ARM64 cross-compilation requires host-native clang-cl.exe next to libclang.dll. Install Microsoft.VisualStudio.Component.VC.Llvm.Clang." + } + Add-PathEntry $env:LIBCLANG_PATH + + $ninja = Resolve-NinjaPath + Add-PathEntry (Split-Path -Parent $ninja) + + Write-Host "==> ARM64 cross-build toolchain" + Write-Host " clang-cl: $clangCl" + Write-Host " ninja: $ninja" + Write-Host " Z3: MSVC cl.exe with the Visual Studio generator" +} + +function Get-HostArch { + switch ([System.Runtime.InteropServices.RuntimeInformation]::OSArchitecture.ToString()) { + "Arm64" { "arm64" } + default { "amd64" } + } +} + +function Get-VsTargetArch([string] $RustTarget) { + switch ($RustTarget) { + "x86_64-pc-windows-msvc" { "amd64" } + "aarch64-pc-windows-msvc" { "arm64" } + default { throw "Unsupported target: $RustTarget" } + } +} + +function Assert-NativeTestTarget([string] $RustTarget) { + $targetArch = Get-VsTargetArch $RustTarget + $hostArch = Get-HostArch + if ($targetArch -ne $hostArch) { + throw "Windows tests require a native runner. Target $RustTarget maps to $targetArch, but the host is $hostArch." + } +} + +function Get-SelectedTargets([string] $RequestedTarget) { + if ($RequestedTarget -eq "native") { + switch (Get-HostArch) { + "arm64" { return @("aarch64-pc-windows-msvc") } + default { return @("x86_64-pc-windows-msvc") } + } + } + if ($RequestedTarget -eq "all") { + $targets = @("x86_64-pc-windows-msvc") + if ($env:OPENSHELL_MXC_SKIP_ARM64 -ne "1") { + $targets += "aarch64-pc-windows-msvc" + } + return $targets + } + return @($RequestedTarget) +} + +function Resolve-Z3HeaderPath([string] $HeaderPath) { + if ([string]::IsNullOrWhiteSpace($HeaderPath)) { + throw "Z3_LIBRARY_PATH_OVERRIDE is set. Set Z3_SYS_Z3_HEADER to the full path of z3.h." + } + + if (-not (Test-Path $HeaderPath -PathType Leaf)) { + throw "Z3_SYS_Z3_HEADER is set but z3.h was not found at: $HeaderPath" + } + if ((Split-Path -Leaf $HeaderPath) -ne "z3.h") { + throw "Z3_SYS_Z3_HEADER must point to z3.h. Got: $HeaderPath" + } + + return (Resolve-Path $HeaderPath).Path +} + +function Assert-BundledZ3Source([string] $SourcePath, [string] $ExpectedRevision) { + if (-not (Test-Path $SourcePath -PathType Container)) { + throw "Bundled Z3 source directory does not exist: $SourcePath" + } + + $header = Join-Path $SourcePath "src\api\z3.h" + if (-not (Test-Path $header -PathType Leaf)) { + throw "Bundled Z3 source directory does not contain src\api\z3.h: $SourcePath" + } + + if (-not [string]::IsNullOrWhiteSpace($ExpectedRevision)) { + $actualRevision = (& git -C $SourcePath rev-parse HEAD 2>$null) + if ($LASTEXITCODE -ne 0 -or [string]::IsNullOrWhiteSpace($actualRevision)) { + throw "Could not verify the bundled Z3 source revision at: $SourcePath" + } + if ($actualRevision.Trim() -ne $ExpectedRevision) { + throw "Bundled Z3 source revision mismatch at ${SourcePath}: expected $ExpectedRevision, found $($actualRevision.Trim())" + } + } + + return (Resolve-Path $SourcePath).Path +} + +function Resolve-BundledZ3Source { + if (-not [string]::IsNullOrWhiteSpace($env:Z3_SYS_BUNDLED_DIR_OVERRIDE)) { + return Assert-BundledZ3Source $env:Z3_SYS_BUNDLED_DIR_OVERRIDE "" + } + + $cargoLock = Get-Content (Join-Path $RepoRoot "Cargo.lock") -Raw + $packagePattern = '(?ms)^\[\[package\]\]\s+name = "z3-sys"\s+version = "([^"]+)"' + $packageMatches = [regex]::Matches($cargoLock, $packagePattern) + if ($packageMatches.Count -ne 1 -or $packageMatches[0].Groups[1].Value -ne $BundledZ3SysVersion) { + throw "Bundled Z3 source pin expects z3-sys $BundledZ3SysVersion. Update the version and revision pins for the z3-sys version in Cargo.lock." + } + + $revisionPrefix = $BundledZ3Revision.Substring(0, 12) + $sourcePath = Join-Path $BundledZ3CacheRoot "z3-source-$revisionPrefix" + if (Test-Path $sourcePath) { + return Assert-BundledZ3Source $sourcePath $BundledZ3Revision + } + + if (-not (Get-Command git.exe -ErrorAction SilentlyContinue)) { + throw "Bundled Z3 source preparation requires git.exe on PATH." + } + if (-not (Test-Path $BundledZ3CacheRoot -PathType Container)) { + New-Item -ItemType Directory -Force -Path $BundledZ3CacheRoot | Out-Null + } + + $stagingPath = "$sourcePath.partial-$([guid]::NewGuid().ToString('N'))" + Write-Host "==> Fetching bundled Z3 source" + Write-Host " repository: $BundledZ3Repository" + Write-Host " revision: $BundledZ3Revision" + Write-Host " cache: $sourcePath" + + & git init --quiet $stagingPath + if ($LASTEXITCODE -ne 0) { + throw "git init failed while preparing bundled Z3 source at: $stagingPath" + } + & git -C $stagingPath remote add origin $BundledZ3Repository + if ($LASTEXITCODE -ne 0) { + throw "git remote add failed while preparing bundled Z3 source at: $stagingPath" + } + & git -C $stagingPath fetch --quiet --depth 1 origin $BundledZ3Revision + if ($LASTEXITCODE -ne 0) { + throw "git fetch failed for bundled Z3 revision $BundledZ3Revision. Partial source remains at: $stagingPath" + } + & git -C $stagingPath checkout --quiet --detach FETCH_HEAD + if ($LASTEXITCODE -ne 0) { + throw "git checkout failed for bundled Z3 revision $BundledZ3Revision. Partial source remains at: $stagingPath" + } + + Assert-BundledZ3Source $stagingPath $BundledZ3Revision | Out-Null + try { + # Directory.Move is an atomic rename on the same volume and, unlike + # Move-Item, fails when the destination already exists. A concurrent + # x64/ARM64 invocation can therefore win publication without the loser + # nesting its staging directory inside the shared cache. + [IO.Directory]::Move($stagingPath, $sourcePath) + } catch { + if (-not (Test-Path $sourcePath -PathType Container)) { + throw + } + Write-Host "==> Reusing bundled Z3 source published by another process" + } finally { + if (Test-Path $stagingPath -PathType Container) { + try { + Remove-Item -LiteralPath $stagingPath -Recurse -Force + } catch { + Write-Warning "Could not remove redundant bundled Z3 staging directory: $stagingPath" + } + } + } + return Assert-BundledZ3Source $sourcePath $BundledZ3Revision +} + +function Configure-Z3 { + if ([string]::IsNullOrWhiteSpace($env:Z3_LIBRARY_PATH_OVERRIDE)) { + Write-Host "==> Z3: bundled" + $env:Z3_SYS_BUNDLED_DIR_OVERRIDE = Resolve-BundledZ3Source + Write-Host " Z3_SYS_BUNDLED_DIR_OVERRIDE=$env:Z3_SYS_BUNDLED_DIR_OVERRIDE" + return [pscustomobject]@{ + WorkspaceFeatures = $BundledZ3WorkspaceFeatures + ServerFeatures = $BundledZ3ServerFeatures + } + } + + if (-not (Test-Path $env:Z3_LIBRARY_PATH_OVERRIDE -PathType Container)) { + throw "Z3_LIBRARY_PATH_OVERRIDE is set but the directory does not exist: $env:Z3_LIBRARY_PATH_OVERRIDE" + } + + $libDir = (Resolve-Path $env:Z3_LIBRARY_PATH_OVERRIDE).Path + $importLib = Join-Path $libDir "libz3.lib" + if (-not (Test-Path $importLib -PathType Leaf)) { + throw "Z3_LIBRARY_PATH_OVERRIDE is set but libz3.lib was not found at: $importLib" + } + + $env:Z3_LIBRARY_PATH_OVERRIDE = $libDir + $env:Z3_SYS_Z3_HEADER = Resolve-Z3HeaderPath $env:Z3_SYS_Z3_HEADER + + if (($env:PATH -split ";") -notcontains $libDir) { + $env:PATH = "$libDir;$env:PATH" + } + + Write-Host "==> Z3: system" + Write-Host " Z3_LIBRARY_PATH_OVERRIDE=$env:Z3_LIBRARY_PATH_OVERRIDE" + Write-Host " Z3_SYS_Z3_HEADER=$env:Z3_SYS_Z3_HEADER" + + return [pscustomobject]@{ + WorkspaceFeatures = "" + ServerFeatures = "" + } +} + +function Invoke-VsCargo { + param( + [Parameter(Mandatory = $true)] [string] $RustTarget, + [Parameter(Mandatory = $true)] [string] $CargoArgs, + [Parameter(Mandatory = $true)] [string] $LogName + ) + + & rustup target add $RustTarget + if ($LASTEXITCODE -ne 0) { + throw "rustup target add $RustTarget failed" + } + + $vsDevCmd = Resolve-VsDevCmd $RustTarget + $targetArch = Get-VsTargetArch $RustTarget + $hostArch = Get-HostArch + $logPath = Join-Path $LogDir $LogName + $environmentSetup = @( + "set `"CARGO_TARGET_DIR=$TargetDir`"", + "set `"CARGO_BUILD_JOBS=$WindowsBuildJobs`"", + "set `"CARGO_INCREMENTAL=0`"", + "set `"RUSTC_WRAPPER=`"" + ) + if ($hostArch -eq "amd64" -and $RustTarget -eq "aarch64-pc-windows-msvc") { + # Let cmake-rs select MSVC cl.exe for bundled Z3. AWS-LC selects + # clang-cl inside its own ARM64 build script. + $environmentSetup += @( + "set `"CC=`"", + "set `"CXX=`"", + "set `"CC_aarch64-pc-windows-msvc=`"", + "set `"CXX_aarch64-pc-windows-msvc=`"", + "set `"CC_aarch64_pc_windows_msvc=`"", + "set `"CXX_aarch64_pc_windows_msvc=`"" + ) + } + $cmd = "call `"$vsDevCmd`" -arch=$targetArch -host_arch=$hostArch && $($environmentSetup -join ' && ') && $CargoArgs" + + Write-Host "==> $CargoArgs" + Write-Host " target: $RustTarget" + Write-Host " log: $logPath" + + $lockAcquired = $false + try { + try { + $lockAcquired = $WindowsCargoMutex.WaitOne(0) + if (-not $lockAcquired) { + Write-Host " waiting for another Windows Cargo task" + $lockAcquired = $WindowsCargoMutex.WaitOne([TimeSpan]::FromHours(2)) + } + } catch [System.Threading.AbandonedMutexException] { + $lockAcquired = $true + } + if (-not $lockAcquired) { + throw "Timed out waiting for another Windows Cargo task to finish." + } + + $cmdWithLog = "$cmd > `"$logPath`" 2>&1" + & cmd /v:on /d /c $cmdWithLog + $exitCode = $LASTEXITCODE + if (Test-Path $logPath) { + Get-Content $logPath + } + if ($exitCode -ne 0) { + throw "Command failed with exit code $exitCode. See $logPath" + } + } finally { + if ($lockAcquired) { + $WindowsCargoMutex.ReleaseMutex() + } + } +} + +function Invoke-Check([string] $RustTarget) { + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo check --workspace $UnsupportedDriverPackageExcludes --target $RustTarget $Z3WorkspaceFeatures" ` + -LogName "build-$RustTarget-check.log" +} + +function Invoke-Lint([string] $RustTarget) { + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo clippy --workspace --all-targets --no-deps $WindowsClippyPackageExcludes --target $RustTarget $Z3WorkspaceFeatures -- $WindowsClippyLintArgs" ` + -LogName "lint-$RustTarget-workspace.log" + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo clippy --manifest-path e2e/rust/Cargo.toml --all-targets --no-deps --target $RustTarget -- $WindowsClippyLintArgs" ` + -LogName "lint-$RustTarget-e2e.log" +} + +function Invoke-Build([string] $RustTarget) { + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo build --release --target $RustTarget --bin openshell-gateway --bin openshell $Z3WorkspaceFeatures" ` + -LogName "build-$RustTarget-release.log" +} + +function Invoke-Test([string] $RustTarget) { + Assert-NativeTestTarget $RustTarget + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo test --workspace $UnsupportedDriverPackageExcludes --target $RustTarget --no-fail-fast $Z3WorkspaceFeatures" ` + -LogName "test-$RustTarget.log" +} + +function Invoke-PreCommitTest([string] $RustTarget) { + Assert-NativeTestTarget $RustTarget + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo test --workspace --exclude openshell-server $UnsupportedDriverPackageExcludes --target $RustTarget --no-fail-fast $Z3WorkspaceFeatures" ` + -LogName "test-$RustTarget-precommit-workspace.log" + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo test -p openshell-server --features test-support --target $RustTarget --no-fail-fast $Z3ServerFeatures" ` + -LogName "test-$RustTarget-precommit-server.log" +} + +function Invoke-UnsupportedContractTests([string] $RustTarget) { + Assert-NativeTestTarget $RustTarget + + $tests = @( + "windows_compute_driver_stubs_report_unsupported", + "windows_spawn_reports_unsupported" + ) + foreach ($test in $tests) { + Invoke-VsCargo ` + -RustTarget $RustTarget ` + -CargoArgs "cargo test -p openshell-server --target $RustTarget $test $Z3ServerFeatures" ` + -LogName "test-$RustTarget-unsupported-$test.log" + } +} + +function Get-Sha256([string] $Path) { + $stream = [System.IO.File]::OpenRead($Path) + try { + $sha256 = [System.Security.Cryptography.SHA256]::Create() + try { + return [BitConverter]::ToString($sha256.ComputeHash($stream)).Replace("-", "") + } finally { + $sha256.Dispose() + } + } finally { + $stream.Dispose() + } +} + +function Show-Artifacts([string[]] $RustTargets) { + $rows = @() + foreach ($rustTarget in $RustTargets) { + foreach ($binary in @("openshell-gateway.exe", "openshell.exe")) { + $path = Join-Path $TargetDir "$rustTarget\release\$binary" + if (-not (Test-Path $path)) { + continue + } + $item = Get-Item $path + $rows += [pscustomobject]@{ + Target = $rustTarget + Binary = $binary + Size = $item.Length + SHA256 = Get-Sha256 $item.FullName + Path = $item.FullName + } + } + } + if ($rows.Count -eq 0) { + Write-Warning "No release artifacts found under $TargetDir" + return + } + $rows | Format-Table -AutoSize +} + +if ($Action -eq "ci" -and (Get-HostArch) -ne "amd64") { + throw "windows:ci is an x64-host contract. On ARM64, run windows:check:arm64, windows:build:arm64, windows:test:arm64, windows:test:unsupported:arm64, and windows:artifacts explicitly." +} + +$targets = Get-SelectedTargets $Target +if ($Action -in @("test", "test-precommit", "test-unsupported")) { + foreach ($rustTarget in $targets) { + Assert-NativeTestTarget $rustTarget + } +} + +if ($Action -in @("check", "lint", "build", "test", "test-precommit", "test-unsupported", "ci")) { + $z3Features = Configure-Z3 + $Z3WorkspaceFeatures = $z3Features.WorkspaceFeatures + $Z3ServerFeatures = $z3Features.ServerFeatures + $env:LIBCLANG_PATH = Resolve-LibclangPath + Add-PathEntry $env:LIBCLANG_PATH + Write-Host "==> LIBCLANG_PATH=$env:LIBCLANG_PATH" + Configure-Arm64CrossBuild $targets +} + +switch ($Action) { + "check" { + foreach ($rustTarget in $targets) { + Invoke-Check $rustTarget + } + } + "lint" { + foreach ($rustTarget in $targets) { + Invoke-Lint $rustTarget + } + } + "build" { + foreach ($rustTarget in $targets) { + Invoke-Build $rustTarget + } + Show-Artifacts $targets + } + "test" { + foreach ($rustTarget in $targets) { + Invoke-Test $rustTarget + } + } + "test-precommit" { + foreach ($rustTarget in $targets) { + Invoke-PreCommitTest $rustTarget + } + } + "test-unsupported" { + foreach ($rustTarget in $targets) { + Invoke-UnsupportedContractTests $rustTarget + } + } + "artifacts" { + Show-Artifacts $targets + } + "ci" { + foreach ($rustTarget in $targets) { + Invoke-Check $rustTarget + } + foreach ($rustTarget in $targets) { + Invoke-Build $rustTarget + } + Invoke-Test "x86_64-pc-windows-msvc" + Invoke-UnsupportedContractTests "x86_64-pc-windows-msvc" + Show-Artifacts $targets + } +} diff --git a/tasks/test.toml b/tasks/test.toml index 96dde276ce..8c0d0c99fd 100644 --- a/tasks/test.toml +++ b/tasks/test.toml @@ -21,16 +21,19 @@ hide = true ["test:install-sh"] description = "Run focused install.sh shell tests" run = "tasks/scripts/test-install-sh.sh" +run_windows = "echo Skipping test:install-sh: Linux glibc installer tests do not apply on Windows." hide = true ["test:build-env"] description = "Run build-env.sh helper shell tests" run = "tasks/scripts/test-build-env.sh" +run_windows = "echo Skipping test:build-env: the Unix build-env.sh helper does not apply on Windows." hide = true ["test:packaging-assets"] description = "Run static packaging asset tests" run = "tasks/scripts/test-packaging-assets.sh" +run_windows = "echo Skipping test:packaging-assets: Linux service and RPM assets do not apply on Windows." hide = true [e2e] @@ -53,6 +56,7 @@ run = [ "cargo test --workspace --exclude openshell-server", "cargo test -p openshell-server --features test-support", ] +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-precommit native" hide = true ["test:python"] diff --git a/tasks/windows.toml b/tasks/windows.toml new file mode 100644 index 0000000000..c7a2e51f99 --- /dev/null +++ b/tasks/windows.toml @@ -0,0 +1,65 @@ +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# Windows MSVC build-only tasks. These intentionally live outside the default +# Linux/macOS `ci` task so the established Linux build path remains unchanged. + +["windows:check"] +description = "Check Windows x64 and ARM64 MSVC targets" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 check all" + +["windows:check:x64"] +description = "Check Windows x64 MSVC target" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 check x86_64-pc-windows-msvc" + +["windows:check:arm64"] +description = "Check Windows ARM64 MSVC target" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 check aarch64-pc-windows-msvc" + +["windows:build"] +description = "Build Windows x64 and ARM64 release binaries" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 build all" + +["windows:build:x64"] +description = "Build Windows x64 release binaries" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 build x86_64-pc-windows-msvc" + +["windows:build:arm64"] +description = "Build Windows ARM64 release binaries" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 build aarch64-pc-windows-msvc" + +["windows:test:x64"] +description = "Run Windows x64 MSVC workspace tests" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test x86_64-pc-windows-msvc" + +["windows:test:arm64"] +description = "Run Windows ARM64 MSVC workspace tests on a native ARM64 host" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test aarch64-pc-windows-msvc" + +["windows:test:unsupported:x64"] +description = "Run focused Windows unsupported compute-driver contract tests" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-unsupported x86_64-pc-windows-msvc" + +["windows:test:unsupported:arm64"] +description = "Run focused Windows unsupported compute-driver contract tests on a native ARM64 host" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-unsupported aarch64-pc-windows-msvc" + +["windows:artifacts"] +description = "Report Windows release artifact sizes and SHA256 hashes" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 artifacts all" + +["windows:ci"] +description = "Run Windows MSVC checks, release builds, x64 tests, and unsupported-driver contract tests" +run = "echo 'windows:* tasks require a Windows MSVC host' && exit 1" +run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 ci all"