feat(p2): deferred-main-spawn — full box semantics for pooled sandboxes#22
Merged
Conversation
added 4 commits
June 11, 2026 17:27
Deferred-main-spawn for IDLE-booted (pooled) sandboxes: full box semantics (exit code + json-file console logs) without a cold boot. - Guest: BOX_DEFERRED_MAIN=1 boots IDLE (skip boot spawn, stash BOX_EXEC_* cmd, CONTAINER_PID=-1; ECHILD-with-no-container keeps waiting instead of exiting). A spawn-main control frame runs the stashed cmd as the MAIN via the exec server's build_command (SAME seccomp/user/no-new-privs as a boot main, async- signal-safe pre_exec) with stdio overridden to inherit (→ console → json-file logs), under reaper::spawn_managed; pid CAS-published while MANAGED, then the supervision loop (reads the atomic each tick) reaps it for the real exit code. - Host: spec.rs passes BOX_DEFERRED_MAIN to the guest; ExecClient::spawn_main sends the trigger; boot sends it post-readiness. KVM-verified: (a) safe multi-threaded fork no deadlock (incl. 8 concurrent execs), (b) stdout/stderr stream-tagged in container.json, (c) exit code via /.a3s_exit_code, and security parity (seccomp + uid identical to a boot main; --user 1000 → uid 1000). Remaining for production: cgroup join (resource limits), config field instead of the env trigger, pool Request::SpawnMain, unit/e2e tests, docs.
`pool start --deferred` boots pooled VMs IDLE; `pool run` then spawns the per- request command as the box's real MAIN via the spawn-main control frame (hybrid: frame carries the command since the VM is pre-warmed before the command is known), giving full box semantics — real exit code (/.a3s_exit_code) + json-file console logs — vs the keepalive+exec MVP's piped exec-stream output. - core: BoxConfig.deferred_main; spec.rs/mod.rs honor it (env override kept for tests). Guest spawn-main now accepts an optional command in the frame. - runtime: VmManager::run_deferred_main (send spawn-main + wait for the main to exit + read the box's container.json split by stream); ExecClient::spawn_main takes an optional spec. - cli: pool --deferred flag; handle_conn routes to run_deferred_main.
Unit: deferred_spec_json builds the spawn-main JSON (executable+args+PATH, shell fallback). Host-backed e2e test_real_pool_deferred_main: pool start --deferred + pool run asserts stdout/stderr come back from the box's json-file logs and the real exit code propagates.
Update the P2 design doc to IMPLEMENTED with usage (pool start --deferred / pool run) and what landed vs the design (incl. cgroup parity is free via VM-level limits). Add a warm-pool + --deferred bullet to the README verified-behavior list.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P2 of the CoW plan (design #21, now implemented). Gives a pre-warmed pool sandbox full box semantics — the command runs as the box's real container main (real exit code via
/.a3s_exit_code+ stdout/stderr in the json-file console logs) — instead of the keepalive+exec MVP's exec-stream output, with no cold boot.Usage
How
BoxConfig.deferred_main/BOX_DEFERRED_MAIN=1): skip the boot spawn; PID 1 keeps waiting (theECHILD-with-no-container case no longer exits early — that was the key prototype finding).runpath) or carrying a command (the pool, which pre-warms before the command is known).build_command(identical seccomp / user / no-new-privs to a boot main —Command::spawn, notspawn_isolated's raw fork, so it's safe from the multi-threaded PID 1) with stdio overridden toinherit(→ console → json-file logs). The pid is CAS-published while MANAGED, then the supervision loop reaps it for the real exit code.pool start --deferredboots IDLE VMs;VmManager::run_deferred_mainsends spawn-main, waits for the main to exit, and reads the box'scontainer.jsonsplit by stream. The boot auto-trigger fires only for the env-runpath — pool VMs are driven explicitly per request.set_vm_config), so a deferred main shares the boot main's limits (no guest cgroup needed;ContainerCgroupis exec-path-only).Verified (KVM)
Exit codes 7/3/0, stdout+stderr from the json-file logs, seccomp applied (
Seccomp: 2),--user 1000→uid 1000 — all from a pre-warmed pool. Single + detached + 8-concurrent-exec (no fork deadlock). Tests:deferred_spec_jsonunit +test_real_pool_deferred_mainhost e2e (passes on KVM).