Skip to content

docs: CoW snapshot-fork backend design (Tier 3)#17

Merged
ZhiXiao-Lin merged 1 commit into
mainfrom
docs/cow-snapshot-fork-design
Jun 11, 2026
Merged

docs: CoW snapshot-fork backend design (Tier 3)#17
ZhiXiao-Lin merged 1 commit into
mainfrom
docs/cow-snapshot-fork-design

Conversation

@ZhiXiao-Lin

Copy link
Copy Markdown
Contributor

Design-only proposal (no implementation) for the high-fan-out agent-sandbox story, completing the CoW optimization plan started in #16.

Core decision: libkrun has no snapshot/restore (verified: 0 of ~75 krun_* are snapshot; guest RAM is anonymous-private; krun_start_enter takes over the process). So rather than fork libkrun's memory VMM, add a second snapshot-capable backend (Firecracker / Cloud Hypervisor) behind the existing backend trait — libkrun stays default; the fork backend serves fan-out.

Maps forkd (template snapshot + UFFD CoW restore + diff-snapshot chaining) and clone (shadow-clone + KSM) onto a3s-box, reusing what already exists: overlay rootfs CoW, the warm pool, and — the key advantage — the guest vsock exec server as the post-fork re-injection hook (reidentify + exec-main handshake, since a forked child needs hostname/IP/box_id/command re-stamped).

Phased plan:

  • P0 (shipped, perf: CoW memory dedup (KSM) + boot-floor trim + reflink rootfs copy #16): KSM dedup + boot-floor trim + reflink — the memory-density half, on libkrun, no fork.
  • P1: per-image template pool + re-inject handshake — removes cold boot from the hot path, doable on current libkrun, de-risks P2.
  • P2: real snapshot-fork backend (Firecracker/CH + UFFD) — shared-RAM "100 ≈ 10" + ~10–150ms spawns.
  • P3: diff-snapshot chaining + balloon reclaim.

Recommendation: build P1 next (cheap, libkrun-native, validates demand); treat P2 as a funded epic.

Design-only proposal: a second snapshot-capable backend (Firecracker/CH) behind
the existing backend trait for high-fan-out agent sandboxes, since libkrun has no
snapshot. Maps forkd (snapshot+UFFD+diff-chaining) and clone (shadow-clone+KSM)
onto a3s-box, reusing the overlay rootfs CoW, warm pool, and — the key advantage —
the existing guest vsock exec server as the post-fork re-injection hook. Phased
plan: P0 KSM/boot-trim (done, #16) → P1 per-image template pool + re-inject
handshake (doable on libkrun, de-risks) → P2 real snapshot-fork backend → P3
diff-chaining + balloon.
@ZhiXiao-Lin ZhiXiao-Lin force-pushed the docs/cow-snapshot-fork-design branch from 56bc3b6 to d5b287e Compare June 11, 2026 08:31
@ZhiXiao-Lin ZhiXiao-Lin merged commit e3076ec into main Jun 11, 2026
7 checks passed
@ZhiXiao-Lin ZhiXiao-Lin deleted the docs/cow-snapshot-fork-design branch June 11, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant