perf(config): cut container-start latency — HZ=1000, voluntary preemption, PSI default-off (ABX-498) - #16
Merged
Merged
Conversation
Container start is a chain of jiffy-rounded scheduler waits (containerd poll, shim handshake completions, runc, dockerd↔containerd gRPC backoff). At HZ=100 each rounds up to 10 ms; at HZ=1000 to 1 ms. Measured docker-run end-to-end 496→378 ms and multi-step builds proportionally, matching the HZ=1000 guests Colima/OrbStack ship. NO_HZ_IDLE keeps the tickless-idle CPU target intact. Refs ABX-496/ABX-498.
The 6.18 bump switched arm64/x86 to lazy preemption on the premise of near-voluntary throughput. That premise fails for this guest's container-churn workload: lazy pulls in PREEMPT_RCU, so rcu_barrier() on the veth/netns teardown path (every container stop) waits for preemptible-RCU callbacks — ~74 ms/veth vs ~0 ms under voluntary — and same-core context-switch cost rises ~25%. Revert to voluntary; the build assertion now pins it so a future kernel that drops the choice fails loudly instead of silently re-selecting lazy. See ABX-498.
This was referenced Jul 24, 2026
Greptile SummaryUpdates both supported architecture configurations to reduce container-start latency:
Confidence Score: 5/5The PR appears safe to merge. No blocking failure remains within the scope of the available previous review threads.
What T-Rex did
Important Files Changed
Reviews (2): Last reviewed commit: "perf(config): disable PSI by default (ke..." | Re-trigger Greptile |
PSI's per-task-state accounting sits on the scheduler wakeup fast path. The container-build gRPC orchestration (buildkit/dockerd/containerd, thousands of wakeups per step) pays ~0.85 s per simple build with it on, and same-core context switch nearly doubles (ABX-498; psi=0 A/B). Keep CONFIG_PSI=y so the balloon pressure watch can still be enabled with psi=1, but default it off. Default-off leaves /proc/pressure uncreated, so the guest agent's balloon watch gets ENOENT opening it and falls back to meminfo/refault sampling — verified in-guest: the agent logs 'PSI unavailable (No such file or directory); staying on the sampling path' and the balloon keeps functioning (event-driven -> 1 s poll). Refs ABX-498.
AprilNEA
force-pushed
the
perf/container-start-latency
branch
from
July 24, 2026 04:17
29547df to
2de9578
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #14 (HZ=1000) and #15 (voluntary-preemption experiment). Consolidates the three validated container-start kernel-config wins from the ABX-498 investigation into one release.
Changes (both arches, identical)
CONFIG_HZPREEMPT_LAZYPREEMPT_VOLUNTARYCONFIG_PSI_DEFAULT_DISABLEDpsi=1re-enables)Build-time assertions updated in both the host and docker build paths: HZ_1000 and PSI_DEFAULT_DISABLED join the olddefconfig-survival
for symloop; the PREEMPT_LAZY conditional assertion is replaced by a hardPREEMPT_VOLUNTARYassertion (a future kernel dropping the voluntary choice now fails loudly instead of silently re-selecting lazy).Validation (ABX-498 four-kernel A/B, same host,
docker run --rm alpine true×10)With all three:
docker run496→246 ms, simple cold build 8.1→1.31 s (Colima 173 ms / 0.99 s). Guest-internal container start reaches parity; the residual is a moby exit-lock phase race (upstream, container-lifetime <30 ms only).Balloon safety (PSI default-off)
CONFIG_PSI=ystays, so the capability is compiled in. The guest agent's balloon pressure watch (PsiTrigger::open) already treats a failed/proc/pressure/memorywrite (EOPNOTSUPP when PSI is default-disabled) as "stay on the meminfo/refault sampling path" — a designed, tested degrade. The balloon keeps functioning; memory-pressure detection moves from event-driven to 1 s polling while shrunk.psi=1on the cmdline restores the trigger.Consumed by arcbox after a kernel release → boot-assets bundle bump →
assets.lock.