Skip to content

flaky: controller pid protocol times out in test_pi_process_tree.py on windows-latest #535

Description

@SUaDtL

The flake

test_pi_process_tree.py failed on windows-latest · Pi 0.80.10:

raise AssertionError("controller pid protocol timed out") from error
AssertionError: controller pid protocol timed out
{"piVersion": "0.80.10", "blocking": true, "result": "failed", "step": 13, "command": "test_pi_process_tree.py"}

Observed on main @ a02ee7d1, run 30365732056, 2026-07-28. The immediately following run on main (8c7600a6, 30366639611) passed with no change to any ca-pi file, so it is intermittent rather than a regression.

Found while gathering AC-4 evidence for #504.

Why this is NOT #504

Worth separating explicitly, because the two share a job and the obvious reading is wrong.

In the very run that failed here, #504's subject passed:

✓ launches and disposes a real Git Bash background job  4567ms

Different test, different module, different symptom. #504 was a launch that resolved without a handle; this is a controller handshake that never completes. Filed separately so neither investigation inherits the other's lead — the same reasoning that split #504 from #495.

Frequency

Across every CI run since 2026-07-26 (59 runs, 30 reaching the windows-latest ca-pi Adapter contract job): 1 failure, 29 successes. So roughly 1 in 30 on that job, on current evidence.

Why it matters

The step is marked "blocking": true, and [GATE ] | [REPO] | Merge readiness aggregates this job — it went red in the same run. The release preflight requires that gate green for the exact commit being tagged (#385), so a flake here blocks the release lane for that SHA. It also trains readers to re-run a red gate without reading it, which is the habit the gate exists to prevent.

Acceptance criteria

  • AC-1: the cause is identified — specifically whether the controller pid handshake can time out on a successful start (a protocol/ordering race) or only when the controller genuinely failed to come up.
  • AC-2: if the handshake can time out while the controller is healthy, the wait is made deterministic rather than the deadline merely raised. Raising a timeout to hide a race is explicitly not a fix here.
  • AC-3: the failure names what it observed — the controller's state, pid, and elapsed time — so the next occurrence is diagnosable from the log rather than by re-running. test(farm): make a flaked artifact-publication run name its own escalation (#515) #518 and fix(pi): a failed background-job spawn now says why (#504) #508 are the precedent for making a flake self-describing before chasing it.
  • AC-4: the test passes across repeated windows-latest runs.

Not in scope

Found while closing #504's AC-4. Deliberately not fixed there: that issue's remaining criterion was evidentiary, and this needs its own diagnosis.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions