fix: check F audits the candidate in a TestPyPI rehearsal, reports the released bootstrap as WARN

[Jammy2211]

Overview

PyAutoHeart is RED with two reasons sharing one cause: install verification FAILED (testpypi; checks F) and release validation FAILED (stage integrate). In run 35195111347 all 707 integrate scripts passed; only verify_install check F (the Colab gate) failed, on blackjax, corner, jax_zero_contour, optax, xxhash and zeus.

In a TestPyPI rehearsal (TARGET_VERSION set) check F seeds the venv pinned to the candidate, but the verbatim setup cell then runs pip install autonerves --no-deps unpinned, and the released autonerves.setup_colab.setup() installs the whole stack unpinned, downgrading everything to the PyPI release. The gate therefore audits the RELEASED bootstrap, never the candidate. The bootstrap fix (PyAutoNerves #168, #169, merged 2026-09-15) is untagged, PyPI still serves 2026.9.15.1, so: Heart RED → nightly rehearsal never publishes → PyPI stays broken → Heart RED. A chicken-and-egg that needs a human override every time.

Decision (2026-09-17): do not release; fix the gate so no authorisation is needed. Scope is PyAutoHeart only, no PyAutoNerves change.

Plan

• In a rehearsal (TARGET_VERSION set), check F gates on the candidate (FAIL → RED as today) and reports the released bootstrap's state as a WARN row that does not change the verdict (the nightly only publishes on GREEN, so YELLOW would still block the release that fixes it).
• The continuous PyPI-index path (no TARGET_VERSION) is unchanged: one verify, audits what a reader gets today.
• Readiness stays verdict-neutral on WARN; the dashboard shows "passed with warnings" with the WARN detail.
• Tests for the usage text, the sidecar writer, readiness and the dashboard.

Detailed implementation plan

Affected Repositories

• PyAutoHeart (primary)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoHeart	main	clean

Suggested branch: feature/colab-gate-candidate-audit

Implementation Steps

1. heart/checks/verify_install.sh check_f, only when TARGET_VERSION is set: after the verbatim setup-cell driver succeeds, (a) run colab_gate.py verify once as the RELEASED facet, advisory only, capturing its detail and the installed autonerves version from the report's packages block; (b) re-pin: pip install --no-deps "${PIP_INDEX_ARGS[@]}" autonerves==$T autofit==$T autoarray==$T autogalaxy==$T autolens==$T, then importlib.reload(autonerves.setup_colab) and pip install --no-deps of _PROJECTS["autolens"]["packages"] (mirrors _colab_setup, does not call setup() again); fold the COLAB_GATE_AUTONERVES_SRC overlay into this step as an extra overlay after the re-pin; (c) run colab_gate.py verify again into a second report as the CANDIDATE facet with today's FAIL semantics; (d) RESULTS rows: candidate FAIL → F|FAIL|<detail>; candidate pass + released failed → extra row F|WARN|released Colab bootstrap (autonerves=<ver>) broken for readers: <detail>; candidate passes, then the notebook cell and F|PASS as today; (e) the results table prints WARN rows, n_fail counts only FAIL so ready stays true on WARN, n_warn joins the Overall line, both gate reports go into the sidecar (VI_F_GATE_VERIFY candidate + new VI_F_GATE_VERIFY_RELEASED), header comment and usage text updated.
2. heart/readiness.py install block: no verdict change for WARN; a comment records that WARN is verdict-neutral by decision 2026-09-17 (a broken released bootstrap is not evidence against shipping; the release is the remedy).
3. heart/dashboard.py Install verify section: ready true + any check WARN → section WARN, summary passed with warnings (<index>; <letters>), WARN details as detail lines. FAIL / find-links unchanged.
4. heart/checks/colab_gate.py: no behaviour change expected (verify already takes --report-json).
5. Tests: tests/test_verify_install_script.py (usage text mentions WARN + rehearsal re-pin; sidecar snippet turns F|WARN|... into status WARN with ready true when no FAIL), tests/test_readiness.py (checks [F PASS, F WARN] ready true → no red, no yellow; ready false with FAIL still red), tests/test_dashboard.py (WARN row renders WARN; FAIL still FAIL).
6. Docs: verify_install.sh header, skills/verify_install/verify_install.md, docs/release_validation.md check F paragraph, health_agent/capabilities.yaml measures line.

Witness (red first; needs python3.12)

bash heart/checks/verify_install.sh F --testpypi --version 2026.9.17.1.dev77201 --report-json /tmp/vi.json

Before: the verify line shows autonerves=2026.9.15.1 and F FAIL. After: released facet → F WARN (6 missing), candidate facet → autonerves=2026.9.17.1.dev77201, F PASS, ready true. Negative control: no --version → one verify, unchanged. Then pytest tests/test_verify_install_script.py tests/test_readiness.py tests/test_dashboard.py.

Key Files

• heart/checks/verify_install.sh — check F driver, results table, sidecar writer
• heart/readiness.py — install-verification leg of the verdict
• heart/dashboard.py — Install verify section
• heart/checks/colab_gate.py — the gate's seed/verify commands

Original Prompt

Click to expand starting prompt

PyAutoMind/draft/bug/pyautoheart/colab_gate_rehearsal_audits_released_bootstrap.md (Mind commit 92e7c04) — "Check F audits the released Colab bootstrap in a TestPyPI rehearsal, not the candidate". Type: bug; Target: PyAutoHeart; Difficulty: moderate; Autonomy: supervised; Priority: high; Consequence: block. Related: PyAutoHeart#228, PyAutoNerves#168, PyAutoNerves#169.

[Jammy2211]

Shipped

Merged via #230 (head 4f7e04e, merge 08c74aa), Heart Tests green on py3.12 and py3.13.

What changed:

• verify_install.sh check F, with --version only: audits the released bootstrap first as an advisory facet (reported as an F|WARN|released Colab bootstrap (autonerves=<ver>) broken for readers: ... row that never moves ready), re-pins the venv to the candidate (five packages ==VERSION from the rehearsal index, setup_colab reloaded and its own package list reinstalled as _colab_setup does), then gates on the candidate with today's FAIL semantics. The continuous run without --version is unchanged. The COLAB_GATE_AUTONERVES_SRC overlay now applies after the re-pin, or before the single audit.
• Results table counts n_warn; the sidecar carries the released report as colab_gate.verify_released; readiness stays verdict-neutral on WARN; the dashboard renders "passed with warnings (; F)" with the detail.

Still to do by hand: the red-first witness on a machine with python3.12 (command in the PR body). The next nightly rehearsal carries PyAutoNerves #168/#169, so check F should pass on the candidate and Heart should go GREEN without a release authorisation.

---

Generated by Claude Code