test: regression-test the Colab sampler install (dynesty/emcee)

[Jammy2211]

Overview

A HowToFit chapter 1 read-through on 2026-09-14 hit ModuleNotFoundError: No module named 'dynesty' (tutorial 4) and 'emcee' (tutorials 5 and 6) on Colab: autonerves/setup_colab.py installed its package list with pip install *packages --no-deps, and neither sampler was named in _SHARED_EXTRAS, so neither could arrive transitively via autofit. Every Colab notebook across all six _PROJECTS that runs a Dynesty or Emcee search died at the fit.

The code half already shipped the same day in #164 (merged, 8336939), which added emcee>=3.1.6 and dynesty==2.1.5 to _SHARED_EXTRAS with the specifiers autofit declares, plus a comment recording why a --no-deps install has to name its siblings' real dependencies. What did not ship is the regression test. #164 was verified only against the existing suite, which asserts nothing about the contents of _SHARED_EXTRAS — test_setup_colab.py checks only spec["packages"][0] == "autonerves". Nothing currently stops a future edit dropping a sampler again, which is exactly how this bug arose.

Note: Colab notebooks pip install autonerves fresh on every run, so #164 reaches users only once a new autonerves is released to PyPI.

Plan

• Cover the shipped fix with a regression test: every _PROJECTS entry must resolve to a packages list containing dynesty, emcee and nautilus-sampler — the three samplers PyAutoFit exposes as af.DynestyStatic, af.Emcee and af.Nautilus. Assert on sampler names, not exact pins, so a version bump does not break the test.
• Because the install is --no-deps, confirm the two newly-named samplers' own runtime requirements are already present in a stock Colab image (dynesty needs numpy + scipy, emcee needs numpy) — otherwise they install but still fail at import.
• Consider (judgement call, not a given) lifting the three samplers out of _SHARED_EXTRAS into their own _SAMPLERS list: a missing sampler is a hard ModuleNotFoundError at fit time, not a degraded experience, so the grouping arguably should say so.
• Keep in view, but out of scope: a broader audit of autofit's declared dependencies against a stock Colab image would say whether others are waiting behind the same --no-deps trap.

Detailed implementation plan

Affected Repositories

• PyAutoNerves (primary)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoNerves	main	clean

Suggested branch: feature/autonerves-colab-sampler-deps

Current state (post-#164)

autonerves/setup_colab.py _SHARED_EXTRAS (~line 36) now reads:

_SHARED_EXTRAS = [
    "pyvis==0.3.2",
    "dill==0.4.0",
    "jaxnnls",
    "nautilus-sampler==1.0.4",
    "timeout_decorator==0.5.0",
    "anesthetic==2.8.14",
    "emcee>=3.1.6",
    "dynesty==2.1.5",
]

The install at ~line 277 is still subprocess.check_call([sys.executable, "-m", "pip", "install", *packages, "--no-deps"]), so the --no-deps constraint that caused the bug is unchanged and is the thing the test defends.

Implementation Steps

1. Add the regression test in test_autonerves/test_setup_colab.py, in class TestRegistry alongside test_all_projects_have_required_fields. For every entry of setup_colab._PROJECTS ("autofit", "autogalaxy", "autolens", "howtofit", "howtogalaxy", "howtolens"), assert the resolved packages list contains dynesty, emcee and nautilus-sampler. Match on the requirement name (split each entry on the version specifier — ==, >=, <, ~= — before comparing) so a pin bump does not break the test, and fail with the project name in the message.
2. Confirm the --no-deps runtime requirements hold. dynesty needs numpy + scipy; emcee needs numpy. Both ship in a stock Colab image, so no further entries are required — record that check in the PR so the next reader does not have to redo it.
3. Weigh the _SAMPLERS split. If taken, move nautilus-sampler, emcee and dynesty into a _SAMPLERS list and build _SHARED_EXTRAS = _NOTEBOOK_EXTRAS + _SAMPLERS (or splice at the _PROJECTS stacks), keeping the installed set byte-identical. The test from step 1 must pass unchanged either way — it asserts on the resolved packages, not on which list a name lives in. If not taken, say why in the PR; the test is the part that carries the value.
4. Run the suite (python3 -m pytest test_autonerves -q); it was 183 passed at fix: install emcee and dynesty in the Colab bootstrap #164.

Key Files

• autonerves/setup_colab.py — _SHARED_EXTRAS (~line 36), _PROJECTS registry (~line 52), the --no-deps install (~line 277).
• test_autonerves/test_setup_colab.py — class TestRegistry (~line 75) is where the new test belongs.

Release note

The fix only reaches users after an autonerves PyPI release: Colab notebooks pip install autonerves fresh on every run.

Original Prompt

Click to expand starting prompt

Regression-test the Colab sampler install (code fix shipped in PyAutoNerves#164)

Type: test
Target: PyAutoNerves
Repos:

• PyAutoNerves
  Difficulty: trivial
  Autonomy: safe
  Priority: normal
  Status: formalised
  Consequence: notify
  Witness: a regression test in PyAutoNerves asserting that every entry of _PROJECTS resolves to a packages list containing dynesty and emcee alongside nautilus-sampler — i.e. that all three samplers PyAutoFit exposes as af.DynestyStatic, af.Emcee and af.Nautilus are installed by the Colab bootstrap.
  Review-minutes: 10
  Unattended: ready
  Filed: 2026-09-14

The code fix already shipped — this is the missing test

This prompt was filed 2026-09-14 from review feedback on HowToFit chapter 1
tutorials 4/5/6. The code half was fixed independently the same day in
#164 (merged), which added emcee>=3.1.6 and
dynesty==2.1.5 to _SHARED_EXTRAS with the specifiers autofit declares, plus
a comment recording why a --no-deps install has to name its siblings' real
dependencies. See complete/2026/09/howtofit-tutorials-1-3.md.

What did not ship is the regression test. #164 was verified only against the
existing suite (183 passed), which asserts nothing about the contents of
_SHARED_EXTRAS — test_setup_colab.py checks only packages[0] == "autonerves".
Nothing currently stops a future edit dropping a sampler again, which is exactly
how this bug arose.

Remaining work

1. Add the test the Witness above describes: for every _PROJECTS entry, the
   resolved packages list contains dynesty, emcee and nautilus-sampler.
   Assert on the sampler names, not exact pins, so a version bump does not
   break it.
2. Consider whether the three samplers belong in their own _SAMPLERS list
   rather than inside _SHARED_EXTRAS. A missing sampler is a hard
   ModuleNotFoundError at fit time, not a degraded experience, so the grouping
   arguably should say so. This is a judgement call, not a given — the test in
   step 1 is the part that carries the value.

Still true, and worth keeping in view

The install runs with --no-deps, so any real autofit dependency that a
stock Colab image does not ship must be named in the list explicitly or it never
arrives. emcee and dynesty were the two that bit; a broader audit of
autofit's dependencies against a stock Colab image would say whether others
are waiting (corner, typing-inspect, gprof2dot, numpydoc, xxhash,
astunparse, array_api_compat, optax are all declared and none is named in
_SHARED_EXTRAS). That audit is not required by this prompt but is the obvious
next question.

Colab notebooks pip install autonerves fresh on every run, so #164 reaches users
only once a new autonerves is released to PyPI.

[Jammy2211]

Shipped — #166 merged (65e76e5).

Adds TestRegistry.test_every_project_installs_every_sampler: for every _PROJECTS entry the resolved packages list must contain dynesty, emcee and nautilus-sampler, matched on requirement name with the version specifier split off so a future re-pin cannot break it.

• Tests: 184 passed (183 before), 21 in test_setup_colab.py. CI green on unittest 3.12, 3.13 and unittest-nojax.
• Control test confirms the assertion is not a tautology: dropping each of the three samplers makes it fail; substituting dynesty==9.9.9 and a bare unpinned emcee keeps it passing.
• autonerves/setup_colab.py untouched — byte-identical to main. The _SAMPLERS regrouping floated in the plan was deliberately not taken.

The user-visible bug is still live. #164 fixed the package list, but the released autonerves 2026.9.14.1 was tagged from 0e7163b (2026-09-08) and uploaded at 09:25 on 2026-09-14 — about nine hours before #164 merged at 18:48. Colab pip install autonerves fresh on every run, so it still gets the sampler-less list and HowToFit chapter 1 tutorials 4, 5 and 6 still die with ModuleNotFoundError. Only a new autonerves PyPI release clears this.

Flagged follow-up, not fixed here: setup_colab.py pins nautilus-sampler==1.0.4 while PyAutoFit's pyproject.toml declares 1.0.5.