Skip to content

feat: PyAutoEyes phase 4 — birth autofit_visualization + autocti_visualization #455

Description

@Jammy2211

Epic: pyautoeyes-birth, phase 4
Heart at the door (2026-09-29): YELLOW — "HowToGalaxy / HowToLens / PyAutoMemory: open PR 7–8d old" (continue; acknowledgement is a ship-time act).

Overview

Birth the last two <lib>_visualization project repos, PyAutoLabs/autofit_visualization (checkout fit/autofit_visualization) and PyAutoLabs/autocti_visualization (checkout cti/autocti_visualization), so the PyAutoEyes cross-project dashboard covers the whole stack. Both mirror the phase-3 autogalaxy_visualization layout: flat producers, tracked tiny datasets, all-true visualize config, render harness, tracked PNGs + GALLERY.md + gallery/viz_manifest.yaml, lint/render workflows. Both instances are then registered across the body map (Mind), Heart drift exclusion, Brain prose and the PyAutoEyes registry.yaml.

Decision: one task, two project repos (the Feature Agent suggested splitting; the organism ceremony is identical for both rows and would double). Built fit-first, cti-second, each with its own PR.

Plan

  • Push a LICENSE+README stub to each empty repo's main, clone at fit/ and cti/, build each on feature/eyes-fit-cti-instances from the galaxy template.
  • autofit_visualization: four flat producers (samples, model, ep, visualizer) rendering every PyAutoFit figure on gaussian_x1 with real cheap searches; render.yml listens for pyautofit-release.
  • autocti_visualization: two flat producers (dataset_1d, imaging_ci) rendering every PyAutoCTI figure via the Visualizer lifecycle plus direct aplt calls for figures the Visualizers never emit, on tiny simulated charge-injection data; workflows use the central Heart install-arcticpy action; render.yml listens for pyautocti-release.
  • Register both: Mind repos.yaml rows + Eyes role + ROUTING + epics + repos_sync --write; Heart excluded: lines; Brain prose; PyAutoEyes registry.yaml rows + tests + regenerated dashboard.
  • Ship library-first: Brain → autofit_visualization → autocti_visualization → Mind → Heart → Eyes (last: its check HEADs every PNG on both mains), plus map-block PRs (Nerves/Gut/Cortex, Scientist if changed). Merge stays human.

Corrections to the draft prompt (survey findings):

  1. PyAutoFit has no plotter-class family (NestPlotter/MCMCPlotter/… are gone): sampler plots are functions in autofit.plot; the only classes are af.ModelPlotter, af.EPPlotter and af.Visualizer (VisualizerExample under af.ex). The producer calls the functions directly.
  2. PyAutoCTI likewise has no *Plotter/MatPlot/Include classes; everything is aplt.* functions plus the fit-time VisualizerDataset1D / VisualizerImagingCI. Datasets are simulated in-repo (30×30 CI, 50-pixel 1D; <1 s) rather than from autocti_workspace/scripts/*/simulators (2000×100).
  3. Neither repo needs instruments/ (1D toy data / CCD layouts live in the simulators); the deviation from the lens/galaxy template is documented in each README.
  4. Nothing sends pyautofit-release/pyautocti-release yet (Mind draft draft/feature/pyautohands/release_fires_visualization_dispatch.md); render.yml runs by workflow_dispatch until that ships.

Human-only steps (surfaced at ship):

  • Grant PAT_PYAUTOLABS to both new repos (org-admin scope; otherwise eyes-refresh skips).
  • eyes-critique label on both repos (agent tries gh label create).
  • .github profile README patch from repos_sync.
Detailed implementation plan

Affected Repositories

  • autofit_visualization (primary; empty GitHub repo, cloned at fit/autofit_visualization)
  • autocti_visualization (empty GitHub repo, cloned at cti/autocti_visualization)
  • PyAutoEyes, PyAutoMind, PyAutoBrain, PyAutoHeart
  • Map-block PRs: PyAutoNerves, PyAutoGut, PyAutoCortex (PyAutoScientist if repos_sync changes it)

Branch Survey

Repository Current Branch Dirty?
fit/autofit_visualization (empty remote; to be cloned) —
cti/autocti_visualization (empty remote; to be cloned) —
organs/PyAutoEyes main @ origin clean (23 MB untracked render leftovers, human git clean -fdx)
organs/PyAutoMind main @ origin clean
organs/PyAutoBrain main @ origin clean
organs/PyAutoHeart main @ origin clean

worktree_check_conflict eyes-fit-cti-instances …: no conflict.

Suggested branch: feature/eyes-fit-cti-instances (all repos)
Worktree: ~/Code/PyAutoLabs-wt/eyes-fit-cti-instances (via /start_library)

Implementation Steps

A. fit/autofit_visualization (template galaxy/autogalaxy_visualization)

Copy verbatim: .claude/, AI_POLICY.md, LICENSE, config/general.yaml
(then extend, below), gallery/gallery_run.sh, scripts/misc/test/__init__.py.
Copy + substitute (table in survey): _viz_cli.py (af., autofit), activate.sh
(_libs = Nerves, Fit only; walks ../.. from fit/), ruff.toml,
CLAUDE.md, .gitignore, gallery/gallery_build.py (STACK=("autofit",),
VERSION_LINE_PREFIX="Rendered with PyAutoFit", autofit_version()),
scripts/misc/test/test_gallery_build.py (FAKE_STACK, assert string),
.github/workflows/lint.yml (checkout Nerves+Fit mains; pip ./PyAutoFit[optional]
so corner/anesthetic/dynesty/nautilus/emcee/zeus install), render.yml
(input autofit_version, types: [pyautofit-release], pip install "autofit[optional]==V"). No instruments/ (1D toy data; document the
deviation in README/AGENTS).

Config (config/): copy fit/autofit_workspace/config/ wholesale (non_linear,
priors, notation, logging, output, general, visualize) then set:
visualize/plots_search.yaml every key true incl. the mle: section;
output.yaml: model_figure: true; general.yaml
output.force_visualize_overwrite: true, workspace_version_check: false;
general.yaml visualize_ep_factor_searches: true.

Datasets (scripts/misc/simulators/gaussian.py): port the gaussian_x1 and
gaussian_x1_0/1/2 recipes from fit/autofit_workspace/scripts/simulators/simulators.py;
write dataset/example_1d/gaussian_x1{,_0,_1,_2}/{data,noise_map,model}.json
(tracked, tiny). _viz_cli.auto_simulate_if_missing guards it.

Producers (each: repo-root walk to ruff.toml, conf.instance.push(config, output_path=scripts/<d>/images), wipe scripts/<d>/images/visualization/,
scratch fits in gitignored output/visualization/<d>/, no test mode):

  • scripts/samples/visualization.py — one search per source subfolder:
    dynesty_static, nautilus, emcee, zeus, lbfgs, multistart_adam
    (JAX, use_jax=True). Run the fit on gaussian_x1 with
    af.ex.Analysis + af.Model(af.ex.Gaussian), then render into
    images/visualization/<source>/ by calling the aplt.* functions directly
    (path=, format="png"): both corners for nest+MCMC samples, the 6 MLE
    variants for lbfgs, figure_of_merit_vs_iteration for multistart. Direct
    calls rather than relying on plot_results so a swallowed corner exception
    surfaces as a missing file caught by --check. Expected ≈ 15 PNGs.
  • scripts/model/visualization.py — af.ModelPlotter(...).figure() for: single
    Gaussian, Collection of 2, model with a fixed + a linked prior, EP global prior
    model; each in detail="names" and detail="priors", plus collapse=True
    and show_fixed=False variants → sources gaussian, collection,
    linked, graph; ≈ 10 PNGs.
  • scripts/ep/visualization.py — port fit/autofit_workspace/scripts/features/expectation_propagation.py
    on gaussian_x1_0/1/2 (DynestyStatic(nlive=150, sample="rwalk", walks=10),
    EPOptimiser(max_steps=5, visualise_interval=1)); harvest graph.png,
    graph_factors.png, graph_model.png, graph_state.png, and
    EPPlotter.figure(kind=model|state, show_prior_factors=True); with
    visualize_ep_factor_searches: true also the per-factor data.png/model_fit.png
    under source factor_<i>. ≈ 12 PNGs, a few minutes runtime — note in README.
  • scripts/visualizer/visualization.py — one quick DynestyStatic fit with
    af.ex.Analysis (its Visualizer = VisualizerExample): copy image/data.png,
    image/model_fit.png, and model.png from the fit output into
    images/visualization/; ≈ 3 PNGs.
  • gallery_build.render_markdown intro sentence + titles for autofit.

B. cti/autocti_visualization

Same copy/substitute set with ac., autocti, STACK=("autocti","autoarray","autofit"),
"Rendered with PyAutoCTI", activate.sh _libs = Nerves, Fit, Array, CTI;
lint.yml checks out those four mains and runs
uses: PyAutoLabs/PyAutoHeart/.github/actions/install-arcticpy@main before the
pip loop; render.yml same action then pip install "autocti[optional]==V",
types: [pyautocti-release], input autocti_version. No instruments/;
instead ccd/ is not needed either — layouts live in the simulators.

Config: copy cti/autocti_workspace/config/ (general, logging, notation,
non_linear, priors, visualize); visualize/plots.yaml every boolean true
(incl. dataset.fpr_non_uniformity: true, combined_only: false);
plots_search.yaml true; general.yaml workspace_version_check: false.

Datasets (scripts/misc/simulators/{dataset_1d,imaging_ci}.py, tracked FITS,
tiny): dataset_1d/simple — Layout1D shape 50, region (5,25), 3 norms,
parallel traps ×2; imaging_ci/parallel_serial — 30×30, parallel ×2 + serial
×1 CTI, 3 norms, column_sigma non-uniform injection, a
SimulatorCosmicRayMap.defaults cosmic-ray map; imaging_ci/parallel — same
without serial (parallel-only region set). Write layout.json/cti.json
alongside so producers rebuild the true model.

Producers:

  • scripts/dataset_1d/visualization.py — build ac.Dataset1D list (one per
    norm), ac.AnalysisDataset1D + Clocker1D, paths = SimpleNamespace(image_path, output_path); call VisualizerDataset1D.visualize_before_fit,
    visualize_before_fit_combined, visualize(instance=true cti), visualize_combined
    into top level; then source direct/: every aplt.figure_fit_dataset_1d
    quantity × region × logy, subplot_dataset_1d_list, subplot_fit_dataset_1d_list.
    ≈ 35–50 PNGs.
  • scripts/imaging_ci/visualization.py — sources parallel_serial and
    parallel (Visualizer lifecycle each, via ac.AnalysisImagingCI and a model
    instance carrying .cti), plus direct/: figure_pre_cti_data_residual_map,
    subplot_noise_scaling_map_dict (dataset built with noise_scaling_map_dict),
    cosmic-ray 5-panel subplot_imaging_ci, figure_fit_ci_region for every
    quantity × region, the *_list variants. ≈ 60–90 PNGs. Check at render time
    that fpr_non_uniformity renders in subplot_dataset_region (survey caveat);
    if it raises, drop that region for the subplot and record it in README.
  • Fit figures use the true model, no search (FitImagingCI(dataset=masked, post_cti_data=clocker.add_cti(data=pre_cti_data, cti=cti))). No corner plots
    here (search plots are covered by the fit instance).

C. PyAutoEyes (organs/PyAutoEyes)

  • registry.yaml: append rows fit (repo autofit_visualization, path
    fit/autofit_visualization, library PyAutoFit, import autofit,
    dispatch_event: pyautofit-release) and cti (autocti_visualization,
    cti/…, PyAutoCTI, autocti, pyautocti-release); manifest/gallery/
    images_base_url per the galaxy row pattern. Order lens, galaxy, fit, cti.
  • tests/test_registry.py: test_the_committed_registry_has_the_fit_row,
    …_cti_row, order assert ["lens","galaxy","fit","cti"].
  • Prose in AGENTS.md, README.md, REFERENCE.md (four instances, both new
    dispatch events). bin/pyauto-eyes board (+--mind) then check — only
    green after both project mains carry the PNGs, so Eyes PR merges last.
  • Note: canonical checkout has 23 MB untracked dataset/ output/ scripts/
    leftovers (human TODO git clean -fdx); work happens in the task worktree.

D. PyAutoMind (organs/PyAutoMind)

  • repos.yaml: two category: project blocks after autogalaxy_visualization
    (roles: "Rendered PyAutoFit figures — every sampler, model-graph, expectation-
    propagation and Visualizer output on the gaussian_x1 example…"; "Rendered
    PyAutoCTI figures — every Dataset1D and ImagingCI visualizer output on small
    simulated charge-injection data…"); PyAutoEyes role → "(the four
    <lib>_visualization repos)"; PyAutoEyes row in the AGENTS route table
    already names both — extend to four.
  • ROUTING.md target list; epics.md phase-4 status line.
  • python3 scripts/repos_sync.py --write from the worktree, then --check.
    Trap (memory --writeSpill): it writes through symlinks into canonical
    Nerves/Gut/Cortex/Scientist/Hands/Heart AGENTS.md + root AGENTS.md — snapshot
    git status, save each spill as tmp/handover/map-block-<repo>.patch, revert
    canonical, keep root AGENTS.md (unversioned live file).
  • Move prompt to active/; register active.md (status: library-dev,
    worktree path, repos claimed: PyAutoEyes PyAutoMind PyAutoBrain PyAutoHeart
    autofit_visualization autocti_visualization).

E. PyAutoHeart — config/repos.yaml excluded: two lines

(- autofit_visualization # fit figure gallery project (PNGs + manifest tracked; re-rendered on release),
same for cti); map block via repos_sync.

F. PyAutoBrain — no conductor code (_eyes.py is repo-name-free; flat

layout keeps it working). Prose: agents/conductors/eyes/AGENTS.md,
skills/eyes/eyes.md, docs/organs/eyes.md, bin/clean_slate.sh comment,
AGENTS.md map block; grep -n autogalaxy_visualization config/ for any
policy row to mirror.

G. Human-only steps (surface at ship, on the issue)

  • Grant PAT_PYAUTOLABS to both new repos (org-admin scope; agent 403s).
  • eyes-critique label on both repos (agent tries gh label create).
  • .github profile README patch from repos_sync.
  • Merges (/prm), in the order in step 5.

Task / branch / worktree

  • Task eyes-fit-cti-instances; branch feature/eyes-fit-cti-instances in all
    six repos; worktree bundle ~/Code/PyAutoLabs-wt/eyes-fit-cti-instances/
    (via /start_library; the two new clones live at fit/… and cti/… and are
    added to the bundle). worktree_check_conflict: no conflict. All four organ
    checkouts on main, clean, at origin.
  • Env traps: root activate.sh PYTHONPATH points at the deleted
    eyes-galaxy-instance worktree (memory activatePP) — each repo's own
    activate.sh / PYTHONPATH=organs/PyAutoNerves:fit/PyAutoFit:array/PyAutoArray:cti/PyAutoCTI
    is used for renders; shell pins 1 thread (1thread) — export 8 for renders;
    PYAUTO_SKIP_WORKSPACE_VERSION_CHECK handled by config/general.yaml.
  • Delegation: three Opus workers in sequence-with-overlap — (1) fit repo,
    (2) cti repo, (3) organism registration (Eyes/Mind/Heart/Brain) once both
    manifests exist — each with a progress file under the scratchpad and a
    Monitor tail; ship via /ship_library per repo.

Verification

  1. Each project repo: ruff check . && ruff format --check .;
    bash gallery/gallery_run.sh --all renders every domain; python gallery/gallery_build.py --check green on the committed tree; pytest scripts/misc/test -q green; pyauto-brain eyes survey <repo> → no gaps, no
    orphans, no stale renders.
  2. organs/PyAutoEyes: pytest tests -q; bin/pyauto-eyes board --offline --from fit=../../fit/autofit_visualization --from cti=../../cti/autocti_visualization
    pre-merge; live bin/pyauto-eyes check green post-merge; dashboard shows fit
    and cti sections; pyauto-brain eyes survey --instance fit --instance cti.
  3. Mind repos_sync.py --check (expect only the pre-existing 28 hook-copy
    drifts + .github table, as in phase 3 — record as not-delivered if so);
    lifecycle.py check; Heart/Brain touched-file test suites.
  4. CI: lint green on both project PRs (arcticpy action on cti); Eyes lint +
    Pages Dashboard green after merge; post-merge gh workflow run render on
    each repo commits nothing new and (once the PAT is granted) fires
    eyes-refresh → Dashboard Refresh green, Pages shows four instances.

Key Files

  • galaxy/autogalaxy_visualization/ @ a0b0177 — template tree (_viz_cli.py, gallery/gallery_build.py, .github/workflows/{lint,render}.yml, scripts/<domain>/visualization.py)
  • fit/PyAutoFit/autofit/non_linear/plot/{samples,nest,mle}_plotters.py, autofit/model_figure/{plotter.py,ep/plotter.py}, autofit/example/visualize.py — the fit figure surface
  • fit/autofit_workspace/scripts/{plot/*.py,features/expectation_propagation.py,simulators/simulators.py}, config/ — recipes and config to port
  • cti/PyAutoCTI/autocti/plot/__init__.py, autocti/{dataset_1d,charge_injection}/{plot,model}/ — the cti figure surface and Visualizers
  • cti/autocti_workspace_test/scripts/plot/subplots.py, cti/autocti_workspace/config/visualize/plots.yaml — fit-without-search recipe and config keys
  • organs/PyAutoHeart/.github/actions/install-arcticpy — arcticpy for cti CI
  • organs/PyAutoEyes/registry.yaml, tests/test_registry.py; organs/PyAutoMind/repos.yaml, ROUTING.md, epics.md, scripts/repos_sync.py; organs/PyAutoHeart/config/repos.yaml; organs/PyAutoBrain/agents/conductors/eyes/AGENTS.md, skills/eyes/eyes.md, docs/organs/eyes.md, bin/clean_slate.sh

Original Prompt

Click to expand starting prompt

PyAutoEyes phase 4 — birth autofit_visualization + autocti_visualization

Type: feature
Target: autofit_visualization
Repos:

  • autofit_visualization
  • autocti_visualization
  • PyAutoEyes
  • PyAutoMind
  • PyAutoBrain
  • PyAutoHeart
    Themes:
  • visualization
  • infrastructure
    Difficulty: large
    Autonomy: supervised
    Priority: normal
    Lane: local-dev
    Status: draft
    Consequence: judge
    Witness: the fit and cti project-repo surveys report no gaps/orphans; each repo's gallery_build.py --check green with a tracked manifest; pyauto-eyes check green over all four registry rows; the PyAutoEyes dashboard shows fit and cti sections; repos_sync.py --check clean
    Review-minutes: 15
    Epic: pyautoeyes-birth
    Phase: 4
    Filed: 2026-09-25

Blocked on: human repo creation only — phase 3 COMPLETE 2026-09-29 (PyAutoMind#452; record complete/2026/09/eyes-galaxy-instance.md); still needs gh repo create PyAutoLabs/autofit_visualization --public and gh repo create PyAutoLabs/autocti_visualization --public.

Task

Two project repos, same steps as phase 3:

  • fit/autofit_visualization — ModelPlotter, EPPlotter, VisualizerExample on
    gaussian_x1.
  • cti/autocti_visualization — Dataset1D + ImagingCI visualizers, simulated
    from autocti_workspace/scripts/*/simulators.

Each: flat producers, all-true plots.yaml, simulated datasets, render
harness, tracked PNGs + GALLERY.md + tracked manifest, lint.yml +
render.yml on its library's release dispatch (pyautofit-release,
pyautocti-release) firing eyes-refresh; Mind/Brain/Heart registration as
phase 1a; PyAutoEyes registry.yaml rows.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions