feat(interferometer): pass data=None on the sparse precomputed-data-term path (Discussion #13 lens parity)

[Jammy2211]

Overview

PyAutoLens parity for phase 1 of https://github.com/orgs/PyAutoLabs/discussions/13 (PyAutoArray#589 + PyAutoGalaxy#637, merged 2026-09-30). On the sparse interferometer path a fit with no non-linear light profile can build its inversion with data=None, so fast_chi_squared reads the cached sparse_operator.data_term and the two per-call N_vis allocations (profile_visibilities, profile_subtracted_visibilities) are never formed. autolens.interferometer.FitInterferometer.tracer_to_inversion still passes data=self.profile_subtracted_visibilities unconditionally, so lens fits — the main consumer — still pay them. This task mirrors the autogalaxy change in autolens.

Plan

• Gate tracer_to_inversion on autogalaxy's uses_precomputed_data_term_from so sparse pixelization-only lens fits pass data=None into the inversion.
• Add FitInterferometer.inversion_with_data (shallow copy carrying the fitted data) for output/plot paths; switch the lens visualizer's inversion subplot to it.
• Make profile_visibilities / profile_subtracted_visibilities cached properties, matching autogalaxy, so "never evaluated" is testable.
• Port autogalaxy's phase-1 tests with a lens-mass + pixelization-only-source tracer; add a bit-equality control against the ungated path.
• Verify nothing in autolens reads inversion.dataset.data; ship as a pending-release PR on PyAutoLens.

Detailed implementation plan

Affected Repositories

• PyAutoLens (primary, only)

Branch Survey

Repository	Current Branch	Dirty?
./PyAutoLens	main (1 behind origin)	clean

Worktree guard: PyAutoLens is also claimed by workspace-config-cleanup (#441); its PyAutoLens PR #751 is already merged (test_autolens/model_figure/* only) and the claim is held for the release gate. Disjoint files; parallel claim approved with the plan on 2026-09-30.

Suggested branch: feature/interferometer-sparse-precomputed-data-term

Implementation Steps

1. autolens/interferometer/fit_interferometer.py: import uses_precomputed_data_term_from from autogalaxy.interferometer.fit_interferometer (next to the existing _has_light_profile_non_linear, sparse_dirty_image_from imports) and import copy. Add @property _uses_precomputed_data_term → uses_precomputed_data_term_from(dataset=self.dataset, galaxies=self.tracer.galaxies, data=self.data, noise_map=self.noise_map).
2. In tracer_to_inversion: data = None if self._uses_precomputed_data_term else self.profile_subtracted_visibilities; the rest of the aa.DatasetInterface(...) call (incl. sparse_dirty_image_from) unchanged.
3. Add @property inversion_with_data mirroring autogalaxy (return self.inversion when it is None or its dataset.data is not None; else evaluate inversion.reconstruction, copy.copy the dataset with data = self.data, copy.copy the inversion with that dataset).
4. profile_visibilities and profile_subtracted_visibilities: @property → the file's cached_property (as profile_image already is). Confirm test__profile_visibilities__linear_light_only__zeros_without_fourier_transform still passes.
5. autolens/interferometer/model/visualizer.py ~L157-160: plotter.inversion(inversion=fit.inversion_with_data).
6. Verify only: autolens/lens/to_inversion.py ~L197-204 propagates data=None into ag.GalaxiesToInversion (already exercised by PyAutoGalaxy#637); no .data read on the preloads path.
7. Tests in test_autolens/interferometer/test_fit_interferometer.py, sparse dataset built inline via interferometer_7.apply_sparse_operator(use_jax=False), tracer = mass-only lens + pixelized source (rectangular mesh, constant regularization):
   • ..._sparse_operator__pixelization_only__data_term_scalar_matches_dense: predicate true, fit.inversion.dataset.data is None, log_evidence == dense at rel 1e-8, inversion_with_data.dataset.data is fit.data sharing reconstruction, non-sparse fit → inversion_with_data is inversion.
   • Spy block: monkeypatch transformer.visibilities_from and aa.Visibilities.zeros; both call lists empty after figure_of_merit; neither profile property in fit.__dict__; profile_visibilities still zeros when accessed.
   • ..._light_profile__unchanged_vs_data_passed: lens light Sersic → predicate false, log_evidence equal to before.
   • ..._pixelization_only__jax_jit_matches_numpy: jitted vs numpy figure of merit.
   • Control: predicate forced off → log_evidence bit-equal to the gated value.
8. Witness script (scratchpad): 1e5-vis NUFFT sparse dataset; gated vs ungated log_evidence bit-equal; tracemalloc peak per figure_of_merit flat in N_vis (1e5 vs 4e5); numbers into the PR body.
9. /ship_library: pytest test_autolens green; label pending-release; Heart YELLOW ack.

Key Files

• autolens/interferometer/fit_interferometer.py — gate, inversion_with_data, cached properties
• autolens/interferometer/model/visualizer.py — inversion subplot uses inversion_with_data
• test_autolens/interferometer/test_fit_interferometer.py — ported tests + control
• autogalaxy/interferometer/fit_interferometer.py (reference, merged chore: rename PyAutoConf → PyAutoNerves in docs/prose #637) — uses_precomputed_data_term_from, inversion_with_data

Original Prompt

Click to expand starting prompt

PyAutoLens parity: precomputed data term on the sparse interferometer path (streaming phase 1)

Type: feature
Target: PyAutoLens
Repos:

• PyAutoLens
  Themes:
• interferometer
• sparse-operator
• memory
  Difficulty: small
  Autonomy: supervised
  Priority: medium
  Status: draft
  Consequence: glance
  Witness: al.FitInterferometer on a sparse-operator dataset with a pixelization-only source (no non-linear light profiles) builds its inversion with data=None, never evaluates profile_visibilities / profile_subtracted_visibilities during figure_of_merit (spy on aa.Visibilities.zeros and transformer.visibilities_from), log_evidence is bit-equal to the data-passed path, and test_autolens/interferometer is green.
  Review-minutes: 3
  Unattended: ready
  Parent: complete/2026/09/interferometer-streaming-visibilities.md

Source: GitHub Discussion https://github.com/orgs/PyAutoLabs/discussions/13 (HRSAstro,
"Streaming visibilities for memory efficiency"); phase 1 = PyAutoArray#588 (PRs on
PyAutoArray + PyAutoGalaxy, branch feature/interferometer-streaming-visibilities).

Why

Phase 1 made the pixelization-only sparse path skip the two N_vis allocations per
likelihood call in PyAutoGalaxy: fast_chi_squared / noise_normalization read the
scalars carried by InterferometerSparseOperator, and
autogalaxy/interferometer/fit_interferometer.py passes data=None into
DatasetInterface when uses_precomputed_data_term_from(dataset, galaxies, data, noise_map) holds. PyAutoLens has its own FitInterferometer, whose
tracer_to_inversion (autolens/interferometer/fit_interferometer.py ~L155) still passes
data=self.profile_subtracted_visibilities unconditionally — so lens fits, the main
consumer, still allocate Visibilities.zeros + data - profile_visibilities over N_vis
on every call.

What

• In autolens/interferometer/fit_interferometer.py tracer_to_inversion, reuse
  ag.interferometer.fit_interferometer.uses_precomputed_data_term_from(self.dataset, self.tracer.galaxies, self.data, self.noise_map) (verify the import path on the
  shipped autogalaxy) and pass data=None when it holds; keep sparse_dirty_image
  as is.
• Add inversion_with_data mirroring autogalaxy's (an inversion rebuilt with the real
  profile_subtracted_visibilities, for consumers that need inversion.data).
• Switch autolens/interferometer/model/visualizer.py (~L139-160) to
  fit.inversion_with_data where it plots inversion quantities that read data.
• Tests in test_autolens/interferometer/test_fit_interferometer.py mirroring
  autogalaxy's phase-1 additions: the data=None gate, the spy, bit-equal log_evidence,
  and the light-profile case still passing data.
• Check autolens/aggregator/ and autolens/plot/ (and al.agg fit reconstruction) for
  inversion.data / inversion.data_vector reads that would break on data=None.

Blocked on the phase-1 PyAutoArray + PyAutoGalaxy PRs merging (library-first). (merged 2026-09-30)

🤖 Generated with Claude Code

[Jammy2211]

Library PR created

• PR: feat(interferometer): pass data=None on the sparse precomputed-data-term path (Discussion #13 lens parity) #757
• test_autolens/: 766 passed, 1 xfailed
• labelled pending-release; workspace impact: none (option iii — no workspace script reads fit.inversion.dataset.data / data_subtracted_dict)
• Next: /prm once CI is green

🤖 Generated with Claude Code

[Jammy2211]

Shipped

PyAutoLens parity for phase 1 of https://github.com/orgs/PyAutoLabs/discussions/13 is merged: #757 (merge 092897e). Sparse pixelization-only lens fits now build their inversion with data=None and allocate no visibility-sized arrays per likelihood call; fit.inversion_with_data carries the fitted visibilities for output paths. Labelled pending-release.

Independent review findings were addressed in PyAutoArray#591 / autogalaxy_workspace#253 / autolens_workspace#581 (also merged).

🤖 Generated with Claude Code