feat: route MGE-only interferometer fits through the W~ sparse operator (#575)

[Jammy2211]

Summary

MGE-only (func-list-only) interferometer inversions now use the W~ sparse operator whenever apply_sparse_operator() has been applied, instead of always taking the dense transform_mapping_matrix path. On the #308 breakdown this is 356–1746x faster on the A100 (dense 0.86–23.8 s vs W~ 1.9–13.5 ms). At alma on CPU under jit, the full fit takes 1.03 s sparse against 12.1 s dense, with identical log likelihood. The dense path also runs out of A100 memory at alma and above.

InversionInterferometerSparse already handled zero mappers, so the only routing change is removing the factory guard. The sparse data_vector can now take a dirty image for data that differs from the dataset the operator was built from (DatasetInterface.sparse_dirty_image). This is the seam fixed in the PyAutoGalaxy and PyAutoLens PRs: sparse fits with ordinary light profiles used to form D from the unsubtracted visibilities. Imaging routing is unchanged; its version of the seam is tracked separately in sparse_inversion_ignores_profile_subtracted_image.

Closes #575 together with the PyAutoGalaxy and PyAutoLens PRs (merge this one first).

API Changes

• Behaviour: with apply_sparse_operator(), an interferometer inversion made only of linear light profiles (e.g. an MGE) now builds InversionInterferometerSparse rather than InversionInterferometerMapping. Results are identical to float precision.
• New optional DatasetInterface(sparse_dirty_image=None), which the sparse interferometer data_vector uses when it is set.
• TransformerDFT.image_from(..., xp=) and transformer_util.image_direct_from(..., xp=np) now honour xp, so they work under jax.jit. This is an incidental fix; before this change they were NumPy-only and crashed inside jit.
  See full details below.

Test Plan

• test_autoarray full suite: 1706 passed
• New: test__interferometer_sparse_operator__func_list_only__identical_to_mapping, ..._sparse_dirty_image_override__used_by_data_vector (1e-10, with a control showing the cached image is wrong for subtracted data), func-list-only JAX-vs-NumPy, test__inversion_interferometer__via_linear_obj_func_list__sparse_operator, test__dft__image_from__jax_jit_matches_numpy
• Numeric: sma and alma MGE-only sparse vs dense log likelihood differ by 0 under jit (1.9e-9 NumPy at alma). The traced classes were confirmed as InversionInterferometerSparse and InversionInterferometerMapping.
• Workspace smoke: 24/24 targeted interferometer scripts pass (details in the Heart RED override section)

Full API Changes (for automation & release notes)

Added

• autoarray.DatasetInterface(..., sparse_dirty_image=None): the noise-weighted dirty image Re(Fᴴ W d) of the interface's data, for the sparse interferometer inversion.

Changed Signature

• autoarray.operators.transformer_util.image_direct_from(visibilities, grid_radians, uv_wavelengths, xp=np)

Changed Behaviour

• inversion_interferometer_from: dataset.sparse_operator is not None alone selects the sparse (W~) formalism, including func-list-only linear object lists. Previously func-list-only lists always used InversionInterferometerMapping.
• InversionInterferometerSparse.data_vector: uses dataset.sparse_dirty_image when it is not None, and otherwise sparse_operator.dirty_image.
• TransformerDFT.image_from: honours xp (JAX-traceable).

Migration

• None required.

Heart RED override (development only)

The human authorized the AUTONOMY.md "Human override for Heart RED (development only)" for this task (PyAutoArray#575, branch feature/interferometer-mge-w-tilde-route) on 2026-09-26. When asked how to proceed with the three library PRs, they chose "Override, open PRs". The override covers commit, push and opening pending-release PRs only. Merging needs a separate /prm with every required check green. This branch does not claim to fix Heart.

Exact RED reasons at ship time:

• release validation FAILED (stage integrate)
• workspace validation not passing (4 failed, cloud#35579888156: autolens notebooks/cluster/modeling.ipynb, autolens notebooks/weak/a2744.ipynb, autolens scripts/cluster/modeling.py, +1 more)
• manifest drift: hub organism blurb (organs present) — 7 mismatch(es) vs PyAutoMind/repos.yaml
• manifest drift: organism-map blocks (generated) — 1 mismatch(es) vs PyAutoMind/repos.yaml

Branch gates passed: 24/24 targeted interferometer smoke scripts pass on the branch (autolens_workspace 11, autogalaxy_workspace 5, autolens_workspace_test 5, autogalaxy_workspace_test 3: MGE, pixelization, linear light profiles, sparse-operator JAX assertions and likelihood pins). autolens_workspace_test/.../jax_likelihood/mge.py first failed identically on the branch and on the mains because the local dataset was stale (2026-08-06, older than the 09-18 simulator change); after regenerating it, the script passes on both. Unit suites: PyAutoArray 1706 passed, PyAutoGalaxy 1240 passed, PyAutoLens 757 passed + 1 xfailed. My own diff review is in-session only; the human reviews at merge.

🤖 Generated with Claude Code