Sparse interferometer inversion to combine linear function lists with a Mapper

[HRSAstro]

Problem

On the imaging/optical side, InversionImagingSparse supports an analytic linear component (an AbstractLinearObjFuncList, e.g. linear light profiles) fitted simultaneously with a pixelized Mapper. On the interferometer side, InversionInterferometerSparse does not:

ImagingSparseOperator provides curvature_matrix_diag_from, curvature_matrix_off_diag_from, curvature_matrix_off_diag_func_list_from

InterferometerSparseOperator only provides curvature_matrix_diag_from

AbstractLinearObjFuncList does not appear in interferometer/sparse.py - its curvature_matrix returns curvature_matrix_diag, which is built from a single mapper.

Proposed Solution

The interferometer equivalents of the two missing operator methods — principally curvature_matrix_off_diag_func_list_from — plus the dispatch in InversionInterferometerSparse.curvature_matrix that uses them.

Alternatives Considered

None unless I have missed an existing route to this.

[Jammy2211]

Hi @HRSAstro — thanks for the report, and for pinning down exactly which operator methods are missing. I'm Jammy's CLI assistant; I'll post back shortly with a plan.

[Jammy2211]

Here's what I'm planning to do — let me know if anything looks off.

You've read the code correctly: InversionInterferometerSparse.curvature_matrix only ever returns the single-mapper diagonal block, and InterferometerSparseOperator has no off-diagonal methods. It's actually slightly worse than "unsupported" — the factory only falls back to the dense path when every linear object is a func list, so a mixed [linear light profiles, Mapper] list with a sparse operator attached is routed to the sparse class and the linear-function terms are silently dropped from F. The fix is to give the interferometer sparse path first-class parity with the imaging one rather than route around it.

Plan

• Add curvature_matrix_off_diag_from (mapper–mapper) and curvature_matrix_off_diag_func_list_from (mapper–linear-function) to InterferometerSparseOperator, mirroring the imaging operator. The interferometer version is simpler than imaging: the linear function's real-space column is scattered onto the FFT extent grid and pushed through the existing apply_operator (W~), then gathered at the mapper's triplet rows — no PSF adjoint step.
• Add the func-list dispatch to InversionInterferometerSparse.curvature_matrix and data_vector, following the imaging structure: mapper diagonal blocks via the sparse operator, mapper–func off-diagonals via the new method, func–func block via the dense transformed mapping matrices (small, n_funcs × n_funcs), then mirror and apply the no-regularization diagonal add.
• Make the factory fail loudly rather than silently for any combination the sparse path can't represent (e.g. a func list with a visibility-space operated_mapping_matrix_override, which has no real-space column to scatter) until that case is covered too.
• Tests: for a small interferometer dataset with linear light profiles + a mapper, assert the sparse curvature_matrix, data_vector, reconstruction and log_evidence match the dense InversionInterferometerMapping path to tolerance; plus a regression test that a mixed list no longer routes to a func-list-blind inversion.

Detailed implementation plan

Affected repositories

• PyAutoArray (primary)

Work classification

Library

Branch survey

Repository	Current Branch	Dirty?
./PyAutoArray	main	clean

Suggested branch: feature/interferometer-sparse-func-list
Worktree root: ~/Code/PyAutoLabs-wt/interferometer-sparse-func-list/ (created later by /start_library)

Implementation steps

1. autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py — add InterferometerSparseOperator.curvature_matrix_off_diag_from(rows0, cols0, vals0, rows1, cols1, vals1, *, S0, S1) (block-batched A0ᵀ W~ A1, same fori_loop shape as curvature_matrix_diag_from) and curvature_matrix_off_diag_func_list_from(curvature_weights, extent_index_for_masked_pixel, rows, cols, vals, *, S) (scatter slim → extent, apply_operator, gather at rows, segment_sum over cols → (S, n_funcs)).
2. autoarray/inversion/inversion/interferometer/sparse.py — import AbstractLinearObjFuncList; factor the existing triplet build into a helper used per mapper; add _curvature_matrix_mapper_diag, _curvature_matrix_off_diag_from, _curvature_matrix_multi_mapper, _curvature_matrix_func_list_and_mapper and the curvature_matrix dispatch (has(func list) → func path; one mapper → diag; else multi-mapper), then curvature_matrix_mirrored_from + curvature_matrix_with_added_to_diag_from; add the matching data_vector dispatch (mapper blocks via Aᵀ · dirty_image, func-list blocks via the dense transformed column against the weighted visibilities, as InversionInterferometerMapping does).
3. autoarray/inversion/inversion/factory.py — interferometer branch: raise InversionException for a sparse-routed linear object the sparse path cannot represent, instead of silently proceeding.
4. test_autoarray/inversion/inversion/interferometer/ — parity tests vs InversionInterferometerMapping for (a) one mapper (unchanged baseline), (b) func list + mapper, (c) two mappers; factory regression test; operator unit tests for the two new methods against a dense A0ᵀ W~ A1 built explicitly.
5. Run the interferometer inversion + factory test modules and the autolens_workspace_test interferometer smoke subset; confirm the existing single-mapper sparse likelihood is bit-identical before/after.

Key files

• autoarray/inversion/inversion/interferometer/inversion_interferometer_util.py — new operator methods
• autoarray/inversion/inversion/interferometer/sparse.py — func-list-aware curvature_matrix / data_vector dispatch
• autoarray/inversion/inversion/factory.py — loud failure for unrepresentable sparse routing
• autoarray/inversion/inversion/imaging/sparse.py — reference implementation (read-only)

I'll post back when the work is in progress and again when there's a PR.

[Jammy2211]

@HRSAstro the implementation is up as #500. Quick summary: InterferometerSparseOperator gained the mapper–mapper off-diagonal, mapper–function and function–function block methods, and InversionInterferometerSparse.curvature_matrix now assembles the full matrix for any mix of linear function lists and mappers.

One thing worth knowing from the derivation: because the interferometer operator W~ = Re(Fᴴ W F) already carries the inverse-variance weighting, the linear-function columns go in un-weighted — unlike the imaging operator, which is split as Hᵀ N⁻¹ H. It matches the dense InversionInterferometerMapping path to ~1e-16 on the test cases.

I'll post again once it's merged and heading for a release.

[Jammy2211]

Shipped

@HRSAstro this is merged — thanks again for the precise report.

• Sparse interferometer inversion: support linear func lists and multiple mappers #500: InversionInterferometerSparse now assembles the full curvature matrix for any mix of linear function lists (e.g. linear light profiles) and mappers, via three new InterferometerSparseOperator block methods (mapper–mapper off-diagonal, mapper–function, function–function). The single-mapper path you were already using is bit-identical to before.
• Add FitInterferometer sparse-vs-mapping parity assertion (PyAutoArray#499) autolens_workspace_test#283 and Add FitImaging / FitInterferometer sparse-vs-mapping parity assertions (PyAutoArray#499) autogalaxy_workspace_test#115: sparse-vs-mapping parity assertions for FitInterferometer (and FitImaging on the autogalaxy side) now run in the smoke suite, so this can't silently regress.

It'll be in the next PyPI release; until then it's on main if you install from source. One thing to be aware of if you build your own noise maps: the sparse W~ reduction assumes each visibility has equal real and imaginary sigma (which the simulator and real datasets satisfy) — unequal sigmas degrade agreement with the dense path on the sparse route generally, not just for the new blocks.