Guard apply_sparse_operator against unequal real/imag noise sigma

[Jammy2211]

Summary

Interferometer.apply_sparse_operator built its W~ = Re(FᴴWF) precision operator from the real-part noise sigma alone, a reduction that is exact only when every visibility has sigma_real == sigma_imag. With unequal sigmas the sparse InversionInterferometerSparse curvature matrix silently disagreed with the dense InversionInterferometerMapping path (5e-16 relative with equal sigmas, 5e-10 to 3e-2 with unequal, geometry dependent) — latent because SimulatorInterferometer and real datasets satisfy the equality, but unguarded. Found while shipping #499 / #500. The method now raises a DatasetException naming the cause and the dense-path workaround, and the precondition is documented on apply_sparse_operator and InterferometerSparseOperator. Closes #502.

API Changes

Interferometer.apply_sparse_operator now raises DatasetException when any visibility has unequal real/imag noise sigma (previously produced a silently wrong sparse curvature matrix). No signature changes.
See full details below.

Test Plan

• pytest test_autoarray — 1290 passed, 81 warnings in 55s (no existing test used unequal real/imag sigma with apply_sparse_operator)
• new tests: unequal sigma raises; equal non-uniform sigma passes

Assessment: general two-operator extension

Writing F_ki = cos(φ_ki) − i sin(φ_ki), the exact curvature for unequal sigmas is
C_ij = Σ_k [ Re(F_ki)Re(F_kj)/σr_k² + Im(F_ki)Im(F_kj)/σi_k² ], which expands to
Σ_k { ½(wr_k+wi_k)·cos(φ_ki − φ_kj) + ½(wr_k−wi_k)·cos(φ_ki + φ_kj) }.
Only the first term is translation-invariant, i.e. a function of the pixel offset — that is exactly today's kernel, reweighted by (wr+wi)/2 (and when wr == wi the second term vanishes and it collapses to the current preload, which is the consistency check). The second term depends on the pixel sum (y_i+y_j, x_i+x_j), so it needs a genuinely second kernel on a sum-index grid weighted by (wr−wi)/2. It is still FFT-expressible — Σ_j K[i+j] f_j is a correlation, i.e. a convolution against a reversed image — but it is not the same kernel and not the same apply: from_nufft_precision_operator's single Khat plus offset-indexed col_offsets machinery would need a parallel sum-indexed path, and the dense translation_invariant_kernel[y_diff, x_diff] assembly a [y_sum, x_sum] twin.
Cost: 2x the (already hours-long) precision-operator setup, 2x Khat memory, ~2 FFT convolutions per apply, and a second index-mapping code path in both the JAX and numba assemblers.
Need: none known. SimulatorInterferometer writes a constant equal-sigma noise map, and measurement-set weights are per-visibility scalars applied to both real and imaginary parts, so unequal sigmas only arise from hand-constructed noise maps.
Recommendation: don't do it. The guard is the right cost/benefit — roughly doubling the most expensive precomputation in the library plus a second assembler path, for a case no real or simulated dataset currently produces. Revisit only if a dataset with genuinely per-part weights appears; the algebra above is the recipe.

Full API Changes (for automation & release notes)

Changed Behaviour

• Interferometer.apply_sparse_operator — raises DatasetException for unequal real/imag noise sigma.

Generated by the PyAutoLabs agent workflow.

🤖 Generated with Claude Code

https://claude.ai/code/session_012JM45sA4YGEUw6KYW8Pm96