perf: reuse Convolver state, cache operated-matrix dict, hoist linear-func pair loop (#496 phase 1) - #497
Merged
Conversation
…hoist linear-func pair loop Phase 1 of the numba CPU likelihood speed restoration (#496). - Convolver.state_from compared the mask shape to the *kernel* shape, so the precomputed ConvolverState that Imaging builds (psf_setup_state) was never returned for convolve_over_sample_size == 1 and every convolution rebuilt it (mask resize + rfft2). It now returns the stored state iff ConvolverState.is_for_mask(mask) (pixel scales, shape and mask array match). - AbstractInversionImaging.linear_func_operated_mapping_matrix_dict is a cached_property (one build per inversion, matching the numba subclass); the numba override reaches the parent via .func (autonerves CachedProperty has no .fget). - Linear-func x linear-func curvature blocks (imaging/sparse.py and imaging_numba/sparse.py): weight each operated matrix by the noise map once, compute only the upper triangle of blocks and mirror the transpose. Exact for any per-pixel noise map (block = M_i^T diag(1/sigma^2) M_j). Tests: state reuse + bit-identical output, mapping-matrix convolution with a blurring matrix vs per-column image convolution, pair-loop vs brute-force double loop with a random non-symmetric noise map (both inversion classes). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N6xyNMYmffpHodBrkc2d91
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Companion PR: PyAutoLabs/PyAutoGalaxy#588
Phase 1 of the numba CPU sparse-operator likelihood speed restoration (#496, epic
numba-cpu-likelihood). Three exact, likelihood-preserving changes on the CPU (use_jax=False) inversion path, plus one companion fix:Convolver.state_fromreuses the precomputed state. The old shape test compared the mask to the kernel, soImaging'spsf_setup_statewas never returned forconvolve_over_sample_size == 1and every convolution rebuiltConvolverState(mask resize +rfft2). Now returns it iffConvolverState.is_for_mask(mask).linear_func_operated_mapping_matrix_dictis acached_propertyonAbstractInversionImaging(one build per inversion; the numba subclass already did this via its memo). Companion fix: the numba override reached the parent through.fget, whichautonerves.CachedPropertydoes not have — now.func.imaging/sparse.py,imaging_numba/sparse.py): noise-weight each operated matrix once instead of inside both loops, compute only the upper triangle of blocks and mirror the transpose. Exact for any per-pixel (non-constant, non-symmetric) noise map sinceblock(i,j) = M_iᵀ diag(1/σ²) M_j.Pairs with PyAutoGalaxy PR (batched MGE convolution) — both are independently mergeable; the measured gain needs both.
Measured (memo disabled, fresh
FitImagingper call, OMP=1): MGE-60 operated matrix hst 1.11 s → 0.30 s, euclid 0.68 s → 0.125 s; matrices bitwise identical. Breakdown steps "Mapped reconstruction" and "Model data + chi²" ~3.9× faster, "Blurred image" ~1.9× (state reuse). hst pinned log-likelihood27661.910133664103unchanged before/after; euclid log-likelihood bit-identical before/after.API Changes
None — internal changes only.
ConvolverStategainssource_maskandis_for_mask();state_fromreturns the stored state in more cases (identical output).See full details below.
Test Plan
pytest test_autoarray— 1237 passedtest__state_from__precomputed_state_reused_for_matching_mask,test__precomputed_state__convolution_bit_identical_to_no_state,test__convolved_mapping_matrix_via_real_space_np__matches_image_convolution_per_column(test_convolver.py);test_curvature_matrix_func_list_blocks.py(both sparse inversion classes, random non-symmetric noise map, 1e-12 vs brute force)likelihood_breakdown/pixelization_numba.pyhst pin PASSED (rtol 1e-6), euclid before/after bit-identicalFull API Changes (for automation & release notes)
Added
autoarray.operators.convolver.ConvolverState.source_mask— the pre-resize mask the state was built fromautoarray.operators.convolver.ConvolverState.is_for_mask(mask)— whether the state can be reused formaskChanged Behaviour
Convolver.state_from(mask)— returns the precomputed_statewheneveris_for_mask(mask); previously only when the mask shape equalled the kernel shape. Output identical.AbstractInversionImaging.linear_func_operated_mapping_matrix_dict—cached_property(wasproperty); value identical, built once per inversion.Notes
convolver.pycome fromblackon pre-existing drift.Generated by the PyAutoLabs agent workflow.
🤖 Generated with Claude Code
https://claude.ai/code/session_01N6xyNMYmffpHodBrkc2d91