Skip to content

fix: EP projection raises ProjectionException on non-finite sufficient statistics; factor_step recovers (#1653) - #1655

Merged
Jammy2211 merged 2 commits into
mainfrom
feature/ep-projection-exception
Sep 30, 2026
Merged

Jammy2211 merged 2 commits into
mainfrom
feature/ep-projection-exception

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

AbstractMessage.project guarded its importance-weighted sufficient statistics with a bare assert np.isfinite(suff_stats).all(). One factor whose samples or log weights went non-finite (nan/inf samples, nan/+inf log weights, all weights zero, or overflow of T(x)·w) therefore raised a context-free AssertionError. That error escaped EP's factor_step recovery tuple and killed the whole graph fit. Under python -O the check vanished and the non-finite message went through silently.

This PR replaces the assert with a typed autofit.exc.ProjectionException(ValueError). The message says which input caused the failure, and the prior id and model parameter path are added on the way up. EP's factor_step now recovers from it the way it already recovers from InitializerException: the factor keeps its previous message for that sweep with StatusFlag.EXCEPTION, and max_consecutive_failures still aborts a factor that keeps failing.

API Changes

  • New exception autofit.exc.ProjectionException, a subclass of ValueError.
  • AbstractMessage.project, Prior.project and Result.projected_model raise it where the bare assert used to fire. The message names the failing input (non-finite samples / nan or +inf log weights / all -inf log weights / weighted-moment overflow), the prior id (Prior.project) and the parameter path (Result.projected_model). The check also survives python -O.
  • EP factor_step lists it in its recovery tuple, so one bad projection becomes a failed sweep update, not a failed graph fit.
  • Behaviour change: non-EP callers of Result.projected_model now get a ValueError subclass instead of AssertionError. Nothing in autofit/ catches AssertionError.
  • No removals, no signature changes.
    See full details below.

Test Plan

  • Red witness on the unmodified source: 10 failed, all AssertionError at autofit/messages/abstract.py:316. That covers 6 parametrised direct cases, 2 prior-id cases, the factor_step / EPOptimiser.run recovery tests and the projected_model path case.
  • New/extended tests green: test_autofit/messages/test_project_nonfinite.py + test_autofit/graphical/functionality/test_factor_failure_recovery.py give 29 passed; test_autofit/graphical/test_unification.py gives 34 passed.
  • Full suite after rebase on origin/main: python -m pytest test_autofit -x, 2946 passed, 2 skipped.
  • Workspace smoke (smoke profile: PYAUTO_TEST_MODE=2, PYAUTO_DISABLE_JAX=1) against this branch, autofit_workspace_test: graphical/analytic_ep_minimal.py PASS (MINIMAL EP SELF-TESTS: PASS, 2 s). graphical/analytic_gaussian_collapse.py PASS (COLLAPSE CONFIG: PASS 5/5 seeds, 18 s).
Full API Changes (for automation & release notes)

Added

  • autofit.exc.ProjectionException(ValueError): non-finite sufficient statistics in an importance-weighted message projection. Deliberately not a MessageException (callers turn those into silent reverts or -inf likelihoods) and not a SearchException (misconfigured search).
  • AbstractMessage._nonfinite_projection_reason(samples, log_weight_list, suff_stats) (private classmethod). Called only on the failure path, it reports the first cause in propagation order.

Changed Behaviour

  • autofit.messages.abstract.AbstractMessage.project: raises ProjectionException instead of assert on non-finite sufficient statistics.
  • autofit.mapper.prior.abstract.Prior.project: re-raises ProjectionException with prior id=<id> (<PriorClass>) context (from e).
  • autofit.non_linear.result.Result.projected_model: re-raises ProjectionException with path=<a.b.c> context (from e). Callers get ValueError rather than AssertionError.
  • autofit.graphical.expectation_propagation.optimiser.factor_step: exc.ProjectionException added to the recovery tuple. The result is the previous message, a StatusFlag.EXCEPTION status, and a logged exception.

Migration

  • None. Code that caught AssertionError from projected_model / project should catch ValueError (or autofit.exc.ProjectionException). No such caller exists in autofit/.

Generated by the PyAutoLabs agent workflow.

Heart RED override (development only)

  • Authorised by the live human in the Claude Code session at 2026-09-30 ~11:50 BST. Question: "Do you authorise the Heart RED development override for push + pending-release PR-open on PyAutoFit#1653, and, when its implementation lands green, on PyAutoFit#1654 (moment projection)? No merge, no release; recorded in the four sinks." Answer: "Yes, both fix: EP projection raises ProjectionException on non-finite stats #1653 and feat: LaplaceOptimiser(projection="moments") for hierarchical scatter #1654 (Recommended)".
  • Exact RED reasons (readiness --json, ts 2026-09-30T09:51:49Z, re-read ~11:45 BST unchanged): "PyAutoArray: 2 commit(s) behind origin"; "PyAutoLens: 2 commit(s) behind origin"; "release validation FAILED (stage integrate)". Yellow: "workspace validation not passing (0 failed, 1 timeout, cloud#36404726969: autolens_test scripts/multi_dataset/rectangular.py)"; "manifest drift: public front-door organ tables (generated) — 1 mismatch(es) vs PyAutoMind/repos.yaml".
  • Branch gates passed: rebase on origin/main clean; pytest test_autofit -x 2946 passed, 2 skipped; red witness confirmed on the old source; graphical smoke 2/2 PASS.
  • This PR does not claim to fix Heart. Heart stays RED for release purposes. Merge needs its own human command (/prm) and green required checks.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Up1fQmjtGAh4RhLrWHfcfg

Jammy2211 and others added 2 commits September 30, 2026 11:27
…— direct project cases + factor_step recovery

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Up1fQmjtGAh4RhLrWHfcfg
…ufficient statistics; factor_step recovers (PyAutoFit#1653)

AbstractMessage.project ended on a bare `assert np.isfinite(suff_stats).all()`.
AssertionError is not in factor_step's recovery tuple, so one factor's bad
projection killed a whole EP run (RAL 342411: ic50 ladder, N=50, sweep 4,
global factor), with no hint of which input was non-finite.

- exc.ProjectionException(ValueError) replaces the assert. The new
  _nonfinite_projection_reason classmethod (failure path only) names the
  cause: non-finite samples (counts, first index), nan / +inf log weights,
  all log weights -inf, or overflow of T(x)·w (max |x|). Moment maths is
  unchanged.
- Prior.project re-raises with the prior id and type (the id does not reach
  AbstractMessage.project through TransformedMessage.project), and
  Result.projected_model re-raises with the parameter path (loop instead of
  a dict comprehension; the zero-weight guard stays).
- factor_step lists exc.ProjectionException in its except tuple (already
  covered via ValueError; listed for intent): a non-finite projection of one
  factor's samples is a failed sweep update, not a failed graph fit. It gets
  StatusFlag.EXCEPTION, keeps the previous message, and counts toward
  max_consecutive_failures. AssertionError is deliberately not caught.

Why not MessageException: mean_field.py, hierarchical.py and laplace/newton.py
turn that into a silent revert or a -inf likelihood. Why not SearchException:
it is uncaught and means a misconfigured search.

Behaviour change: outside EP, Result.projected_model callers now get a clear
ProjectionException (a ValueError) where they got AssertionError; no autofit/
code catches AssertionError. Under `python -O` the old assert vanished and the
non-finite statistics flowed on silently.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Up1fQmjtGAh4RhLrWHfcfg
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Sep 30, 2026
@Jammy2211
Jammy2211 merged commit 07f6e0b into main Sep 30, 2026
4 checks passed
@Jammy2211
Jammy2211 deleted the feature/ep-projection-exception branch September 30, 2026 13:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release PR queued for the next release build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant