research: trial ecosystem responsibility routing

[Jammy2211]

Overview

Evaluate the routing convention proposed by research #440/#441 after terminology adoption #442/#443. The human asked "ok do it" in response to the recommendation to trial agent routing across visualization, profiling and inference; this task executes that approved research step.

Plan

• Fix six hypothetical cases, identical evidence, expected routes and scoring before running evaluators.
• Compare fresh-context responses under current guidance versus current guidance plus the proposed checklist, using the same model and effort.
• Score decisions and proposed context selections against the predeclared key; retain no-change as a valid outcome.
• Write a cited report and retain bounded raw/protocol evidence; independently review the conclusions and validate docs/JSON.

Detailed plan

Primary and only modified repo: PyAutoBrain, clean on main; no active claim conflict.
Branch: feature/ecosystem-routing-trial.
Worktree: /home/jammy/Code/PyAutoLabs/.worktrees/ecosystem-routing-trial/PyAutoBrain.

Write docs/research/ecosystem_routing_trial.md and bounded JSON/text evidence under
research/ecosystem_routing_trial/. Six cases (two/domain) test visualization
producer/organ ownership, profiling comparability/library changes, and inference
science/code routes. Fix expected answers, scoring dimensions and confounds before
evaluation; keep answer keys hidden from evaluators. Preserve exact requests,
responses, source commits and resolved model/settings. Assess producer, decision
owner, change owner, existing door, boundary and suggested minimal context.
Do not equate selected context or response length with observed retrieval savings.
This is a one-pass illustrative comparison, not a statistically powered benchmark.

Use read-only isolated evaluation sessions; no scientific execution or repo writes
by evaluators. Baseline uses current guidance; treatment adds only the proposed
routing convention. Keep same source evidence, case order and output contract;
report order/one-sample/hypothetical limits. Independent review checks raw evidence
and conclusions; run Sphinx/link/JSON/whitespace checks. No new framework, routing
policy, metadata, organs or unrelated tasks. Any follow-up ideas are proposed
before writing the ideas registry. Heart entry feed STALE; shipping gate separate.

Original prompt

Trial ecosystem role routing against current guidance

Type: research
Target: @PyAutoBrain
Repos:

• PyAutoBrain
  Difficulty: medium
  Autonomy: supervised
  Priority: normal
  Consequence: judge
  Filed: 2026-10-01
  Status: draft

Original user request (verbatim)

ok do it

Accepted recommendation and context

In response to the completed layer-design research and terminology adoption,
the assistant recommended: "Trial the agent-routing convention. Use visualization,
profiling and inference examples to check whether agents correctly distinguish
the evidence producer, decision owner and repository needing a change. This is
the next step I recommend." The user authorized that next step above.

Research #440 / PR #441 proposed the convention; adoption #442 / PR #443
established responsibility terminology but did not change routing machinery.

Plan

Run a bounded exploratory comparison of six hypothetical routing cases (two per
domain), using fresh isolated evaluation contexts and identical case/evidence
packets. Baseline is current guidance after terminology adoption; treatment adds
the proposed evidence/producer/decision/change-owner checklist. Use the same
available model and effort for both; record resolved model and settings. Do not
present model identity or a hypothesis as a preferred answer to evaluators.

Before evaluation, fix the protocol, cases, expected decisions and scoring.
Include project versus organ visualization defects, environment versus library
profiling drift, and scientific inference follow-up versus a code defect. Missing
information must permit a conditional route, not an invented owner or verdict.
Do not run scientific jobs or use the trial to change scientific conclusions.

Score evidence attribution, decision owner, change owner, existing action door,
approval/record boundary and proposed minimal context. Distinguish predicted
context choices from observed tool reads; do not claim token/cost savings from
answer length. Report per-case paired outcomes, not statistical significance.
Retain no-change as a valid outcome; recommend a larger/real-world trial only if
justified by the results. Do not alter runtime routing, role schemas or organs.

Deliverables

• docs/research/ecosystem_routing_trial.md: protocol, outcomes, counterexamples,
  limits, and recommendation on whether to adopt the checklist, revise it, or
  retain current guidance.
• research/ecosystem_routing_trial/: bounded versioned protocol/cases, evaluator
  responses and assessment, with enough source/model/hash provenance to inspect
  the comparison. No new evaluation framework or reusable harness.
• Independent review of conclusions against raw evidence; Sphinx/link/JSON checks.

All outputs stay in Brain. Other repositories are read-only evidence, with no
new claims. The separately filed inference documentation and profiling-board
registration tasks are not silently folded in. Candidate follow-up work is
proposed before being appended to Mind ideas, per the research skill.

[Jammy2211]

Routing trial protocol fixed before evaluation

Six hypothetical cases, two per visualization/profiling/inference domain. One fresh
Fable/high session per condition; identical current source evidence, case order and
structured response fields. Treatment adds only the explicit routing procedure.
Evaluators have no tools or MCP and do not receive the answer key.

Six binary dimensions per case (36/condition): producer, decision, change owner,
existing door, boundary and proposed minimal context. Exact change-owner accuracy
also reported. No token-efficiency, real-retrieval or statistical significance claim.
The shared output schema already scaffolds role reasoning, a ceiling-effect risk.

Protocol frozen UTC: 2026-10-01T19:42:32.380671+00:00
Protocol SHA256: 3ac0d3096a752bf1f89cc5caecdcff2426953bb91efeae95b216babe395b62fa
Answer-key SHA256: 791989f021721f2f1fd397642b5708a5df965776705f36e149dee7dd1d0acf2d

Source/case/prompt hashes are in protocol.json. Both conditions run once; no answer-quality retries. The decision rule permits no checklist adoption if no gain is observed.

[Jammy2211]

Research deliverable — PR #445

PR: #445
Commit: 6ea1571

Both conditions chose all six change targets correctly. The checklist conflated decision owner and repair repository in three explicit fields, while its action routes remained appropriate. Retain adopted terminology and current routing; no mandatory checklist or runtime/schema/organ change. The report discloses grading-time context leniency, possible richer checklist contract coverage, the extra output field, and the one-pair synthetic/no-retrieval limitations. Follow-up organ specification is proposed, not filed.

Validation: Sphinx HTML zero warnings; frozen protocol/input/response hashes, parsed JSON equality, score totals, download paths and all 36 source-excerpt paragraphs against pinned git objects verified. Heart GREEN score 100 at 2026-10-01T19:53:08.542623+00:00; earlier manifest YELLOW cleared on canonical refresh without override. Fixed research data only, so no scientific smoke run applies.

Independent Claude Fable review found two reporting gaps, both corrected. The focused re-review below is CLEAN. It reviewed working-tree files; SHA256 comparison independently confirms all 14 committed files match that reviewed snapshot. ReviewSurface flags no-test-changes/config-or-schema refer to a prose/static-JSON evidence bundle, not production configuration.

Independent re-review

Verdict: CLEAN. Both required corrections from the prior review are present and accurate, the optional wording follow-ups were adopted, and nothing in the frozen inputs or model responses shows signs of change.

Required correction 1, context leniency. Verified in the report at docs/research/ecosystem_routing_trial.md:45-50 and :83-91, and in the assessment's method string and grading note 3.

• The report now says the leniencies were decided at grading time and that the protocol did not predeclare them. That matches protocol.json, whose frozen context rule only requires relevant bounded scope without whole-ecosystem loading.
• It states that a stricter reading could mildly favour the checklist and names the four cases. The assessment evidence supports this: the checklist's context lists cite REFERENCE.md, repos.yaml, the bug conductor's AGENTS.md or the Cortex ledger in V1, V2, P2, I1 and I2, while the baseline cites none of these.
• The fifth-step claim is accurate. The checklist prompt's step five reads "load only the source/contract context needed for that action."
• The headline is qualified as depending on an interpretive choice, and the choice-independent findings are correctly separated: change-target parity and the three decision-field conflations. The recommendation is restated as holding under either reading. The opening summary rests only on the choice-independent findings, which is the right framing.

Required correction 2, unrequested P1 field. The report at :101-103 and grading note 7 both disclose the extra domain_note: null key, say it is retained in exact and parsed copies, and assign no deduction. The key is present in the parsed JSON and in the exact response text, so the disclosure matches the artifacts.

Optional follow-ups adopted. The compile-threshold wording is now "stated only for the compile axis" and the P1 treatment criticism is narrowed to the dependency-version difference. The research directory placement and Sphinx build are settled per your note.

Scope of this review. I did not re-run hashes, equality checks, source-paragraph comparisons or the Sphinx build. I checked the prose against the assessment, protocol, checklist prompt and response files by reading them.

Remaining required corrections: none. One cosmetic note only: the paragraph at :83 has an irregular line wrap that will render fine in Markdown and needs no action.

[Jammy2211]

Trial ecosystem role routing against current guidance

Type: research
Target: @PyAutoBrain
Repos:

• PyAutoBrain
  Difficulty: medium
  Autonomy: supervised
  Priority: normal
  Consequence: judge
  Filed: 2026-10-01
  Status: issued
  Issued: 2026-10-01
  Issue: research: trial ecosystem responsibility routing #444

Original user request (verbatim)

ok do it

Accepted recommendation and context

In response to the completed layer-design research and terminology adoption,
the assistant recommended: "Trial the agent-routing convention. Use visualization,
profiling and inference examples to check whether agents correctly distinguish
the evidence producer, decision owner and repository needing a change. This is
the next step I recommend." The user authorized that next step above.

Research #440 / PR #441 proposed the convention; adoption #442 / PR #443
established responsibility terminology but did not change routing machinery.

Plan

Run a bounded exploratory comparison of six hypothetical routing cases (two per
domain), using fresh isolated evaluation contexts and identical case/evidence
packets. Baseline is current guidance after terminology adoption; treatment adds
the proposed evidence/producer/decision/change-owner checklist. Use the same
available model and effort for both; record resolved model and settings. Do not
present model identity or a hypothesis as a preferred answer to evaluators.

Before evaluation, fix the protocol, cases, expected decisions and scoring.
Include project versus organ visualization defects, environment versus library
profiling drift, and scientific inference follow-up versus a code defect. Missing
information must permit a conditional route, not an invented owner or verdict.
Do not run scientific jobs or use the trial to change scientific conclusions.

Score evidence attribution, decision owner, change owner, existing action door,
approval/record boundary and proposed minimal context. Distinguish predicted
context choices from observed tool reads; do not claim token/cost savings from
answer length. Report per-case paired outcomes, not statistical significance.
Retain no-change as a valid outcome; recommend a larger/real-world trial only if
justified by the results. Do not alter runtime routing, role schemas or organs.

Deliverables

• docs/research/ecosystem_routing_trial.md: protocol, outcomes, counterexamples,
  limits, and recommendation on whether to adopt the checklist, revise it, or
  retain current guidance.
• research/ecosystem_routing_trial/: bounded versioned protocol/cases, evaluator
  responses and assessment, with enough source/model/hash provenance to inspect
  the comparison. No new evaluation framework or reusable harness.
• Independent review of conclusions against raw evidence; Sphinx/link/JSON checks.

All outputs stay in Brain. Other repositories are read-only evidence, with no
new claims. The separately filed inference documentation and profiling-board
registration tasks are not silently folded in. Candidate follow-up work is
proposed before being appended to Mind ideas, per the research skill.

Result — awaiting merge

PR: #445
Commit: 6ea1571

Both conditions name all six change targets correctly. Checklist decision-owner
fields conflate repair repository and conductor in three cases, but action routes
remain appropriate. Retain current guidance; do not mandate an extra checklist.
Scores 36/36 versus 33/36 depend on disclosed grading-time context leniency;
contract coverage can favour the checklist under a stricter reading. No measured
retrieval or efficiency claim. Full frozen inputs, exact answers and assessment
are versioned with the report. No runtime/schema/organ change.

Independent Claude Fable review: FINDINGS (context grading transparency and extra
output-field disclosure), corrected; focused re-review CLEAN. Reviewed file hashes
match committed bytes. Sphinx passes with zero warnings; input/response integrity,
JSON, score totals, local download paths and pinned source excerpts verified.
Heart GREEN score 100, 2026-10-01T19:53:08.542623+00:00. Earlier transient manifest
YELLOW cleared after canonical vitals refresh; no override was used.

Authorized continuation — 2026-10-01

Original user request (verbatim):

continue, stop asking fable for reviews

Continue the recommended profiling/inference organ specification in the same
research task and existing Brain worktree/PR. Do not invoke Fable reviews again.
The preceding trial evidence stays frozen. This extension is design prose only,
not an organ birth, source migration, schema rollout or science campaign.

Plan: inspect the current project outputs and Eyes/Cortex/Brain contracts; add
docs/research/profiling_inference_organs.md describing ownership, registry and
versioned project-read contracts, domain-specific comparison rules, freshness and
partial failure handling, dashboard/action boundaries, rollout and acceptance
criteria. Use pinned public repository sources and distinguish existing fields
from proposed fields. Link it from the trial report. Validate source citations,
examples and Sphinx locally; no independent-review claim for the extension.
Update the existing issue and PR around the combined final deliverable.

[Jammy2211]

Authorized continuation: organ specification ready locally

The profiling/inference organ specification is complete in the existing task
worktree (docs/research/profiling_inference_organs.md), linked from the trial
report. It defines project/organ ownership, separate project-read and cockpit
interfaces, proposed registry/envelope contracts, domain comparison rules,
freshness/partial failures, rollout and acceptance cases. No organs, project
exporters, schema migrations or new agent commands are implemented.

Validation: Sphinx HTML succeeds with zero warnings; all seven pinned source
objects resolve; frozen trial evidence is unchanged. Author checked contracts;
no further Fable review was requested, per the human instruction. The earlier
independent review covers the routing trial, not this extension.

Shipping is blocked: Heart RED score 80 at
2026-10-01T20:02:44.036627+00:00. Exact reason:
autofit_workspace: Smoke Tests failure on main.
The extension is uncommitted/unpushed. PR #445 remains at 6ea1571 with its earlier
three CI jobs green. Resume at ship-workspace once fresh Heart is GREEN or the
human explicitly authorizes the development-only override for the current reasons.

Candidate next implementation: project summary exporters, then the two organ
readers/dashboards and real second-project onboarding. Proposed only, not filed.

[Jammy2211]

Human-authorized development-only Heart RED override

For ecosystem-routing-trial / Brain #444 / PR #445, the user replied verbatim:

I authorize, continue

This directly answers the request to push the completed profiling/inference organ
specification under a development-only override. Heart's current RED reason is:

autofit_workspace: Smoke Tests failure on main

Authoritative snapshot: 2026-10-01T20:02:44.036627+00:00, RED, score 80.
Passed branch gates: Sphinx HTML build with zero warnings; seven pinned source
citations resolve; whitespace check passes; frozen trial evidence unchanged.
The specification was checked by the author against current contracts. Its scope
is documentation only, so scientific smoke tests do not apply. The earlier trial
has its own independent review; no independent review is claimed for the extension
and no further Fable review was requested, per the user's instruction.

Authorization permits commit/push/update of this development PR. It does not
repair Heart, authorize release, bypass checks or authorize merge.

[Jammy2211]

Shipped

Merged PR #445 at 13d0da5a4ca7b2bd4be571655f90e1f5fa1f5525 after both exact-head CI runs and all three jobs passed.

Delivered the bounded routing trial and profiling/inference organ specification. Keep current routing without a mandatory checklist; proposed organ read contracts, ownership, dashboard behavior and rollout are documented. No organs or exporters implemented. No further Fable reviews were requested. Heart development override remains recorded; this merge does not clear release readiness.