feat: say why a run failed when the LLM quota runs out or the key is rejected - #136
Conversation
…rejected The engine exits 3 when the LLM quota is exhausted and 2 when the key is rejected, and stops without publishing an analysis. analyze_repository.py maps those codes to one line saying what to fix, raises it as the error annotation, and writes it to the step's failure_reason output, which the review failure comment now shows above the link to the logs. Any other failure is reported as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CodeBoarding reviewStatus: 1 changed component See the full change in CodeBoarding. graph LR
n_action_scripts["action_scripts"]
classDef added fill:#1f883d,stroke:#0b5d23,color:#ffffff;
classDef modified fill:#bf8700,stroke:#7d4e00,color:#ffffff;
classDef deleted fill:#cf222e,stroke:#82071e,color:#ffffff,stroke-dasharray:5 3;
class n_action_scripts modified;
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: db3e96c18f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| # The engine's exit codes for an LLM refusal the user has to fix, and what to tell them. | ||
| ENGINE_REFUSALS = { | ||
| 2: "The LLM provider rejected the API key. Check the key's secret and re-run.", | ||
| 3: "The LLM provider's token or credit quota is exhausted. Add credits or raise the quota, then re-run.", |
There was a problem hiding this comment.
action.yml:362 still installs codeboarding==0.14.4, which does not emit the newly handled exit code 3; the corresponding engine change requires a later release. Consequently, quota exhaustion can retain the old behavior—including publishing partial analysis—while this commit's README promises an actionable failure and no partial output. Ship the supporting engine release and update both the pin and mirrored provider table with this change.
AGENTS.md reference: AGENTS.md:L12-L16
Useful? React with 👍 / 👎.
| 2: "The LLM provider rejected the API key. Check the key's secret and re-run.", | ||
| 3: "The LLM provider's token or credit quota is exhausted. Add credits or raise the quota, then re-run.", |
There was a problem hiding this comment.
When llm: hosted or llm: license is selected, neither message describes a fix the caller controls: these modes use the constant placeholder key from configure-auth.sh and authenticate through the OIDC relay, which forwards upstream HTTP failures unchanged. A rejected OIDC/license credential can therefore be reported as a provider API-key secret to check, while a hosted-quota rejection tells the caller to add provider credits they do not own. Pass the resolved tier into this mapping and give hosted/license users the corresponding OIDC, licence, or plan remedy.
Useful? React with 👍 / 👎.
1 participant
Summary
The Action side of CodeBoarding/CodeBoarding#610, which makes the engine fail on the first LLM error instead of shipping folder-named components. With that engine, a failed LLM call already fails the job here: the engine exits non-zero,
analyze_repository.pyraises, and there is no retried full run becauseincrementalno longer answersrequiresFullAnalysisfor it. What was missing is why.analyze_repository.pymaps those two codes to one line saying what to fix. It becomes the::error::annotation, in sync and review mode alike, and is written to the step'sfailure_reasonoutput.failure_reasonabove the link to the logs. It is empty for any other failure, so that comment reads as before.This replaces the quota half of #124 (exit code only, no JSON contract, no diagnostics reading).
What it looks like
In review mode, the failure comment for an exhausted quota reads:
Before it takes effect
Exit 2 works with the pinned engine (
codeboarding==0.14.4). Exit 3 needs a release containing #610 and a bump of that pin.Tests
tests/test_analyze_repository.py: exit 3 names the quota and writesfailure_reason, exit 2 names the key, and exit 1 keeps today's message and writes nothing. 178 tests pass;blackis clean.🤖 Generated with Claude Code