Skip to content

feat: follow redirects under the response body cap - #152

Merged
lesnik512 merged 3 commits into
mainfrom
feat/cap-follows-redirects
Oct 3, 2026
Merged

lesnik512 merged 3 commits into
mainfrom
feat/cap-follows-redirects

Conversation

@lesnik512

Copy link
Copy Markdown
Member

max_response_body_bytes and follow_redirects=True used to be refused together (ValueError at construction), because httpx2 reads every intermediate redirect body without a limit. With a cap set, httpware now follows redirects itself and caps only the final response.

How

  • Each hop is client.send(request, stream=True, follow_redirects=False). httpx2 builds the next request (response.next_request), so method rewrites, cookies, body reuse on 307/308 and Authorization stripping stay httpx2's.
  • Intermediate responses are closed unread and kept in response.history. More than max_redirects hops raises httpx2.TooManyRedirects, which maps to the same TransportError as without a cap.
  • Hops after the first pass auth=None. Otherwise _build_request_auth would re-apply the client's auth on every hop, including one to another origin, undoing httpx2's stripping.
  • client.follow_redirects is read at call time, so a caller-provided httpx2_client works too.
  • _terminal uses this only when a cap is set; stream() switches from client.stream(...) to build_request plus the same loop when a cap is set. Without a cap nothing changes.
  • Fixed on the way: _terminal passed the original request to the capped reader, so after a redirect response.url and the bodiless-response check (HEAD after a 303) used the wrong request.

Trade-offs (cap set and following redirects)

  • Responses in history have unread bodies; .content on them raises ResponseNotRead.
  • An intermediate HTTP/1.1 connection is closed rather than reused.
  • Multi-step auth such as DigestAuth runs on the first hop only, so a challenge after a redirect is not answered.

Not in this PR

httpx2's _send_handling_auth reads intermediate auth responses (the DigestAuth 401) without the cap, with or without redirects. That is a separate bug fix.

Behaviour change

Construction no longer raises ValueError for this combination. Nothing that worked before behaves differently.

Verification

just lint-ci, just test (971 passed), just docs-build, just adr-check. Mutations of the history and auth=None lines are caught by the new tests in both worlds.

With max_response_body_bytes set and follow_redirects=True, httpware now
follows redirects itself, closing intermediate responses unread and
capping only the final one, instead of rejecting the combination.
@lesnik512
lesnik512 merged commit 67d742c into main Oct 3, 2026
13 checks passed
@lesnik512
lesnik512 deleted the feat/cap-follows-redirects branch October 3, 2026 09:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant