Skip to content

Resume informer watches safely with bounded recovery - #2718

Open
ilcm96 wants to merge 1 commit into
kubernetes-client:masterfrom
ilcm96:fix/informer-recovery
Open

ilcm96 wants to merge 1 commit into
kubernetes-client:masterfrom
ilcm96:fix/informer-recovery

Conversation

@ilcm96

@ilcm96 ilcm96 commented Sep 26, 2026

Copy link
Copy Markdown

What type of PR is this?

/kind bug
/kind api-change

What this PR does / why we need it:

SharedInformer previously saved a resource version (RV) as soon as Watch received an event. If processing failed or the Informer stopped before updating its cache, a later WATCH could skip that event. Short failures also caused repeated requests, while reconnecting postponed periodic LISTs.

  • Resume from the last event successfully applied to the cache, or from a BOOKMARK.
  • Let Informer own LIST and WATCH retries, with separate jittered backoffs capped at 60 seconds. Schedule periodic LISTs from the completion of the last successful LIST, and recover from an expired RV (410) with a new LIST.
  • Keep Watch's last valid RV when malformed or unsupported events arrive. Such events no longer reset the consecutive-410 retry limit.
  • Close interrupted stream responses and treat an intentional resync close separately from a failure.
  • Add Watch(..., retry=False) so each stream() invokes the API callable at most once. The default remains retry=True; HTTP transport retry and timeout settings are unchanged. The default resync_period=0 does not inject timeout_seconds.

Does this PR introduce a user-facing change?

SharedInformer now resumes WATCH streams from the last successfully cached resource version and uses bounded recovery for LIST and WATCH failures.

@kubernetes-prow kubernetes-prow Bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/bug Categorizes issue or PR as related to a bug. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API labels Sep 26, 2026
@linux-foundation-easycla

linux-foundation-easycla Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: ilcm96 / name: Yun Seongmin (951ccdc)

@kubernetes-prow kubernetes-prow Bot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Sep 26, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

Welcome @ilcm96!

It looks like this is your first PR to kubernetes-client/python 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-client/python has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@kubernetes-prow kubernetes-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Sep 26, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ilcm96
Once this PR has been reviewed and has the lgtm label, please assign roycaihw for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Sep 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/api-change Categorizes issue or PR as related to adding, removing, or otherwise changing an API kind/bug Categorizes issue or PR as related to a bug. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant