Add client option FetchOnlyKnownKinds - #1396
Merged
Merged
Conversation
bgentry
approved these changes
Sep 27, 2026
bgentry
left a comment
bgentry
left a comment
Contributor
There was a problem hiding this comment.
Awesome! My main concern here is making sure that this performs well both in OSS and Pro and that the latter maintains support for this as well. Can you make sure to add support on the Pro side as well so we can ship it at the same time, and run similar benchmarks over there? We have a solid benchmark foundation over there that your agent should be able to pick up and run with easily.
bgentry
reviewed
Sep 27, 2026
|
|
||
| ### Added | ||
|
|
||
| - Added `Config.FetchOnlyKnownKinds` to restrict job fetching to registered worker kinds, including aliases. Clients with different workers can share a queue while leaving unknown jobs available without consuming attempts. Disabled by default; leader election and stuck-job rescue behavior are unchanged. |
brandur
force-pushed
the
brandur-fetch-only-known-kinds
branch
from
September 28, 2026 16:56
34703b5 to
65f7671
Compare
Here, add a client option for `FetchOnlyKnownKinds` which modifies producer queries to only fetch job kinds that are registered with the client. While not generally needed, I was thinking that this might be useful for users who are trying to do cross-language migrations within River. e.g. Trying to move from Go to Rust and needing a clean way to make the changeover. The target language could handle all job kinds, while the source language would use `FetchOnlyKnownKinds` and fetch a dwindling number of job kinds as they're incrementally moved over and verified. I benchmarked performance on this and there's very little difference unless you have large numbers of unknown jobs, and even then, in SQLite only. | Database / workload | Disabled | Enabled | Difference | |---|---:|---:|---:| | PostgreSQL: all known, 1 kind | 1.10 ms | 1.05 ms | Within noise | | PostgreSQL: all known, 100 kinds | 1.19 ms | 1.34 ms | +13% | | PostgreSQL: 1% known | 1.33 ms | 1.35 ms | +2% | | SQLite: all known, 100 kinds | 1.07 ms | 1.10 ms | +3% | | SQLite: 50% known | 1.05 ms | 1.19 ms | +13% | | SQLite: 1% known | 1.11 ms | 6.39 ms | **5.8×** |
brandur
force-pushed
the
brandur-fetch-only-known-kinds
branch
from
September 28, 2026 18:48
65f7671 to
fa33a7b
Compare
Contributor
Author
|
Thanks! Okay I opened another PR over on the Pro side to add support there, with a planned release at the same time this next version of River goes out. Benchmarks confirm no major performance regression. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Here, add a client option for
FetchOnlyKnownKindswhich modifiesproducer queries to only fetch job kinds that are registered with the
client.
While not generally needed, I was thinking that this might be useful for
users who are trying to do cross-language migrations within River. e.g.
Trying to move from Go to Rust and needing a clean way to make the
changeover. The target language could handle all job kinds, while the
source language would use
FetchOnlyKnownKindsand fetch a dwindlingnumber of job kinds as they're incrementally moved over and verified.
I benchmarked performance on this and there's very little difference
unless you have large numbers of unknown jobs, and even then, in SQLite
only.