Skip to content

Treat missing as optional in UpdateNamespacePropertiesResponse - #4074

Open
mayuriphad wants to merge 1 commit into
apache:mainfrom
mayuriphad:fix/update-namespace-properties-optional-missing
Open

mayuriphad wants to merge 1 commit into
apache:mainfrom
mayuriphad:fix/update-namespace-properties-optional-missing

Conversation

@mayuriphad

Copy link
Copy Markdown

Closes #4066

Rationale for this change

In the REST spec, UpdateNamespacePropertiesResponse requires only updated and removed. missing is optional and nullable: true ("Server's do not need to implement this"). pyiceberg's response model declared missing: list[str] = Field() as required. So against a server that omits it (Lakekeeper, or anything built on iceberg-rust's response type), RestCatalog.update_namespace_properties raised:

pydantic_core._pydantic_core.ValidationError: 1 validation error for UpdateNamespacePropertiesResponse
missing
  Field required [type=missing, input_value={'updated': [], 'removed': ['a', 'b']}, input_type=dict]

That happens after the server had already applied the update.

missing now defaults to an empty list, and an explicit null is mapped to []. That keeps PropertiesUpdateSummary.missing a list[str] as before, so callers see no type change.

Are these changes tested?

Yes. test_update_namespace_properties_200_without_missing covers a response that omits missing and one with "missing": null. Both fail on main with the ValidationError above and pass with this change. The rest of tests/catalog/test_rest.py passes locally, except test_rest_catalog_with_google_credentials_path, which fails on main too because google-auth isn't installed in my environment.

Are there any user-facing changes?

Yes, a bug fix: update_namespace_properties works with REST servers that omit the optional missing field.

The REST spec marks the missing field of UpdateNamespacePropertiesResponse
as optional and nullable, and servers such as Lakekeeper omit it when no
requested removal was missing. pyiceberg required it, so
update_namespace_properties raised a ValidationError after the server had
applied the change. Default it to an empty list and map null to [].

Closes apache#4066
Copilot AI balanced review requested due to automatic review settings October 4, 2026 04:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

RestCatalog.update_namespace_properties fails when the server omits the optional "missing" field

2 participants