Skip to content

gh-158219: Fix bytearray constructor to not use hashed object - #158329

Merged
vstinner merged 2 commits into
python:mainfrom
vstinner:bytearray_hash
Sep 29, 2026
Merged

vstinner merged 2 commits into
python:mainfrom
vstinner:bytearray_hash

Conversation

@vstinner

@vstinner vstinner commented Sep 28, 2026 •

Copy link
Copy Markdown
Member
  • Add _PyBytes_GET_CACHED_HASH() static inline function.
  • _PyBytes_IsMutable() makes sure that the hash value is not cached yet.
  • bytearray_reinit_from_bytes() checks that the bytes object is mutable.

* Add _PyBytes_GET_CACHED_HASH() static inline function.
* _PyBytes_IsMutable() makes sure that the hash value is not cached
  yet.
* bytearray_reinit_from_bytes() checks that the bytes object is
  mutable.

Co-Authored-by: Cody Maloney <cmaloney@theoreticalchaos.com>
@vstinner

Copy link
Copy Markdown
Member Author

cc @cmaloney

Another approach would be to remove the bytearray optimization to reuse a bytes object if its refcount is 1: always create a copy.

Note: _PyUnicode_IsModifiable() already checks if PyUnicode_HASH(unicode) is -1.

@cmaloney cmaloney left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. How I found this, and forward looking a bit, my hope is to add more of these cases. The Argument Clinic vectorcall work is in that same line. Hopefully make it so that bytes() and bytearray() constructed on temporaries can avoid the copy work they currently do which should make code faster without having to write very specific patterns.

That is for 3.16+ though. As I add more cases will look for simpler ways to do the cached hash + mutable check.

Comment thread Objects/bytearrayobject.c
if (_PyObject_IsUniquelyReferenced(encoded)
&& PyBytes_CheckExact(encoded))
&& PyBytes_CheckExact(encoded)
&& _PyBytes_GET_CACHED_HASH((PyBytesObject*)encoded) == -1)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the check exact + get cached for "can this be taken" feel like very internal APIs. For now bytearray and bytes are really close to eachother and only have one instance of this (_PyBytes_Resize takes care of generally).

@vstinner
vstinner merged commit 969af80 into python:main Sep 29, 2026
55 checks passed
@vstinner
vstinner deleted the bytearray_hash branch September 29, 2026 09:31
@miss-islington-app

Copy link
Copy Markdown

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @vstinner, I could not cleanly backport this to 3.15 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 969af80daf09fcd2ce8f358e5f9d54e75f8ef330 3.15

@bedevere-app

bedevere-app Bot commented Sep 29, 2026

Copy link
Copy Markdown

GH-158397 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Sep 29, 2026
@vstinner

Copy link
Copy Markdown
Member Author

I created a small 3.15 backport which omits _PyBytes_IsMutable(): PR gh-158397, since _PyBytes_IsMutable() was not backported to 3.15 yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants