Skip to content

Specialisation fails when subclassing built in types #158639

Description

@johng

Bug report

import dis, sys


class D(dict):
    pass


def call_get(d):
    return d.get(1)


def call_site(f):
    [call] = [i for i in dis.get_instructions(f, adaptive=True) if i.baseopname == "CALL"]
    raw = dict((name, data) for name, _, data in call.cache_info)["counter"]
    return call.opname, int.from_bytes(raw, "little") >> 3     # drop 3 backoff bits


for label, d in [("exact dict     ", {1: 2}), ("dict subclass  ", D({1: 2}))]:
    call_get.__code__ = call_get.__code__.replace()            # fresh, unspecialized site
    for _ in range(100):
        call_get(d)
    op, before = call_site(call_get)
    for _ in range(10):
        call_get(d)
    _, after = call_site(call_get)
    misses = "no misses" if before == after else f"MISS on every call ({before} -> {after})"
    print(f"{label
python3.14 investigation/demo_bug.py 
exact dict      CALL_METHOD_DESCRIPTOR_FAST    counter after 100 calls = 104, after 10 more = 104: no misses
dict subclass   CALL_METHOD_DESCRIPTOR_FAST    counter after 100 calls = 10, after 10 more = 94: MISS on every call (10 -> 94)

This is due to the type checking for the exact type rather than the parent type

        op(_GUARD_CALLABLE_METHOD_DESCRIPTOR_O, (callable, self_or_null, args[oparg] -- callable, self_or_null, args[oparg])) {
            PyObject *callable_o = PyStackRef_AsPyObjectBorrow(callable);
            PyMethodDescrObject *method = (PyMethodDescrObject *)callable_o;
            EXIT_IF(!Py_IS_TYPE(method, &PyMethodDescr_Type));
            EXIT_IF((method->d_method->ml_flags & _Py_METH_CALL_FLAGS) !=
                    METH_O);
            int total_args = oparg;
            if (!PyStackRef_IsNull(self_or_null)) {
                total_args++;
            }
            EXIT_IF(total_args != 2);
            PyObject *self = PyStackRef_AsPyObjectBorrow(
                PyStackRef_IsNull(self_or_null) ? args[0] : self_or_null);
            EXIT_IF(!Py_IS_TYPE(self, method->d_common.d_type));  // Checking here
        }

This bug impacts main branch and also all versions of python with the specialisation logic.

cost of the miss per 10,000 calls (same data, subclass rows only)

call receiver guard hits (fixed) guard misses every call (unfixed) miss penalty per 10K calls
dict.get defaultdict 127 us 179 us +52 us, +41%
dict.get OrderedDict 130 us 181 us +51 us, +39%
dict.get class D(dict) 128 us 180 us +52 us, +41%
tuple.count namedtuple 140 us 198 us +58 us, +41%
str.upper str subclass 155 us 220 us +65 us, +42%

CPython versions tested on:

CPython main branch

Operating systems tested on:

Macos 26

Linked PRs

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions