Skip to content

gh-135385: Fix memory regression for classes with both __slots__ and __dict__ #135389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

shuimu5418
Copy link

@shuimu5418 shuimu5418 commented Jun 11, 2025

EDIT: Apologies, I commited my pr before seeing Eclips4's comment. I have since confirmed locally by compiling and testing the main branch that this issue is indeed fixed there.
This PR should NOT be merged or backported. It would fail to fix the bug on 3.13 and could potentially mask the real issue or introduce other instabilities.


This patch fixes a significant memory regression in Python 3.13 where classes that inherit __slots__ but also have a __dict__ consume ~4x more memory than necessary.

The root cause was that such subclasses incorrectly inherited a non-zero tp_itemsize from their base. This prevented the Py_TPFLAGS_INLINE_VALUES flag from being set, disabling the key-sharing dictionary optimization.

The fix, located in Objects/typeobject.c, adjusts the logic to ensure a subclass only inherits a non-zero tp_itemsize if it defines its own _slots_. Otherwise, tp_itemsize is correctly set to 0, re-enabling the memory optimization.

test:

PS F:\github\cpython001> .\python.bat -m test -j3 test_class test_types test_dict
Running Debug|x64 interpreter...
Using random seed: 1566459836
0:00:00 Run 3 tests in parallel using 3 worker processes
0:00:01 [1/3] test_class passed
0:00:02 [2/3] test_dict passed
0:00:03 [3/3] test_types passed

== Tests result: SUCCESS ==

All 3 tests OK.

Total duration: 3.5 sec
Total tests: run=273
Total test files: run=3/3
Result: SUCCESS

benchmark:

PS F:\github\cpython001> .\python.bat .\build\o-test.py  
Running Debug|x64 interpreter...
1M Point3D (only __slots__) instances:         64_448_944 bytes
1M Point3D (only __dict__) instances:          104_452_144 bytes
1M Point3D (__dict__ and __slots__) instances: 120_452_144 bytes
# o-test.py
import tracemalloc
import gc


class _Point2D:
    __slots__ = ("x", "y")


class Point3D_OnlySlots(_Point2D):
    __slots__ = ("z",)

    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z


class Point3D_DictAndSlots(_Point2D):
    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z


class Point3D_OnlyDict:
    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z


gc.collect()  # clear freelists
tracemalloc.start()
_ = [Point3D_OnlySlots(1, 2, 3) for _ in range(1_000_000)]
print(
    f"1M Point3D (only __slots__) instances:         {tracemalloc.get_traced_memory()[0]:_} bytes"
)

gc.collect()  # clear freelists
tracemalloc.start()
_ = [Point3D_OnlyDict(1, 2, 3) for _ in range(1_000_000)]
print(
    f"1M Point3D (only __dict__) instances:          {tracemalloc.get_traced_memory()[0]:_} bytes"
)

gc.collect()  # clear freelists
tracemalloc.start()
_ = [Point3D_DictAndSlots(1, 2, 3) for _ in range(1_000_000)]
print(
    f"1M Point3D (__dict__ and __slots__) instances: {tracemalloc.get_traced_memory()[0]:_} bytes"
)

@shuimu5418 shuimu5418 requested a review from markshannon as a code owner June 11, 2025 13:25
@python-cla-bot
Copy link

python-cla-bot bot commented Jun 11, 2025

All commit authors signed the Contributor License Agreement.

CLA signed

@bedevere-app

This comment was marked as resolved.

@ZeroIntensity ZeroIntensity added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Jun 11, 2025
Copy link
Member

@ZeroIntensity ZeroIntensity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to force-push, we squash at the end.

@ariebovenberg
Copy link
Contributor

I'm not that familiar with the underlying type machinery. Does the fix also work for the opposite: __slots__ class inheriting from __dict__ class?

class _NoSlots:
    pass

class Point3D_BrokenSlots(_NoSlots):
    __slots__ = ("x", "y")

    def __init__(self, x, y, z):
        self.x, self.y, self.z = x, y, z

@AA-Turner
Copy link
Member

Adding label per:

This PR should NOT be merged or backported. It would fail to fix the bug on 3.13 and could potentially mask the real issue or introduce other instabilities.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants