Skip to content

gh-135321: Changing data type of size variable for BINSTRING in _pickle #135322

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Legoclones
Copy link
Contributor

@Legoclones Legoclones commented Jun 10, 2025

I changed the data type to int from Py_ssize_t for the size variable. This variable is also used as an argument to 3 other functions but should be auto-casted (similar to load_counted_long). I also changed the error message to match the error message in the Python load_binstring() function.

I would like to add tests for this specific case, but the payload that would show whether or not it works as intended requires a 2GB large pickle, and I'm not sure if you'd like that to be in the tests. Let me know what you think.

…opcode `BINSTRING` to `int` from `Py_ssize_t`
@Legoclones Legoclones changed the title gh-135321: Changing data type of size variable for C implementation of pickle … gh-135321: Changing data type of size variable for BINSTRING in _pickle Jun 10, 2025
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bug was introduced in bpo-17810/gh-62010. The code should use calc_binsize() instead of calc_binsize().

And it is worth to add a test:

diff --git a/Lib/test/pickletester.py b/Lib/test/pickletester.py
index 9d6ae3e4d00..2a85e31078c 100644
--- a/Lib/test/pickletester.py
+++ b/Lib/test/pickletester.py
@@ -1100,6 +1100,11 @@ def test_large_32b_binunicode8(self):
         self.check_unpickling_error((pickle.UnpicklingError, OverflowError),
                                     dumped)
 
+    def test_large_binstring(self):
+        errmsg = 'UnpicklingError: BINSTRING pickle has negative byte count'
+        with self.assertRaisesRegex(pickle.UnpicklingError, errmsg):
+            self.loads(b'T\0\0\0\x80')
+
     def test_get(self):
         pickled = b'((lp100000\ng100000\nt.'
         unpickled = self.loads(pickled)

Legoclones and others added 3 commits June 10, 2025 16:12
@Legoclones
Copy link
Contributor Author

I made all the requested changes - thank you!

@serhiy-storchaka serhiy-storchaka added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels Jun 11, 2025
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@serhiy-storchaka serhiy-storchaka enabled auto-merge (squash) June 11, 2025 09:53
@serhiy-storchaka serhiy-storchaka merged commit 2b8b477 into python:main Jun 11, 2025
47 checks passed
@miss-islington-app
Copy link

Thanks @Legoclones for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 11, 2025
…ent > 0x7fffffff in pickle (pythonGH-135322)

(cherry picked from commit 2b8b477)

Co-authored-by: Justin Applegate <[email protected]>
Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Jun 11, 2025

GH-135382 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.14 bugs and security fixes label Jun 11, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jun 11, 2025
…ent > 0x7fffffff in pickle (pythonGH-135322)

(cherry picked from commit 2b8b4774d29a707330d463f226630185cbd3ceff)

Co-authored-by: Justin Applegate <[email protected]>
Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Jun 11, 2025

GH-135383 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.13 bugs and security fixes label Jun 11, 2025
serhiy-storchaka added a commit that referenced this pull request Jun 11, 2025
…ment > 0x7fffffff in pickle (GH-135322) (GH-135383)

(cherry picked from commit 2b8b477)

Co-authored-by: Justin Applegate <[email protected]>
Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit that referenced this pull request Jun 11, 2025
…ment > 0x7fffffff in pickle (GH-135322) (GH-135382)

(cherry picked from commit 2b8b477)

Co-authored-by: Justin Applegate <[email protected]>
Co-authored-by: Serhiy Storchaka <[email protected]>
@Legoclones Legoclones deleted the pickle-binstring-pyssizet-to-int branch June 11, 2025 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants