Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-34656: Avoid relying on signed overflow in _pickle memos. #9261

Merged
merged 5 commits into from Sep 21, 2018

Conversation

@benjaminp
Copy link
Contributor

@benjaminp benjaminp commented Sep 13, 2018

@benjaminp benjaminp force-pushed the benjamin-pickle-overflow branch from ea95daa to 59c974e Sep 14, 2018
@benjaminp benjaminp changed the title bpo-1621: Avoid relying on signed overflow in _PyMemoTable_ResizeTable. bpo-34656: Avoid relying on signed overflow in _pickle memos. Sep 14, 2018
@@ -7061,7 +7057,7 @@ Unpickler_set_memo(UnpicklerObject *self, PyObject *obj)
error:
if (new_memo_size) {
i = new_memo_size;

This comment has been minimized.

@sir-sigurd

sir-sigurd Sep 14, 2018
Contributor

It looks like this line is redundant now.

Py_ssize_t to_process;

assert(min_size > 0);

/* Find the smallest valid table size >= min_size. */
while (new_size < min_size && new_size > 0)
while (new_size < min_size)

This comment has been minimized.

@serhiy-storchaka

serhiy-storchaka Sep 15, 2018
Member

Wouldn't this cause an infinite loop if min_size > PY_SSIZE_T_MAX?

This comment has been minimized.

@benjaminp

benjaminp Sep 15, 2018
Author Contributor

Yes, I will fix.

Modules/_pickle.c Outdated Show resolved Hide resolved
@@ -909,7 +909,8 @@ PyMemoTable_Set(PyMemoTable *self, PyObject *key, Py_ssize_t value)
* Very large memo tables (over 50K items) use doubling instead.
* This may help applications with severe memory constraints.
*/
if (!(self->mt_used * 3 >= (self->mt_mask + 1) * 2))
size_t triple_used = self->mt_used * 3;
if (triple_used > self->mt_used && triple_used < self->mt_allocated * 2)

This comment has been minimized.

@sir-sigurd

sir-sigurd Sep 15, 2018
Contributor

triple_used > self->mt_used

Is this a wrapping/overflow detection?

This comment has been minimized.

@benjaminp

benjaminp Sep 15, 2018
Author Contributor

yes

@benjaminp benjaminp force-pushed the benjamin-pickle-overflow branch from 7975314 to d8b389b Sep 20, 2018
@benjaminp benjaminp merged commit a4ae828 into master Sep 21, 2018
7 of 9 checks passed
7 of 9 checks passed
Linux-PR #Linux-PR_20180920.04 failed
Details
Linux-PR-Coverage #Linux-PR-Coverage_20180920.04 failed
Details
Windows-PR #Windows-PR_20180920.04 succeeded
Details
bedevere/issue-number Issue number 34656 found
Details
bedevere/news "skip news" label found
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
docs #docs_20180920.04 succeeded
Details
macOS-PR #macOS-PR_20180920.04 succeeded
Details
@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Sep 21, 2018

Thanks @benjaminp for the PR 🌮🎉.. I'm working now to backport this PR to: 3.6, 3.7.
🐍🍒🤖

@benjaminp benjaminp deleted the benjamin-pickle-overflow branch Sep 21, 2018
miss-islington added a commit to miss-islington/cpython that referenced this pull request Sep 21, 2018
…ythonGH-9261)

(cherry picked from commit a4ae828)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Sep 21, 2018

GH-9465 is a backport of this pull request to the 3.7 branch.

miss-islington added a commit to miss-islington/cpython that referenced this pull request Sep 21, 2018
…ythonGH-9261)

(cherry picked from commit a4ae828)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Sep 21, 2018

GH-9466 is a backport of this pull request to the 3.6 branch.

miss-islington added a commit that referenced this pull request Sep 21, 2018
…H-9261)

(cherry picked from commit a4ae828)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
miss-islington added a commit that referenced this pull request Sep 21, 2018
…H-9261)

(cherry picked from commit a4ae828)

Co-authored-by: Benjamin Peterson <benjamin@python.org>
vstinner added a commit to vstinner/cpython that referenced this pull request Feb 15, 2019
vstinner added a commit to vstinner/cpython that referenced this pull request Feb 15, 2019
vstinner added a commit to vstinner/cpython that referenced this pull request Feb 15, 2019
larryhastings added a commit that referenced this pull request Feb 25, 2019
…H-9261) (#11870)

* bpo-34656: Avoid relying on signed overflow in _pickle memos (GH-9261)

(cherry picked from commit a4ae828)
larryhastings added a commit that referenced this pull request Feb 26, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

6 participants