[2.7] bpo-36337: Use ssize_t and size_t for socket.send()/sendall() #12397

matrixise · 2019-03-18T09:11:56Z

This PR is not yet completed (missing tests), I have "backported" a patch from 3.x.

https://bugs.python.org/issue36337

vstinner · 2019-03-18T09:28:47Z

I don't think that a test is needed: I cannot find a test on master.

Your PR is wrong on Windows. Look at the master branch:

#ifdef MS_WINDOWS
    if (ctx->len > INT_MAX)
        ctx->len = INT_MAX;
    ctx->result = send(s->sock_fd, ctx->buf, (int)ctx->len, ctx->flags);
#else
    ctx->result = send(s->sock_fd, ctx->buf, ctx->len, ctx->flags);
#endif

On Windows, the type of send() length argument is int, not size_t.

Modules/socketmodule.c

…ckport the solution from 3.x with Py_ssize_t

vstinner · 2019-03-18T12:47:47Z

Modules/socketmodule.c

@@ -2797,7 +2797,8 @@ static PyObject *
 sock_send(PySocketSockObject *s, PyObject *args)
 {
    char *buf;
-    int len, n = -1, flags = 0, timeout;
+    int flags = 0, timeout;
+    Py_ssize_t len, n = -1;


return PyInt_FromLong((long)n); below is wrong

bedevere-bot · 2019-03-18T12:47:52Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

matrixise · 2019-03-18T13:05:09Z

I have made the requested changes; please review again

bedevere-bot · 2019-03-18T13:05:12Z

Thanks for making the requested changes!

@vstinner: please review the changes made to this pull request.

vstinner · 2019-03-18T13:12:15Z

Modules/socketmodule.c

@@ -2813,12 +2814,18 @@ sock_send(PySocketSockObject *s, PyObject *args)
    BEGIN_SELECT_LOOP(s)
    Py_BEGIN_ALLOW_THREADS
    timeout = internal_select_ex(s, 1, interval);
-    if (!timeout)
+    if (!timeout) {
 #ifdef __VMS
        n = sendsegmented(s->sock_fd, buf, len, flags);


That's wrong: sendsegmented() len type is int, not Py_ssize_t. Don't forget the VMS! :-)

vstinner · 2019-03-18T13:13:39Z

Misc/NEWS.d/next/Library/2019-03-18-10-08-30.bpo-36337.QhJnXy.rst

@@ -0,0 +1,3 @@
+Use ssize_t and size_t instead of int for the send() function (see
+http://man7.org/linux/man-pages/man2/send.2.html (Linux) and
+https://nixdoc.net/man-pages/FreeBSD/man2/send.2.html (FreeBSD)).


I propose:

Fix buffer overflow in :meth:~socket.socket.send and :meth:~socket.socket.sendall methods of :func:socket.socket for data larger than 2 GiB.

bedevere-bot · 2019-03-18T13:13:56Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

matrixise · 2019-03-18T13:22:59Z

I have made the requested changes; please review again

bedevere-bot · 2019-03-18T13:23:05Z

Thanks for making the requested changes!

@vstinner: please review the changes made to this pull request.

vstinner · 2019-03-18T13:25:07Z

Modules/socketmodule.c

 #ifdef __VMS
-        n = sendsegmented(s->sock_fd, buf, len, flags);
+        n = sendsegmented(s->sock_fd, buf, (int)len, flags);


(int)len is unsafe if len is larger than INT_MAX. You should modify sendsegmented() to accept Py_ssize_t rather than int.

👍 good catch and I was thinking about this problem.

vstinner

LGTM.

The subtle side effect of this change is that send() and sendall() now always return a Python 'long' object, whereas they always returned a Python 'int' object previously. I think that it's acceptable, but it might be risky to push so change late in 2.7 lifetime (EOL scheduled at the beginning of next year).

On the other side, currently, calling sendall() with more than 2 GiB data loose data. But well, who does that with Python 2.7?

I would prefer to have the review of another core dev.

@benjaminp @serhiy-storchaka @pablogsal: Would you mind to review this change? Do you think that it's ok to change send/sendall return type (int => long, supposed to be compatible)?

Modules/socketmodule.c

matrixise · 2019-03-18T22:57:00Z

I have made the requested changes; please review again

@serhiy-storchaka @vstinner

bedevere-bot · 2019-03-18T22:57:04Z

Thanks for making the requested changes!

@vstinner: please review the changes made to this pull request.

pablogsal · 2019-03-18T23:27:39Z

Modules/socketmodule.c


    while (remaining > 0) {
        unsigned int segment;

-        segment = (remaining >= SEGMENT_SIZE ? SEGMENT_SIZE : remaining);
+        segment = ((size_t)remaining >= SEGMENT_SIZE ? SEGMENT_SIZE : (unsigned int) remaining);


Wait, is this cast safe? In C99 ssize_t can be an unsigned long, unless I am missing something. I think you need:

Py_SAFE_DOWNCAST(remaining, Py_ssize_t, unsigned int)

SEGMENT_SIZE is 32 KiB: remaining is only casted for values between [0; 32 KiB]:

#define SEGMENT_SIZE (32 * 1024 -1)

pablogsal

LGTM

bpo-36337: Use ssize_t and size_t for socket.send()/sendall()

17bf06c

the-knights-who-say-ni added the CLA signed label Mar 18, 2019

bedevere-bot added the awaiting review label Mar 18, 2019

matrixise changed the title ~~bpo-36337: Use ssize_t and size_t for socket.send()/sendall()~~ [2.7] bpo-36337: Use ssize_t and size_t for socket.send()/sendall() Mar 18, 2019

kmiku7 reviewed Mar 18, 2019

View reviewed changes

Modules/socketmodule.c Outdated Show resolved Hide resolved

matrixise added 3 commits March 18, 2019 12:22

On Windows, the type of send() length argument is int, not size_t: ba…

4c59062

…ckport the solution from 3.x with Py_ssize_t

Fix on Windows

59c20ea

Fix PEP7

c78ef94

matrixise marked this pull request as ready for review March 18, 2019 12:18

vstinner requested changes Mar 18, 2019

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting review labels Mar 18, 2019

convert to long from ssize_t

b4e16ea

bedevere-bot added awaiting change review and removed awaiting changes labels Mar 18, 2019

vstinner requested changes Mar 18, 2019

View reviewed changes

bedevere-bot added awaiting changes and removed awaiting change review labels Mar 18, 2019

matrixise added 2 commits March 18, 2019 14:20

On VMS, sendsegmented(), len type is int, not Py_ssize_t

405a5d2

Improve the blurb entry

995c41d

bedevere-bot added awaiting change review and removed awaiting changes labels Mar 18, 2019

vstinner reviewed Mar 18, 2019

View reviewed changes

matrixise added 3 commits March 18, 2019 14:30

avoid a buffer overflow when len is greater than INT_MAX on VMS

3300ad6

use Py_ssize_t for sendsegmented

55f2b36

Improvement with Victor

40dbaf4

vstinner approved these changes Mar 18, 2019

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting change review labels Mar 18, 2019

pablogsal self-requested a review March 18, 2019 14:26

serhiy-storchaka reviewed Mar 18, 2019

View reviewed changes

Modules/socketmodule.c Outdated Show resolved Hide resolved

Improvement with the remark of Serhiy

84a8d9f

bedevere-bot added awaiting change review and removed awaiting merge labels Mar 18, 2019

pablogsal reviewed Mar 18, 2019

View reviewed changes

pablogsal approved these changes Mar 19, 2019

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting change review labels Mar 19, 2019

vstinner merged commit f70b884 into python:2.7 Mar 19, 2019

bedevere-bot removed the awaiting merge label Mar 19, 2019

Uh oh!

[2.7] bpo-36337: Use ssize_t and size_t for socket.send()/sendall() #12397

[2.7] bpo-36337: Use ssize_t and size_t for socket.send()/sendall() #12397

Uh oh!

Conversation

matrixise commented Mar 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Mar 18, 2019

Uh oh!

Uh oh!

vstinner Mar 18, 2019

Choose a reason for hiding this comment

Uh oh!

bedevere-bot commented Mar 18, 2019

Uh oh!

matrixise commented Mar 18, 2019

Uh oh!

bedevere-bot commented Mar 18, 2019

Uh oh!

vstinner Mar 18, 2019

Choose a reason for hiding this comment

Uh oh!

vstinner Mar 18, 2019

Choose a reason for hiding this comment

Uh oh!

bedevere-bot commented Mar 18, 2019

Uh oh!

matrixise commented Mar 18, 2019

Uh oh!

bedevere-bot commented Mar 18, 2019

Uh oh!

vstinner Mar 18, 2019

Choose a reason for hiding this comment

Uh oh!

matrixise Mar 18, 2019

Choose a reason for hiding this comment

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

matrixise commented Mar 18, 2019

Uh oh!

bedevere-bot commented Mar 18, 2019

Uh oh!

pablogsal Mar 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vstinner Mar 19, 2019

Choose a reason for hiding this comment

Uh oh!

pablogsal left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

matrixise commented Mar 18, 2019 •

edited

Loading

pablogsal Mar 18, 2019 •

edited

Loading