Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-20082: fix misbehavior of buffered writes to raw files in append mode #21729

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

embray
Copy link
Contributor

@embray embray commented Aug 4, 2020

This is a patch for a bug I apparently wrote many years ago but never converted to a PR.

https://bugs.python.org/issue20082

@embray
Copy link
Contributor Author

embray commented Aug 4, 2020

Hmm, this seems to suddenly be causing some failures I didn't expect :/

Lib/test/test_io.py Outdated Show resolved Hide resolved
Modules/_io/bufferedio.c Outdated Show resolved Hide resolved
Modules/_io/bufferedio.c Outdated Show resolved Hide resolved
Modules/_io/bufferedio.c Outdated Show resolved Hide resolved
Modules/_io/bufferedio.c Outdated Show resolved Hide resolved
Modules/_io/bufferedio.c Outdated Show resolved Hide resolved
Modules/_io/bufferedio.c Outdated Show resolved Hide resolved
@@ -1795,6 +1827,8 @@ _io_BufferedWriter___init___impl(buffered *self, PyObject *raw,
self->readable = 0;
self->writable = 1;

_bufferedwriter_set_append(self);
Copy link
Contributor

@ZackerySpytz ZackerySpytz Aug 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This call should be checked for failure.

@@ -2309,6 +2351,8 @@ _io_BufferedRandom___init___impl(buffered *self, PyObject *raw,
self->readable = 1;
self->writable = 1;

_bufferedwriter_set_append(self);
Copy link
Contributor

@ZackerySpytz ZackerySpytz Aug 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This call should be checked for failure.

@@ -1767,6 +1778,27 @@ _bufferedwriter_reset_buf(buffered *self)
self->write_end = -1;
}

static void
Copy link
Contributor

@ZackerySpytz ZackerySpytz Aug 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function should be checked for failure, so it should not return void.

Copy link
Contributor Author

@embray embray Aug 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not so sure. What is its failure state? It either sets self->appending = 1 or self->appending = 0. Neither case represents a failure or success. AFAICT all calls within it check for failure or success and are handled appropriately.

@embray
Copy link
Contributor Author

embray commented Aug 11, 2020

Resolved all the trivial code style issues. Remaining question is whether _bufferedwriter_set_append should have a return value (if so I'm not clear on what it should be).

I think the issue that was causing test failures is fixed.

@embray
Copy link
Contributor Author

embray commented Aug 11, 2020

Hmm, now the CI systems are all consistently showing a new test failure, but I cannot reproduce it locally even with the exact same test command they're running.

Update: I can reproduce them now by configuring --with-pydebug.

embray added 4 commits Aug 11, 2020
if the raw file is appending there is no reason to attempt to rewind it before
flushing
…t have a mode attribute) and ensure that the mode is in fact a string
@embray
Copy link
Contributor Author

embray commented Aug 12, 2020

I think this is ready now for the next round of review. Not sure why the Azure pipeline failed but it doesn't seem related.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants