Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-98778: Update HTTPError to initialize properly even if fp is None #99966

Merged
merged 3 commits into from Dec 8, 2022

Conversation

corona10
Copy link
Member

@corona10 corona10 commented Dec 3, 2022

@corona10
Copy link
Member Author

corona10 commented Dec 3, 2022

Since the _TemporaryFileCloser is used for internal usage, this change will not impact to 3rd party libraries.

@corona10
Copy link
Member Author

corona10 commented Dec 3, 2022


(.oss) ➜  cpython git:(gh-98778) ✗ ./python.exe -m test test_urllib2 -R 3:3
Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 1.44 Run tests sequentially
0:00:00 load avg: 1.44 [1/1] test_urllib2
beginning 6 repetitions
123456
......

== Tests result: SUCCESS ==

1 test OK.

Total duration: 1.5 sec
Tests result: SUCCESS

@netlify
Copy link

netlify bot commented Dec 7, 2022

Deploy Preview for python-cpython-preview canceled.

Name Link
🔨 Latest commit c3a75a1
🔍 Latest deploy log https://app.netlify.com/sites/python-cpython-preview/deploys/63913fd79fffe30008780f7a

Copy link
Contributor

@ambv ambv left a comment

You made changes to _TemporaryFileCloser but urllib.response.addinfourl here is a subclass of _TemporaryFileWrapper. Now after your changes the .close() and .cleanup() operations work but the wrapper is still in an inconsistent state. It cannot be used as a context manager, and cannot be iterated. The wrapper proxies attribute access to the file handle so asking things like exc.mode, exc.seekable(), etc. will all fail.

The cleanest solution would be to change HTTPError to not subclass this thing at all... but that ship has sailed a long time ago.

Your change weakens _TemporaryFileCloser by allowing it to silently accept None in lieu of a file handle. This makes tempfile worse, I'm not convinced this is a good compromise.

Instead, since HTTPError is already a hack (and there was literally a comment about why it's hacky where the bug was created), I would just replace the hack with a different one: set fp to io.StringIO() if the passed fp is None. It's still not perfect but I feel that would be more robust.

@bedevere-bot
Copy link

bedevere-bot commented Dec 7, 2022

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

@corona10
Copy link
Member Author

corona10 commented Dec 7, 2022

Instead, since HTTPError is already a hack (and there was literally a comment about why it's hacky where the bug was created), I would just replace the hack with a different one: set fp to io.StringIO() if the passed fp is None. It's still not perfect but I feel that would be more robust.

+1

@corona10
Copy link
Member Author

corona10 commented Dec 7, 2022

@ambv

Thank you for the review and for pointing out what I missed :)

I have made the requested changes; please review again

@bedevere-bot
Copy link

bedevere-bot commented Dec 7, 2022

Thanks for making the requested changes!

@ambv: please review the changes made to this pull request.

@bedevere-bot bedevere-bot requested a review from ambv Dec 7, 2022
ambv
ambv approved these changes Dec 7, 2022
Lib/urllib/error.py Outdated Show resolved Hide resolved
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
@corona10 corona10 merged commit dc8a868 into python:main Dec 8, 2022
20 checks passed
@miss-islington
Copy link
Contributor

miss-islington commented Dec 8, 2022

Thanks @corona10 for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒🤖

@corona10 corona10 deleted the gh-98778 branch Dec 8, 2022
@bedevere-bot
Copy link

bedevere-bot commented Dec 8, 2022

GH-100096 is a backport of this pull request to the 3.11 branch.

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 8, 2022
… None (pythongh-99966)

(cherry picked from commit dc8a868)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Dec 8, 2022
… None (pythongh-99966)

(cherry picked from commit dc8a868)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
@bedevere-bot
Copy link

bedevere-bot commented Dec 8, 2022

GH-100097 is a backport of this pull request to the 3.10 branch.

miss-islington added a commit that referenced this pull request Dec 8, 2022
…h-99966)

(cherry picked from commit dc8a868)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
miss-islington added a commit that referenced this pull request Dec 8, 2022
…h-99966)

(cherry picked from commit dc8a868)

Co-authored-by: Dong-hee Na <donghee.na@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants