TST, BUG: Re-raise MemoryError exception in test_large_zip's process#16890
Conversation
Since numpy#15893, test_large_zip's actual test is run in a child process, so when this test raises a MemoryError exception, the exception is lost and the @requires_memory decorator can't catch it to return an xfail. This commit uses a boolean variable in shared memory to flag if the exception was raised, and in that case, re-raise it in the parent process. Fixes numpy#16889
anirudh2290
left a comment
anirudh2290
left a comment
There was a problem hiding this comment.
Added a few nits, overall LGTM ! Thank you for the fix @antlarr-suse !
| def test_large_zip(self): | ||
| def check_large_zip(): | ||
| def check_large_zip(memoryerror_raised): | ||
| memoryerror_raised.value=False |
There was a problem hiding this comment.
nit:
| memoryerror_raised.value=False | |
| memoryerror_raised.value = False |
There was a problem hiding this comment.
Oops, of course! I don't know how they slipped by. Fixed.
| try: | ||
| np.savez(os.path.join(tmpdir, 'test.npz'), test_data=test_data) | ||
| except MemoryError: | ||
| memoryerror_raised.value=True |
There was a problem hiding this comment.
nit:
| memoryerror_raised.value=True | |
| memoryerror_raised.value = True |
charris
added
00 - Bug
05 - Testing
09 - Backport-Candidate
|
Thanks! Was just going to merge it, but I think we may want to put the try/except clause to include the array creation, which can error out almost as plausibly? |
Move the try...except block out so it also catches the exceptions raised when allocating test_data, as suggested by @seberg in numpy#16890
It never happened here, but indeed, in a system with less than ~2GBs available it can raise that exception too. I've added that change and adjusted the lines so they are within 80 chars width |
|
Thanks @antlarr will putting it in. Should be reasonable enough for the moment, although I suppose tweaking the decorator or skipping for 32bits could be an option on top of it. |
…umpygh-16890) Since numpy#15893, test_large_zip's actual test is run in a child process, so when this test raises a MemoryError exception, the exception is lost and the @requires_memory decorator can't catch it to return an xfail. This commit uses a boolean variable in shared memory to flag if the exception was raised, and in that case, re-raise it in the parent process. Fixes numpy#16889
5 participants
Since #15893, test_large_zip's actual test is run in a child process,
so when this test raises a MemoryError exception, the exception is
lost and the @requires_memory decorator can't catch it to return
an xfail.
This commit uses a boolean variable in shared memory to flag if
the exception was raised, and in that case, re-raise it in the
parent process.
Fixes #16889