Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Crash in clear_weakref() in pydantic test suite, in 3.12.0rc1 and newer #108295

Open
mgorny opened this issue Aug 22, 2023 · 4 comments
Open

Crash in clear_weakref() in pydantic test suite, in 3.12.0rc1 and newer #108295

mgorny opened this issue Aug 22, 2023 · 4 comments
Labels
topic-typing type-crash A hard crash of the interpreter, possibly with a core dump

Comments

@mgorny
Copy link
Contributor

mgorny commented Aug 22, 2023 •
edited by github-actions bot

Crash report

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.12.0rc1+ (heads/3.12:149d70c254, Aug 22 2023, 16:11:47) [GCC 13.2.0]

What happened?

The pydantic test suite started causing pytest to segfault on exit recently, with Python 3.12. I've been able to bisect it to pydantic commit pydantic/pydantic@f5e3aa9 that looks relatively harmless. I've also been able to bisect CPython into commit 58f9c88:

gh-106403: Restore weakref support for TypeVar and friends (GH-106418)

I have been able to reproduce the problem with tip of main as well, though I have to note it's a bit of heisenbug. I've basically had a very lucky day today that I've managed to correctly bisect on the second attempt.

To reproduce:

git clone https://github.com/pydantic/pydantic/
cd pydantic
python -m venv .venv
. .venv/bin/activate
pip install -e . pytest dirty-equals
# may need to be repeated a few times
python -m pytest -o addopts= tests/test_generics.py --deselect tests/test_generics.py::test_partial_specification_name -s

Error messages

Backtrace 
Core was generated by `python -m pytest -o addopts= tests/test_generics.py --deselect tests/test_gener'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  clear_weakref (self=self@entry=0x7f0a9a171df0) at Objects/weakrefobject.c:62
62	        if (*list == self)
(gdb) bt
#0  clear_weakref (self=self@entry=0x7f0a9a171df0) at Objects/weakrefobject.c:62
#1  0x0000557c3663d843 in _PyWeakref_ClearRef (self=self@entry=0x7f0a9a171df0) at Objects/weakrefobject.c:102
#2  0x0000557c366ec77e in handle_weakrefs (unreachable=unreachable@entry=0x7ffeec923ad0, 
    old=old@entry=0x557c36a40f68 <_PyRuntime+76552>) at Modules/gcmodule.c:804
#3  0x0000557c366ecad3 in gc_collect_main (tstate=0x557c36a9e7b0 <_PyRuntime+459600>, generation=generation@entry=2, 
    n_collected=n_collected@entry=0x0, n_uncollectable=n_uncollectable@entry=0x0, nofail=nofail@entry=1) at Modules/gcmodule.c:1284
#4  0x0000557c366ed333 in _PyGC_CollectNoFail (tstate=tstate@entry=0x557c36a9e7b0 <_PyRuntime+459600>) at Modules/gcmodule.c:2135
#5  0x0000557c366c39bb in finalize_modules (tstate=tstate@entry=0x557c36a9e7b0 <_PyRuntime+459600>) at Python/pylifecycle.c:1602
#6  0x0000557c366c524e in Py_FinalizeEx () at Python/pylifecycle.c:1863
#7  0x0000557c366eb4f9 in Py_RunMain () at Modules/main.c:691
#8  0x0000557c366eb56e in pymain_main (args=args@entry=0x7ffeec923c10) at Modules/main.c:719
#9  0x0000557c366eb63d in Py_BytesMain (argc=<optimized out>, argv=<optimized out>) at Modules/main.c:743
#10 0x0000557c3655979e in main (argc=<optimized out>, argv=<optimized out>) at ./Programs/python.c:15
(gdb) p self
$1 = (PyWeakReference *) 0x7f0a9a171df0
(gdb) p *self
$2 = {ob_base = {{ob_refcnt = 1, ob_refcnt_split = {1, 0}}, ob_type = 0x557c38bda680}, wr_object = 0x7f0a9a171c70, wr_callback = 0x0, 
  hash = -1, wr_prev = 0x0, wr_next = 0x0, vectorcall = 0x557c3663aa08 <weakref_vectorcall>}
(gdb) p *list
Cannot access memory at address 0xfe15355ba7e0
(gdb) p list
$3 = (PyWeakReference **) 0xfe15355ba7e0
@mgorny mgorny added the type-crash A hard crash of the interpreter, possibly with a core dump label Aug 22, 2023
@Eclips4
Copy link
Member

Eclips4 commented Aug 22, 2023

cc @JelleZijlstra

@mgorny mgorny mentioned this issue Aug 22, 2023
Open
1 task
@mgorny
Copy link
Contributor Author

mgorny commented Aug 22, 2023

Oh, and this is also filed at pydantic/pydantic#7181. I'm not really 100% sure whether pydantic isn't doing something wrong that got exposed here.

@Eclips4
Copy link
Member

Eclips4 commented Aug 22, 2023

Oh, and this is also filed at pydantic/pydantic#7181. I'm not really 100% sure whether pydantic isn't doing something wrong that got exposed here.

It would be nice if you could provide a pure python code(without pydantic) which crashes =)

@mgorny
Copy link
Contributor Author

mgorny commented Aug 22, 2023

I'm afraid that's as far as I managed (reducing it to one test file). Even with that, there are times when I can't reproduce it a single time — and times when it fails all the time. Right now I'm at the former, so I have no clue how to move forward.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
topic-typing type-crash A hard crash of the interpreter, possibly with a core dump
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mgorny @AlexWaygood @Eclips4