Skip to content

/ cpython

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-39481: WIP: PEP 585 #18239

Open
wants to merge 20 commits into
base: master
from
Open

bpo-39481: WIP: PEP 585 #18239

wants to merge 20 commits into from
+342 −0

Conversation

@gvanrossum
Copy link
Member

gvanrossum commented Jan 28, 2020 •
edited by bedevere-bot

https://bugs.python.org/issue39481
gvanrossum and others added 13 commits Jan 27, 2020
@gvanrossum
PEP 585 step 2: Write a Py_GenericAlias() function that returns origin
b4602c4
@gvanrossum
PEP 585 step 3: Add Py_GenericAlias() as __class_getitem__ to tuple, …
e843294 
…list, dict, set, frozenset
@gvanrossum
PEP 585 step 4: Write some tests
dc6bb67
@gvanrossum
Define a GenericAlias (proxy) object (with lots of TODOs still)
aa7d262
@gvanrossum
More tests
5c698b0
@gvanrossum
PEP 585 steps 5-6: Finish basic GenericAlias object and test
ea97476
@gvanrossum
Fix bug with class methods
ce78557
@gvanrossum
Define tp_new instead of tp_init
1bad422
@gvanrossum
Test that list[int][int] fails
68558f0
@gvanrossum
Test that class L(list): pass; L[int] works
347710c
@ethanhs @gvanrossum
Implement PEP 585 for collections.deque and ... (#2)
64c7d80 
... collections.defaultdict
@gvanrossum
index on pep585: 347710c Test that class L(list): pass; L[int] works
a324448
@gvanrossum
Merge commit 'a324448e5e' into pep585
Loading status checks…
2d5f115
@gvanrossum gvanrossum requested a review from serhiy-storchaka Jan 28, 2020
@gvanrossum gvanrossum requested review from methane and rhettinger as code owners Jan 28, 2020
@gvanrossum gvanrossum removed the request for review from rhettinger Jan 28, 2020
@serhiy-storchaka
serhiy-storchaka reviewed Jan 28, 2020
View changes
Objects/descrobject.c Outdated Show resolved Hide resolved
Objects/descrobject.c Outdated Show resolved Hide resolved
Objects/descrobject.c Outdated Show resolved Hide resolved
Objects/descrobject.c Outdated Show resolved Hide resolved
Objects/descrobject.c
};

static PyMemberDef ga_members[] = {
{"__origin__", T_OBJECT, offsetof(gaobject, origin), READONLY},

This comment has been minimized.

Sign in to view
Copy link
@serhiy-storchaka

serhiy-storchaka Jan 28, 2020

Member

Maybe T_OBJECT_EX?

This comment has been minimized.

Sign in to view
Copy link
@gvanrossum

gvanrossum Jan 28, 2020

Author Member

I think not. The two attributes should never be allowed to be NULL. (This is a change from an earlier design.)
Objects/descrobject.c Outdated Show resolved Hide resolved
Objects/descrobject.c Outdated
static PyObject *
ga_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
if (kwds != NULL) {

This comment has been minimized.

Sign in to view
Copy link
@serhiy-storchaka

serhiy-storchaka Jan 28, 2020

Member

It is worth to use Argument Clinic which generates simpler code using non-public C API similar to the following:

if (!_PyArg_NoKeywords("GenericAlias", kwds) ||
    !_PyArg_CheckPositional("GenericAlias", PyTuple_GET_SIZE(args), 2, 2))
{
    return NULL;
}
Objects/descrobject.c Outdated Show resolved Hide resolved
Objects/descrobject.c Outdated Show resolved Hide resolved
Objects/descrobject.c
};

static PyObject *
ga_new(PyTypeObject *type, PyObject *args, PyObject *kwds)

This comment has been minimized.

Sign in to view
Copy link
@serhiy-storchaka

serhiy-storchaka Jan 28, 2020

Member

Do we need a __new__ method? Why not set tp_new = NULL?

This comment has been minimized.

Sign in to view
Copy link
@gvanrossum

gvanrossum Jan 28, 2020

Author Member

I first had a tp_init. Then calling GenericAlias() somehow worked and produced an object with NULL attributes (or maybe it was some more clever incantation, I forget, but I definitely saw it). I do want the class instantiable from Python (so it can potentially be used from typing.py). Using tp_new instead solved that problem.
@gvanrossum
Respond to Serhiy's code review
Loading status checks…
6e95523
@gvanrossum gvanrossum changed the title WIP: PEP 585 bpo-39481: WIP: PEP 585 Jan 28, 2020
blurb-it bot and others added 6 commits Jan 28, 2020
@blurb-it
📜🤖 Added by blurb_it.
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
9e8fc68
@ethanhs
PEP 585 steps 7-8: Implement repr for GenericAlias (#1)
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
78415a2 
By Ethan Smith.
@ethanhs
Add static keyword to ga_repr_item (#3)
Verified
This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.
GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits
Loading status checks…
a125463
@gvanrossum
Add GenericAlias to types.py
Loading status checks…
6d77f0b
@gvanrossum
Add test for types.py
Loading status checks…
c2323a0 
Also strengthen test for repr(MyList[int])
@gvanrossum
Fix tests that were sensitive to the presence of __class_getitem__
Loading status checks…
3353bb8 
Note that UserDict and UserList are now generic
@gvanrossum

This comment has been minimized.

Sign in to view
Copy link
Member Author

gvanrossum commented Jan 29, 2020

So I'm looking into the test failures. I've fixed two simple ones, but now I'm stuck with at least two:

  • test_typing.py, see below
  • test_genericalias.py (i.e. the very PEP 585 test) crashes on Ubuntu, but not for me locally

The problem with test_typing.py is that when we have a class that derives from Tuple[xxx], like so:

from typing import Tuple
class C(Tuple[int]): pass
print(C.mro())  # C, tuple, typing.Generic, object

its MRO contains tuple before Generic, and the new tuple.__class_getitem__ shadows the intended Generic.__class_getitem__. I'm not sure what to do about this. @ambv @ilevkivskyi @serhiy-storchaka
@gvanrossum

This comment has been minimized.

Sign in to view
Copy link
Member Author

gvanrossum commented Jan 29, 2020

Using a debug build I can repro the test_genericalias.py crash. The upper few lines of the stack trace are:

* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGABRT
  * frame #0: 0x00007fff72e217fa libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff72edebc1 libsystem_pthread.dylib`pthread_kill + 432
    frame #2: 0x00007fff72da8a1c libsystem_c.dylib`abort + 120
    frame #3: 0x00007fff72da7cd6 libsystem_c.dylib`__assert_rtn + 314
    frame #4: 0x00000001000df936 python.exe`_PyType_Lookup(type=0x0000000100401d30, name=0x00000001035f1d60) at typeobject.c:3155:5
    frame #5: 0x00000001000c0b8f python.exe`_PyObject_GenericGetAttrWithDict(obj=0x0000000100401ec8, name=0x00000001035f1d60, dict=0x0000000000000000, suppress=0) at object.c:1319:13
    frame #6: 0x00000001000c0ab3 python.exe`PyObject_GenericGetAttr(obj=0x0000000100401ec8, name=0x00000001035f1d60) at object.c:1407:12
    frame #7: 0x00000001000c0268 python.exe`PyObject_GetAttr(v=0x0000000100401ec8, name=0x00000001035f1d60) at object.c:1013:16
    frame #8: 0x00000001000c01a4 python.exe`PyObject_GetAttrString(v=0x0000000100401ec8, name="__module__") at object.c:918:11
    frame #9: 0x0000000100056154 python.exe`ga_repr_item(writer=0x00007ffeefbe9078, p=0x0000000100401ec8) at descrobject.c:1828:24
    frame #10: 0x0000000100054015 python.exe`ga_repr(self=0x00000001040349b0) at descrobject.c:1890:13
    frame #11: 0x00000001000beefb python.exe`PyObject_Repr(v=0x00000001040349b0) at object.c:543:11
    frame #12: 0x00000001001e9d49 python.exe`builtin_repr(module=0x0000000101221dd0, obj=0x00000001040349b0) at bltinmodule.c:2141:12

@ethanhs does this give you a clue? Methinks there's a refcount bug in ga_repr(). :-(
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@serhiy-storchaka serhiy-storchaka

@methane methane

Assignees
No one assigned
Labels
CLA signed DO-NOT-MERGE awaiting core review
Projects
None yet
Milestone
No milestone
5 participants
@gvanrossum @serhiy-storchaka @the-knights-who-say-ni @bedevere-bot @ethanhs
You can’t perform that action at this time.