Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

typing._type_check no longer recognizes several kinds of non-types #92601

Closed
davidfstr opened this issue May 10, 2022 · 6 comments
Closed

typing._type_check no longer recognizes several kinds of non-types #92601

davidfstr opened this issue May 10, 2022 · 6 comments
Labels
3.11 3.12 expert-typing stdlib type-bug

Comments

@davidfstr
Copy link
Contributor

@davidfstr davidfstr commented May 10, 2022

Bug report

The internal function typing._type_check() is used extensively to check whether a particular value represents a type. Its docstring begins with:

def _type_check(arg, msg, is_argument=True, module=None, *, allow_special_forms=False):
    """Check that the argument is a type, and return it (internal helper).

In Python 3.11b1 this function no longer errors on various kinds of non-types:

>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=11, micro=0, releaselevel='beta', serial=1)
>>> import typing
>>> typing._type_check(5, "Expected a type.")
# no error!
>>> typing.ClassVar[5]
typing.ClassVar[5]  # no error!

By contrast on Python 3.10:

>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=10, micro=1, releaselevel='final', serial=0)
>>> import typing
>>> typing._type_check(5, "Expected a type.")
TypeError: Expected a type. Got 5.
>>> typing.ClassVar[5]
TypeError: typing.ClassVar accepts only single type. Got 5.

Your environment

Regression introduced somewhere between Python 3.10 and Python 3.11b1.

@davidfstr davidfstr added the type-bug label May 10, 2022
@davidfstr
Copy link
Contributor Author

@davidfstr davidfstr commented May 10, 2022

@AlexWaygood AlexWaygood added stdlib 3.11 3.12 labels May 10, 2022
@Fidget-Spinner
Copy link
Member

@Fidget-Spinner Fidget-Spinner commented May 10, 2022

I think commit 870b22b changed this behavior. It's intentional. Please see the explanation at #31151 (review).

@davidfstr
Copy link
Contributor Author

@davidfstr davidfstr commented May 10, 2022

I think commit 870b22b changed this behavior. It's intentional.

Understood. I'm currently using typing._type_check() inside the trycast library - which is a runtime type checker - to provide better error messages to users who mistakenly pass a regular Python value where a typeform is expected. For example the following common case:

trycast(int, 5)  # OK
trycast(5, int)  # ERROR: TypeError: trycast() requires a type as its first argument. Got 5.

It looks like typing._type_check() was intentionally loosened to not require the maybe-typeform it was checking to be callable so that:

  1. ParamSpecArgs can be used inside Annotation,
  2. InitVar can be used inside Annotation,
  3. future typeform-likes can be used inside Annotation

It was understood that loosening this check would cause _type_check() to potentially deviate from how static type checkers recognize a "type", but that giving additional flexibility to the runtime typing implementation was considered to be a good tradeoff.

My particular usage of typing._type_check() in trycast desires to only allow garden-variety/universal typeforms (that are generally useable anywhere) but not "only-allowed-in-special-places" typeforms like ParamSpecArgs or InitVar. And it also doesn't want to allow "definitely-not-typeforms" like values of {int, float, bool} (ex: 5).

I would argue that questionable constructions like ClassVar[5] also would benefit from disallowing "definitely-not-typeforms" as well, as they did in Python ≤ 3.10.

Suggested remediations at the typing.py layer:

  • Revert typing._type_check() to only permit "universal" typeforms (than can be used anywhere) by default, and have things like Annotated[] explicitly allow non-universal typeforms via something like typing._type_check(maybe_typeform, allow_nonuniversal=False).
    • Pro: The majority of typing._type_check() callers revert to disallowing non-typeforms and non-universal typeforms, with certain special callers like Annotated[] being able to opt-in to them.
    • Detail: Ways that a "universal" typeform could be recognized:
      • Proposal 1: callable(maybe_typeform) # former status-quo
      • Proposal 2: isinstance(maybe_typeform, (type, typing._SpecialForm))
      • Proposal 3: isinstance(maybe_typeform, typing._UniversalSpecialForm), where typing._UniversalSpecialForm is a new ABC, which at least {type, typing._SpecialForm} implement
  • Add a deny-list to typing._type_check() to guard against "definitely-not-typeforms" like values of {int, float, bool}
    • Pro: Provides better best-effort recognition (and denial) of non-typeforms than the current implementation.
    • Con: A deny-list will necessarily be incomplete.
    • Con: A deny-list, since it will add potentially a lot of extra checks, will be slow to execute at runtime.

Suggested remediations at the trycast layer (for @davidfstr to consider if no remediations at the typing.py layer are accepted):

  • Alter trycast to use a wrapper around typing._type_check() that reverts it to the Python ≤3.10 behavior of disallowing non-callables (which is currently a very reasonable way to filter out non-universal typeforms).

@davidfstr
Copy link
Contributor Author

@davidfstr davidfstr commented May 10, 2022

CC @GBeauregard as the originator of bpo-46644
CC @AlexWaygood as a +1er to @JelleZijlstra 's comment on that PR to accept it

@JelleZijlstra
Copy link
Member

@JelleZijlstra JelleZijlstra commented May 10, 2022

This was an intentional change to reduce the number of bugs where typing.py would reject current or future exotic but valid type annotations.

Also, _type_check is an internal implementation detail, subject to change at any time. It's also not very good at recognizing valid type annotations: things like ClassVar[len] were always accepted. You should implement your own, more precise, check for valid types in your library.

@davidfstr
Copy link
Contributor Author

@davidfstr davidfstr commented May 11, 2022

You should implement your own, more precise, check for valid types in your library.

Will do. Thanks for taking a look!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.11 3.12 expert-typing stdlib type-bug
Projects
None yet
Development

No branches or pull requests

5 participants