type annotated arguments in zip function lead to wrong return types when using the star operator #9143

hoelzeli · 2022-11-09T18:22:11Z

This issue is a followup to the following issue:
microsoft/pylance-release#3598

Here's the code that is wrongly annotated:

test_list: list[tuple[int, str]] = [(1, 'testa'), (2, 'testb')]

a: tuple[int]
b: tuple[str]

a, b = zip(*test_list)

Pylance warns about an assignment of typle[int | str] to the tuples a and b.
The tuple annotation is being converted to an Iterator annotation, which does not support a sequence of types (see Eric's reply to the previous issue for details). Therefore, the resulting tuples after the unpacking operation are not as precisely typed as they could be.

And here is the typeshed code that leads to the 'wrong' type annotations:

typeshed/stdlib/builtins.pyi

Lines 1673 to 1715 in 7b3fff7

    
           class zip(Iterator[_T_co], Generic[_T_co]): 
        
               if sys.version_info >= (3, 10): 
        
                   @overload 
        
                   def __new__(cls, __iter1: Iterable[_T1], *, strict: bool = ...) -> zip[tuple[_T1]]: ... 
        
                   @overload 
        
                   def __new__(cls, __iter1: Iterable[_T1], __iter2: Iterable[_T2], *, strict: bool = ...) -> zip[tuple[_T1, _T2]]: ... 
        
                   @overload 
        
                   def __new__( 
        
                       cls, __iter1: Iterable[_T1], __iter2: Iterable[_T2], __iter3: Iterable[_T3], *, strict: bool = ... 
        
                   ) -> zip[tuple[_T1, _T2, _T3]]: ... 
        
                   @overload 
        
                   def __new__( 
        
                       cls, 
        
                       __iter1: Iterable[_T1], 
        
                       __iter2: Iterable[_T2], 
        
                       __iter3: Iterable[_T3], 
        
                       __iter4: Iterable[_T4], 
        
                       *, 
        
                       strict: bool = ..., 
        
                   ) -> zip[tuple[_T1, _T2, _T3, _T4]]: ... 
        
                   @overload 
        
                   def __new__( 
        
                       cls, 
        
                       __iter1: Iterable[_T1], 
        
                       __iter2: Iterable[_T2], 
        
                       __iter3: Iterable[_T3], 
        
                       __iter4: Iterable[_T4], 
        
                       __iter5: Iterable[_T5], 
        
                       *, 
        
                       strict: bool = ..., 
        
                   ) -> zip[tuple[_T1, _T2, _T3, _T4, _T5]]: ... 
        
                   @overload 
        
                   def __new__( 
        
                       cls, 
        
                       __iter1: Iterable[Any], 
        
                       __iter2: Iterable[Any], 
        
                       __iter3: Iterable[Any], 
        
                       __iter4: Iterable[Any], 
        
                       __iter5: Iterable[Any], 
        
                       __iter6: Iterable[Any], 
        
                       *iterables: Iterable[Any], 
        
                       strict: bool = ..., 
        
                   ) -> zip[tuple[Any, ...]]: ...

I'm not sure if it is currently possible to fix this. I had the idea of adding a type hint to the *args keyword, so something like this:

@overload
        def __new__(cls, __iter1: _T2:=Sequence[_T1], *args: _T2, strict: bool = ...) -> zip[tuple[_T1]]:

I'm unsure if this is valid python syntax and if it would result in the desired behavior. I'm also unsure how python would decide between the proposed constructor and the one that is currently being used during typechecking my example:

typeshed/stdlib/builtins.pyi

Lines 1675 to 1676 in 7b3fff7

    
           @overload 
        
           def __new__(cls, __iter1: Iterable[_T1], *, strict: bool = ...) -> zip[tuple[_T1]]: ...

The text was updated successfully, but these errors were encountered:

sobolevn · 2022-11-16T22:10:49Z

This code works for me:

from typing_extensions import assert_type

test_list: list[tuple[int, str]] = [(1, 'testa'), (2, 'testb')]

a: tuple[int, ...]  # note
b: tuple[str, ...]

a, b = zip(*test_list)
assert_type(a, tuple[int, ...])
assert_type(b, tuple[str, ...])

However, it does not work without explicit a and b annotations:

Running mypy --platform darwin --python-version 3.10 on the standard library test cases... failure

test_cases/stdlib/builtins/check_zip.py:9: error: Expression is of type "Tuple[Any, ...]", not "Tuple[int, ...]"  [assert-type]
test_cases/stdlib/builtins/check_zip.py:10: error: Expression is of type "Tuple[Any, ...]", not "Tuple[str, ...]"  [assert-type]

JelleZijlstra · 2022-11-25T04:44:45Z

A reduced version of the problem is:

test_list: list[tuple[int, str]] = []
reveal_type(zip(*test_list))

Mypy infers this as zip[tuple[Any, ...]], pyright as zip[tuple[int | str]]. Mypy's answer works better here, but pyright's overload heuristic has other advantages and probably this cannot be fixed in pyright.

Ideally we'd write the stub in such a way that it works well for all major type checkers, but I'm not sure how to do that in this case without regressing support for other important use cases.

erictraut mentioned this issue Nov 12, 2022

zip two Tuple[*Ps]s results wrong sized Tuple microsoft/pyright#4186

Closed

type annotated arguments in zip function lead to wrong return types when using the star operator #9143

type annotated arguments in zip function lead to wrong return types when using the star operator #9143

hoelzeli commented Nov 9, 2022

sobolevn commented Nov 16, 2022 •

edited

JelleZijlstra commented Nov 25, 2022

type annotated arguments in zip function lead to wrong return types when using the star operator #9143

type annotated arguments in zip function lead to wrong return types when using the star operator #9143

Comments

hoelzeli commented Nov 9, 2022

sobolevn commented Nov 16, 2022 • edited

JelleZijlstra commented Nov 25, 2022

sobolevn commented Nov 16, 2022 •

edited