Crash with recursive types

[dvarrazzo]

I just notice #731 closed and I thought giving it a go.

git clone git@github.com:psycopg/psycopg.git
cd psycopg
git reset --hard 82b84e33
python3 -m venv .venv
source .venv/bin/activate
pip install ./psycopg[test,dev]

# Install mypy at release-0.990
pip install "git+https://github.com/python/mypy.git@42a4c6978"

# Enable a recursive type
patch -p1 < HERE
diff --git a/psycopg/psycopg/abc.py b/psycopg/psycopg/abc.py
index 1edde3db..beab963d 100644
--- a/psycopg/psycopg/abc.py
+++ b/psycopg/psycopg/abc.py
@@ -30,9 +30,7 @@ Params: TypeAlias = Union[Sequence[Any], Mapping[str, Any]]
 ConnectionType = TypeVar("ConnectionType", bound="BaseConnection[Any]")
 PipelineCommand: TypeAlias = Callable[[], None]
 
-# TODO: make it recursive when mypy will support it
-# DumperKey: TypeAlias = Union[type, Tuple[Union[type, "DumperKey"]]]
-DumperKey: TypeAlias = Union[type, Tuple[type, ...]]
+DumperKey: TypeAlias = Union[type, Tuple[Union[type, "DumperKey"]]]
 
 # Waiting protocol types
 
HERE

mypy --show-traceback 2>&1 | tee mypy.log

Traceback

psycopg/psycopg/pq/misc.py:99: error: Redundant cast to "PGconn"  [redundant-cast]
psycopg/psycopg/adapt.py:94: error: Return type "Union[type, Tuple[type, ...]]" of "get_key" incompatible with return type "DumperKey" in supertype 
"Dumper"  [override]
Traceback (most recent call last):
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/bin/mypy", line 8, in <module>
    sys.exit(console_entry())
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/__main__.py", line 15, in console_entry
    main()
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/main.py", line 95, in main
    res, messages, blockers = run_build(sources, options, fscache, t0, stdout, stderr)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/main.py", line 174, in run_build
    res = build.build(sources, options, None, flush_errors, fscache, stdout, stderr)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/build.py", line 193, in build
    result = _build(
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/build.py", line 276, in _build
    graph = dispatch(sources, manager, stdout)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/build.py", line 2903, in dispatch
    process_graph(graph, manager)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/build.py", line 3287, in process_graph
    process_stale_scc(graph, scc, manager)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/build.py", line 3388, in process_stale_scc
    graph[id].type_check_first_pass()
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/build.py", line 2309, in type_check_first_pass
    self.type_checker().check_first_pass()
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 465, in check_first_pass
    self.accept(d)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/nodes.py", line 1143, in accept
    return visitor.visit_class_def(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 2132, in visit_class_def
    self.accept(defn.defs)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/nodes.py", line 1218, in accept
    return visitor.visit_block(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 2569, in visit_block
    self.accept(s)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 573, in accept
    stmt.accept(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/nodes.py", line 822, in accept
    return visitor.visit_func_def(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 938, in visit_func_def
    self._visit_func_def(defn)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 942, in _visit_func_def
    self.check_func_item(defn, name=defn.name)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 1011, in check_func_item
    self.check_func_def(defn, typ, name, allow_empty)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 1208, in check_func_def
    self.accept(item.body)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/checker.py", line 573, in accept

... # 28k lines...

  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/typestate.py", line 99, in is_assumed_subtype
    if get_proper_type(l) == get_proper_type(left) and get_proper_type(
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 2814, in get_proper_type
    typ = typ._expand_once()
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 277, in _expand_once
    return replace_alias_tvars(
psycopg/psycopg/_transform.py:73: error: INTERNAL ERROR -- Please try using mypy master on GitHub:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.990+dev.42a4c69789d855e2a07a5826b4e40331bc407e95
psycopg/psycopg/_transform.py:73: : note: use --pdb to drop into pdb
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 3229, in replace_alias_tvars
    new_tp = tp.accept(replacer)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 2592, in accept
    return visitor.visit_union_type(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 273, in visit_union_type
    return UnionType(self.translate_types(t.items), t.line, t.column)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 276, in translate_types
    return [t.accept(self) for t in types]
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 276, in <listcomp>
    return [t.accept(self) for t in types]
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 2172, in accept
    return visitor.visit_tuple_type(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 249, in visit_tuple_type
    self.translate_types(t.items),
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 276, in translate_types
    return [t.accept(self) for t in types]
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 276, in <listcomp>
    return [t.accept(self) for t in types]
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 2592, in accept
    return visitor.visit_union_type(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 273, in visit_union_type
    return UnionType(self.translate_types(t.items), t.line, t.column)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 276, in translate_types
    return [t.accept(self) for t in types]
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/type_visitor.py", line 276, in <listcomp>
    return [t.accept(self) for t in types]
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 321, in accept
    return visitor.visit_type_alias_type(self)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 3193, in visit_type_alias_type
    return typ.copy_modified(args=[t.accept(self) for t in typ.args])
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 354, in copy_modified
    return TypeAliasType(
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 262, in __init__
    super().__init__(line, column)
  File "/home/piro/dev/fs/mypy/tmp/psycopg/.venv/lib/python3.8/site-packages/mypy/types.py", line 208, in __init__
    super().__init__(line, column)
RecursionError: maximum recursion depth exceeded while calling a Python object

I tried the same using mypy 0.982 (adding enable_recursive_aliases = true in pyproject.toml) and it segfaults.

[JelleZijlstra]

cc @ilevkivskyi

[ilevkivskyi]

#14007 should fix this.

Btw @dvarrazzo this type looks suspicious:

• You don't need unions both inside and outside of the tuple, you can keep only one depending on whether plain type is needed or not.
• Maybe you are missing ...? Otherwise only valid values are nested 1-element tuples.

Probably you wanted something like

DumperKey: TypeAlias = Union[type, Tuple["DumperKey", ...]]

(which btw is not affected by this bug).

[dvarrazzo]

@ilevkivskyi Thank you for fixing, and you are right: our type definition (a type or an arbitrarily nested tuple of types) was wrong (it was untested, of course). With the type fixed, mypy 0.982 works correctly.