TYP: future annotations

[twoertwein]

• Closes Use PEP 604 style annotations #150
• Tests added: Please use assert_type() to assert the type of any return value

[twoertwein]

I will try to separate it in two commits: automatic (sed+pre-commit) and then manual changes to make it easier to review

[Dr-Irv]

I will try to separate it in two commits: automatic (sed+pre-commit) and then manual changes to make it easier to review

Thanks. I'll review when you do that.

Did you just do sed to convert everything? Or use some other tool?

[twoertwein]

The new PR will:

• add future to all files using sed and run pre-commit
• manually remove unused imports (and workaround mypy bugs: def type() causes issues)

This will not satisfy flake8-pyi as pre-commit will not rewrite Union/... outside of annotations (TypeAliases), I had many ugly sed's in this PR but I think I will skip this and try to address that later.

[twoertwein]

• first commit: sed -i '1s;^;from __future__ import annotations\n;' + pre-commit + isort + black
• second commit: removing unused imports and getting mypy/pyright to pass
• thrid commit: manually chaning offending (flake8-pyi Y022) Union/List/Dict/Tuple/Set cases
• fourth commit: remove unused flake8 ignore comments

pyi files do not need the future import but without it pre-commit while not re-write the annotations.

[bashtage]

2, 3, and 4 all look good. 1 is harder to check, but probably can be trusted.

[bashtage]

Same

[Dr-Irv]

I somehow stared at all of these. Not sure I caught everything. General comments:

1. In pandas we have type_t in pandas/_typing.py, so we can then do things like type_t[Dtype], etc. So I think we should use that convention here rather than type[Dtype]
2. There are places where a parameter has one line of its type, but parentheses got inserted and are unnecessary. I caught many of them, but may have missed a few
3. There are places where we have None | str | int (or similar pattern), where we should use the convention of str | int | None

There was one place where a type was dropped. Otherwise OK.

[twoertwein]

But neither Dict nor List is at those lines? (locally poe test_all works for me - on 3.10)

pandas-stubs/core/frame.pyi:194: error: Name "Dict" is not defined [name-defined]
pandas-stubs/core/frame.pyi:194: note: Did you forget to import it from "typing"? (Suggestion: "from typing import Dict")
pandas-stubs/core/frame.pyi:197: error: Name "List" is not defined [name-defined]
pandas-stubs/core/frame.pyi:197: note: Did you forget to import it from "typing"? (Suggestion: "from typing import List")

[twoertwein]

Is there a bug in the git checkout? The CI has:

$ /usr/bin/git log -1 --format='%H'
'4a93f11ab848da036b9afe2c33b46839307d85e3'

But locally I have:

$ git log -1 --format='%H'
0433b15f2cf22bc85c5c1920c714212fd3daa919

I'll squash the commits when it started falling (maybe the commit message triggers some weird bug - probably something different is wrong but I cannot see it)

[twoertwein]

Since I cannot reproduce the error locally, I will revert until the latest working commit and then add slowly make changes to see what is going on.

[Dr-Irv]

You can retrigger the CI from github. I had a failure due to network connectivity, and just went it and told it to rerun the jobs.

[twoertwein]

You can retrigger the CI from github. I had a failure due to network connectivity, and just went it and told it to rerun the jobs.

Thanks! It seems that github requires that the branch has no conflict with main (that might also explain why Dict and List appeared out of nowhere):
https://github.community/t/run-actions-on-pull-requests-with-merge-conflicts/17104/26

I resolved all conflicts.

[Dr-Irv]

Let me know when I should look at this again.

[Dr-Irv]

So I think there is a better workaround for the type issue, where we define type_t once, export it from _typing.pyi and then import type_t instead of always creating _type

Otherwise, a few small things.

[Dr-Irv]

I think we should keep using _str in frame.pyi so that the methods that overlap with series.pyi have similar signatures, since we have to use _str in series.pyi

[Dr-Irv]

Thanks @twoertwein . I now have to get used to not using Union, Dict, List, etc. !