TYP: use Self for type checking

[topper-123]

We recently upgraded the mypy requirement to mypy v1.0. One of the new capabilities of mypy 1.0 is that it can now type check the Self type. I propose we use Self for self type checking in Pandas instead if as we currently do, using TypeVars.

One issue with type checking using Self, is that Self was added only in python 3.11, while Pandas supports Python >= 3.8. However, Self has been added to typing_extensions , so if we add type_extensions as a Pandas dependency for Python <3.11, we can use Self for type checking purposes in Pandas.

[twoertwein]

Pandas-stubs is using typing_extensions.Self now and is enforcing it with flake8-pyi (checks only pyi files).

If enforcing Self is important, it might be good to wait until ruff implements more of the flake8-pyi checks and hope that they also implement it for regular py files.

[topper-123]

The main benefit of adding Self is helping mypy (and maybe pyright) catch typing errors in the pandas code base. This is different than the stated purpose of ruff. They say:

Ruff is a linter, not a type checker.

and

It's recommended that you use Ruff in conjunction with a type checker, like Mypy, Pyright, or Pyre, with Ruff providing faster feedback on lint violations and the type checker providing more detailed feedback on type errors.

So I don't think this should be connected to updates to ruff.

[twoertwein]

I'm all for Self but it would also be nice to enforce it: you can achieve exactly the same with TypeVars already - so it would be nice to discourage TypeVars in cases where Self would be a more readable alternative (which flake-8pyi does for pyi files). Obviously, we could already start using Self (typing_extensions wouldn't even need to be a runtime dependency).

[twoertwein]

I guess a first step would be to enable Y019 by removing it here https://github.com/pandas-dev/pandas/blob/main/setup.cfg#L19
to at least use typing_extensions.Self in pyi files.

[simonjayhawkins]

I'm all for Self but it would also be nice to enforce it: you can achieve exactly the same with TypeVars already - so it would be nice to discourage TypeVars in cases where Self would be a more readable alternative (which flake-8pyi does for pyi files).

If Self is no more than syntactic sugar then this is probably not a strong enough argument alone to adopt typing_extensions for the pandas codebase at this time.

But I'm sure that other features such as TypeGuard could be #51309

Obviously, if/when typing_extensions is a dev-only dependency for pandas, then it makes sense to keep the pandas types annotations consistent with pandas-stubs and use Self.

But for now, -1