CI: Add mambaforge to ci builds

[phofl]

No description provided.

[mroeschke]

Nice! Any sense of how much faster the environment building is?

[phofl]

The conda incubator takes around 3 minutes I think. With conda it takes around 4 in a best case scenario. Unfortunately we have a lot of builds where this takes more than 15, while mamba should stay at around 3 for them. It will keep our build times consistent.

Unfortunately we have 2 failing pipelines which do seem relevant, will check what this is about

[jonashaag]

Also add here https://github.com/pandas-dev/pandas/blob/master/.github/workflows/sdist.yml#L54 and here https://github.com/pandas-dev/pandas/blob/master/.github/workflows/asv-bot.yml#L44?

[phofl]

This is not finished yet. I will have to figure out how to fix the installation issues before adding this somewhere else.

[jonashaag]

FYI, on my machine (macOS) those tests run fine when using the script.

[phofl]

This is not a test issue per se. Mamba resolves the dependencies differently, which results in different versions. The 3.9 build installs arrow 2.0.1 instead of 6.0.0, causing the tests to fail. I think I found a solution for this, but have to look a bit further

[jonashaag]

What I meant to say is that for me Mamba seems to resolve correctly/in a way that doesn't break the tests.

[phofl]

With the 3.9 dependency file? Could you post the output of conda info?

[jonashaag]

@phofl see #44979, seems to work ok

[phofl]

Yeah, would want to avoid the pin, but we'll see. Will finish this tomorrow after the setuptools pin is merged

[phofl]

This is ready for review.

We have a few builds where creating the environment regularly takes more than one hour. Mamba does speed up the process a bit in general, but more important: it does not take forever in some instances.

The 3.9 envs are solved a bit weird. Hence I added the pyarrow pin, which avoids this

[lithomas1]

Can we avoid adding defaults as a channel here and elsewhere, if possible? Having multiple channels tends to slow things down a bit.

[phofl]

Yep, works.

[jreback]

do we still have pyarrow 1.* testing?

[phofl]

Yep, for example in Database / Linux_py38_IO (ci/deps/actions-38-db-min.yaml)

[jreback]

thanks @phofl