CI: network tests running even with "not network"

[mzeitlin11]

While looking into windows timeouts, noticed network tests were still being run with -m "not network". tm.network adds the keyword network, but does not add a mark, so -m "not network" was still running network tests. While a mark could probably be set somehow in that decorator, I couldn't find a pytest documented way to do that.

Instead, this pushes pytest_runtest_setup into pytest_collection_modifyitems (which I think is more intuitive since we know if a test should be skipped or not at collection time, and shouldn't need these skip checks at setup-time for each test).

The other reason for this change was that pytest_runtest_setup is too late in the pipeline to add the network marker. By moving the skip logic to pytest_collection_modifyitems, we can consolidate the skip checks with adding the corresponding marker if the "network" keyword is found.

[jreback]

can you confirm that this doesn't change the number of tests run (e.g. compare a couple of builds which uses these keywords vs master)

[mzeitlin11]

Some different builds and results in the details block below. These look as expected I think - the patterns with "not network" run slightly fewer depending on the subset of tests included since "not network" is now respected.

The one with the "slow" pattern (the only one not specifying "not network") has an exactly matching number of tests

macOS py38_macos_1
Cmd: pytest -m "not slow and not network
69160 passed, 4425 skipped, 646 xfailed  # pr
69161 passed, 4425 skipped, 645 xfailed  # master

Windows py38_np18_2
Cmd: pytest -m "not slow and not network
53929 passed, 2511 skipped, 400 xfailed, 8 xpassed  # pr
53966 passed, 2523 skipped, 399 xfailed, 8 xpassed  # master

Linux_py38_IO
Cmd: pytest -m "((not slow and not network and not clipboard) or (single and db))
9808 passed, 1313 skipped, 214 xfailed  # pr
9845 passed, 1321 skipped, 214 xfailed  # master

pytest (actions-38-minimum_versions
Cmd: -m "not slow and not network and not clipboard"
170922 passed, 7056 skipped, 1047 xfailed, 5 xpassed  # pr
170948 passed, 7079 skipped, 1046 xfailed, 5 xpassed  # master

actions-39-slow.yaml
Cmd: -m "slow"
2436 passed, 11 skipped, 6 xfailed, 5 xpassed  # pr
2436 passed, 11 skipped, 6 xfailed, 5 xpassed  # master

[jreback]

thanks @mzeitlin11