Skip to content

CI: Mark slow tests as slow to speed up ci #41140

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 26, 2021
Merged

Conversation

phofl
Copy link
Member

@phofl phofl commented Apr 24, 2021

We have a few tests which are really slow and not marked as such. I went through the list below and marked everything greater than 4 seconds as slow. Should give us between 4 and 5 minutes performance boost

# 29.68s call pandas/tests/io/sas/test_sas7bdat.py::TestSAS7BDAT::test_iterator_loop # 12.98s call pandas/tests/test_sorting.py::TestSorting::test_int64_overflow_moar # 11.43s call pandas/tests/indexing/multiindex/test_chaining_and_caching.py::test_indexer_caching # 10.13s call pandas/tests/groupby/transform/test_transform.py::test_cython_transform_frame[shift-args3-] # 9.87s call pandas/tests/io/sas/test_sas7bdat.py::TestSAS7BDAT::test_from_iterator # 9.79s call pandas/tests/groupby/transform/test_transform.py::test_cython_transform_frame[shift-args2-] # 9.41s teardown pandas/tests/window/moments/test_moments_rolling_quantile.py::test_center_reindex_frame[1.0] # 8.82s call pandas/tests/io/sas/test_xport.py::TestXport::test1_basic # 8.43s call pandas/tests/io/parser/common/test_chunksize.py::test_warn_if_chunks_have_mismatched_type[python] # 8.38s call pandas/tests/io/parser/common/test_chunksize.py::test_chunks_have_consistent_numerical_type[python] # 8.13s call pandas/tests/io/parser/test_c_parser_only.py::test_grow_boundary_at_cap[c_low] # 8.12s call pandas/tests/io/parser/test_c_parser_only.py::test_grow_boundary_at_cap[c_high] # 6.97s call pandas/tests/indexes/datetimes/test_date_range.py::TestDateRanges::test_date_range_int64_overflow_stride_endpoint_different_signs # 5.90s call pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_mrecarray # 5.84s teardown pandas/tests/window/moments/test_moments_rolling_skew_kurt.py::test_center_reindex_frame[skew] # 5.61s call pandas/tests/io/parser/test_c_parser_only.py::test_precise_conversion[c_high] # 5.29s call pandas/tests/io/sas/test_sas7bdat.py::TestSAS7BDAT::test_path_pathlib # 5.13s call pandas/tests/groupby/transform/test_transform.py::test_cython_transform_frame[cumsum-args1-] # 4.93s call pandas/tests/io/sas/test_sas7bdat.py::TestSAS7BDAT::test_from_file # 4.86s call pandas/tests/io/sas/test_sas7bdat.py::TestSAS7BDAT::test_from_buffer # 4.84s call pandas/tests/io/sas/test_sas7bdat.py::TestSAS7BDAT::test_path_localpath # 4.59s call pandas/tests/indexes/multi/test_integrity.py::test_hash_collisions # 4.26s call pandas/tests/indexing/interval/test_interval.py::TestIntervalIndex::test_loc_getitem_large_series # 3.99s call pandas/tests/io/parser/test_c_parser_only.py::test_precise_conversion[c_low]

@phofl phofl added CI Continuous Integration Testing pandas testing functions or related to the test suite labels Apr 24, 2021
@phofl phofl changed the title Mark slow tests as slow to speed up ci CI: Mark slow tests as slow to speed up ci Apr 24, 2021
@jbrockmendel
Copy link
Member

some of these occasionally fail for unknown reasons, e.g. im pretty sure ive seen test_warn_if_chunks_have_mismatched_type and test_int64_overflow_moar in the logs. can we keep those? or possibly (i know this is much more labor-intensive) try to trim down these tests?

@phofl
Copy link
Member Author

phofl commented Apr 24, 2021

Do you know if test_warn_if_chunks_have_mismatched_type fails for the python or c engine? The python is the slow one

Added them back in

@jbrockmendel
Copy link
Member

Do you know if test_warn_if_chunks_have_mismatched_type fails for the python or c engine? The python is the slow one

not off the top of my head, no

@jreback jreback added this to the 1.3 milestone Apr 26, 2021
@jreback jreback merged commit 4854605 into pandas-dev:master Apr 26, 2021
@jreback
Copy link
Contributor

jreback commented Apr 26, 2021

thanks @phofl

@phofl phofl deleted the slow_tests branch April 26, 2021 21:58
yeshsurya pushed a commit to yeshsurya/pandas that referenced this pull request May 6, 2021
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants