Rust 1.81-nightly slower than Rust 1.79 ?

[GuillaumePressiat]

Hi,

I've build my rust project with Rust 1.79 and now test with Rust 1.81 nightly.

I observe slower runtime.

This project is about parsing text files with polars.
Cargo.toml is the same in both context.

With Rust 1.79

# cargo build release with 1.79.0
cargo +1.79.0 build --release      
...

# benchmark on release run 
bench ./target/release/pmsi_rust   
benchmarking ./target/release/pmsi_rust
time                 236.9 ms   (226.6 ms .. 249.0 ms)
                     0.998 R²   (0.991 R² .. 1.000 R²)
mean                 232.1 ms   (228.1 ms .. 236.8 ms)
std dev              5.880 ms   (3.339 ms .. 8.859 ms)
variance introduced by outliers: 14% (moderately inflated)

With Rust 1.81

# With 1.81 version
% cargo --version      
cargo 1.81.0-nightly (bc89bffa5 2024-06-22)
# build release with 1.81
% cargo build --release           
...
# benchmark on release run
% bench ./target/release/pmsi_rust
benchmarking ./target/release/pmsi_rust
time                 614.8 ms   (528.0 ms .. 710.4 ms)
                     0.997 R²   (0.989 R² .. 1.000 R²)
mean                 644.2 ms   (622.9 ms .. 664.8 ms)
std dev              25.50 ms   (11.26 ms .. 31.11 ms)
variance introduced by outliers: 19% (moderately inflated)

It's an issue just to keep trace of this observation, is this feedback something you've encountered before?
Maybe it's just normal as 1.81 is a beta.

If not, I can make a reproducible example I think.

Thanks

[the8472]

is this feedback something you've encountered before?

Not sure what you're asking. Yes, we do get performance regression reports and where possible we do want to fix them. But your report is not detailed enough to tell if it's similar to any existing issue.

Maybe it's just normal as 1.81 is a beta.

Beta builds are not expect to be slower than stable builds.

If not, I can make a reproducible example I think.

Yes, that would be necessary to narrow down what's causing it.

[GuillaumePressiat]

Yes, that would be necessary to narrow down what's causing it.

I'm on it!

[GuillaumePressiat]

Thanks for your response.

Here is a repo with a comparison in a reproducible manner (as a rust project..).

https://github.com/GuillaumePressiat/rust_fwf_polars_example

I hope it's fine like this

[GuillaumePressiat]

Do you want some export of samply profile?

[apiraino]

hey @GuillaumePressiat thanks for reporting and bulding a sample of your code. It depends heavily on polars so I wish there a way to reduce polars dependencies and compile a real minimal piece of code to point us to the rustc regression.

For this we tend to use our cargo-bisect-rustc tool to isolate where the regression started but in this case I think it's a bit a long process.

@rustbot label +I-slow +E-needs-mcve

[GuillaumePressiat]

Hi @apiraino

Thanks for your answer.

Do you mean that I have to isolate where the regression started using cargo-bisect-rustc (and lighten the reproducible example without polars) or it's a task that you'll do? To be clear on what I have to do to go beyond.

[apiraino]

Anyone having time and motivation :-) can attempt to reduce the code to an actionable MCVE, then use that code to bisect the compiler using cargo-bisect-rustc and try narrowing down where the regression started. So, yes, if you want you can help, that would be greatly appreciated!

I did a quick test with your code and on my machine the difference between nightly and stable compiler doesn't look that different (both are around ~100ms), so the bisection would need to reduce this noise. maybe your bench tool can help?

[GuillaumePressiat]

Thanks,

If no difference on your computer maybe it's just on (my) Mac silicon m2 pro?
Are rustc nightly optimized (for apple chips I mean)?

bench is here (for Mac):
https://formulae.brew.sh/formula/bench

also beginning with rust-bisect I'm not able to find a regression