_TIER2_RESUME_CHECK is costly in the JIT

[Fidget-Spinner]

Feature or enhancement

Proposal:

_TIER2_RESUME_CHECK deopts around every few thousand loop iterations. This is really costly to the JIT, as we want to stay in the JITted code as far as possible.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-134728: Don't deopt due to eval breaker in _TIER2_RESUME_CHECK #134729

[markshannon]

Why is the eval breaker set so often, and why is executing a few instructions in the interpreter so expensive?

[Fidget-Spinner]

Why is the eval breaker set so often, and why is executing a few instructions in the interpreter so expensive?

I have no clue to the former. To the latter, it's not that executing a few instructions in the interpreter is expensive, more like every few thousand iterations, we're forced to exit to the interpreter for no reason other than some eval breaker being set. This deopt is extremely costly.

[brandtbucher]

I’d also like to run some benchmarks on this, and separately, learn what’s being set. My guess is that it’s either GC or our cold trace invalidation. If it’s the latter, it should be fixed.

In general, I’d like to kick back into the interpreter for failed eval breaker checks, since handling them is an escaping call, and also just a bulky chunk of machine code that basically never runs.

[brandtbucher]

FYI, here’s the threshold for cold trace cleanup. Maybe we shouldn’t leave JIT code for this? Or possibly the mechanism could be reworked. I’d rather not just tweak the threshold to make microbenchmarks happy:

cpython/Include/internal/pycore_optimizer.h

Line 118 in ac539e7

#define JIT_CLEANUP_THRESHOLD 100000

[brandtbucher]

But yeah, I think we should move in the opposite direction than you’re going here (change things like _CHECK_PERIODIC to deopt instead of handling breakers themselves).