Coverage regressions on 3.10.rc1

[skirpichev]

See up to date codecov status and the build artifact. Coveragepy at 57a691f.

I'll select some cases (codecov shows more regressions, but only ones in the power.py and rde.py seems to be in the build artifact, perhaps it's a codecov issue).

1. L1032, L1037 of power.py: no jumps to exit. These lines could be covered, for example, by: exp(Symbol('', real=True, transcendental=True)).is_algebraic and exp(Symbol('', integer=True)).is_algebraic.
2. L309 of rde.py and similar lines with the DecrementLevel context manager. These can be triggered by tests in the diofant/tests/integrals/test_rde.py.

PS: You can see more "green" lines counted in the codecov statistics. Maybe it's a codecov issue. I've reopened the pr request as diofant/diofant#1162 to see if this and mentioned above disagreement with the codecov.xml in the build artifact will disappear.

[nedbat]

Please provide specific instructions about how I can reproduce the problem. Specific commits of your repo to clone, commands to run, and specific lines that you believe are incorrectly measured.

[skirpichev]

I though I did this...

Ok, next try. You can use diofant/diofant#1162, last commit (c59bd75).

1. L1032 and L1037 of the diofant/core/power.py are not covered (no jumps to exit). This should be covered by tests:

exp(Symbol('', integer=True)).is_algebraic  # L1032
exp(Symbol('', real=True, transcendental=True)).is_algebraic  # L1037

In principle, pytest --cov diofant -m 'not slow' diofant/tests/core should also cover these lines.

2. L309 of the diofant/integrals/rde.py (and similar lines with the DecrementLevel context manager). I'm unsure, maybe it's not a bug, but this line was marked as covered on the master, see the codecov output. Now coveragepy report "line 309 didn't except from function 'bound_degree', because the raise on line 320 wasn't executed or the raise on line 342 wasn't executed" (pytest --cov diofant -m 'not slow' diofant/tests/integrals/test_rde.py should trigger all tests for the bound_degree() function).

[skirpichev]

FYI: here is the codecov bugreport

[nedbat]

I've reported the L1032 issue against CPython: https://bugs.python.org/issue44840

[nedbat]

For the L309 issue, I see this in my coverage results:

It has executed line 336, but then neither line 339 or line 341. That makes it seem like some other exception was raised at line 336, which could explain why the context manager on line 309 never jumped to 371. Can you confirm this theory?

[skirpichev]

Hmm, I did pytest -m 'not slow and not xfail and not regression' diofant/tests/integrals/. It seems, L344 (exit from with) has hit from the test_integration() of diofant/tests/integrals/test_integrals.py.

[nedbat]

@skirpichev I'm sorry, I don't understand what your last comment means.

I tried adding another except clause. It shows that line 336 is raising an uncaught exception:

So I think the L309 issue is not a regression.

[skirpichev]

I don't understand what your last comment means.

Well, perhaps I misunderstood you, but it seems your guess was that the only exit from the with clause was by an exception (from L336). That's not true. E.g. you can replace L344 with print(111) (i.e. right after "with" statement) and run pytest -q -n0 -s diofant/tests/integrals/test_integrals.py -k test_integration - you will see some hits. Or run in console from diofant import *; integrate(exp(-log(x)**2)).

So I think the L309 issue is not a regression.

Clearly, there is an exit by an uncaught exception AND a normal exit.

I'll try to reduce this example, of course.

[skirpichev]

PS: You can see more "green" lines counted in the codecov statistics. Maybe it's a codecov issue.

Regarding this, it seems to be a coveragepy "feature". Indeed, some new lines are counted. For instance, L1489 of rings.py, which is "while True" statement.

I'll try to reduce this example, of course.

Another example is in the cancel_primitive():

coverage.xml:
<line number="546" hits="1" branch="true" condition-coverage="66% (2/3)" missing-branches="exit"/>
IIUC, three branches are: normal exit from "with", an uncaught exception from some function inside of "with" and the NotImplementedError on L552?

Probably, you were right, this is not an issue. On another hand, in this example we have only a NotImplementedError, which is ignored by my config. Shouldn't then this branch be ignored by the coveragepy (like for if/else blocks)?

At least, the coveragepy behaviour was changed significantly, could be this documented?

[skirpichev]

FYI, unless I miss something, 1st issue wasn't solved in cpython:

Coveragepy at 57a691f. I've tested this issue on the recent 3.10 head, which has backport from your issue:

$ python
Python 3.10.0rc1+ (heads/3.10:d5c8ad2471, Aug  9 2021, 15:31:13) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.

[nedbat]

@skirpichev I really appreciate your dedication to finding these problems, but please please please please please PLEASE always include the steps you used to produce the results you are showing.

And can you say specifically what about that output is incorrect?

This is what I did (with output showing versions):

$ mktmpenv -p /usr/local/cpython/bin/python3 -n
$ python -VV
Python 3.10.0rc1+ (heads/3.10:d5c8ad2471, Aug  9 2021, 10:24:02) [Clang 12.0.0 (clang-1200.0.32.29)]
$ cd diofant
$ git log --oneline | head -1
c59bd753e XXX removed all workarounds for issue2506 & skipif ground types != gmpy
$ pip install -e '.[develop,docs,tests]'
$ pip install git+https://github.com/nedbat/coveragepy@57a691f15dc8296f91ec1e321d3ed53e165a1f10
$ pytest --cov diofant -m 'not slow' diofant/tests/core

My HTML report looks like this:

We seem to be running different tests.