Coverage sometimes report missing branches for excluded lines

[skirpichev]

A minimal reproducible example:

$ cat pyproject.toml 
[tool.coverage.report]
exclude_lines = ['raise NotImplementedError']
$ cat -n test_a.py
     1  def foo(x):
     2      if x == 1:
     3          raise NotImplementedError
     4      return x
     5  
     6  def bar(x, y):
     7      if all(_ == __ for _, __ in zip(x, y)):
     8          raise NotImplementedError
     9      return x
    10  
    11  
    12  def test_0():
    13      foo(0)
    14      bar([0], [1])
$ pytest -q --cov test_a test_a.py --cov-branch --cov-report=term-missing 
.                                                                                                                                                      [100%]

----------- coverage: platform linux, python 3.9.2-final-0 -----------
Name        Stmts   Miss Branch BrPart  Cover   Missing
-------------------------------------------------------
test_a.py       9      0      2      2    82%   7->8, 7->exit
-------------------------------------------------------
TOTAL           9      0      2      2    82%

1 passed in 0.06s
$ pip list |egrep 'pytest|coverage'
coverage                      6.0.2
coverage-enable-subprocess    1.0
pytest                        6.2.5
pytest-cov                    3.0.0
pytest-doctestplus            0.9.0
pytest-forked                 1.3.0
pytest-timeout                2.0.1
pytest-xdist                  2.4.0

It seems for me, that the branch condition on the line 7 should be ignored as well as on the line 2.

See also this example, notice "pragma" statement to workaround the problem.

[nedbat]

Thanks, this is now fixed in 9209c55.

[skirpichev]

@nedbat, I'm not sure if this is fixed. Here is my experiment to remove redundant pragma's from the Diofant codebase. Unfortunately, sometimes this still not doesn't work:

• one example, maybe not a bug
• another, with exception
• and third one with some complex statement

I'll try to provide more short examples. Let me know if I should reopen the issue or create new.

PS: I've in the config:

$ grep exclude_lines -A3 pyproject.toml 
exclude_lines = ['pragma: no cover',
                 'except NotImplementedError:',
                 'raise NotImplementedError',
                 'return NotImplemented']

[nedbat]

Shorter examples would be great, but even just instructions to run the tests and coverage locally would be a help.

[skirpichev]

Here is a tiny example for the 2nd case (an exception):

$ cat -n test_bug2.py 
     1  def foo(x):
     2      try:
     3          return x + 1
     4      except TypeError:
     5          return NotImplemented
     6  
     7  
     8  def test_foo():
     9      foo(1)
$ pytest -q --cov test_bug2 test_bug2.py
.                                                                                                  [100%]

---------- coverage: platform linux, python 3.10.0-final-0 -----------
Name           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------
test_bug2.py       6      1      0      0    83%   4
----------------------------------------------------------
TOTAL              6      1      0      0    83%

1 passed in 0.07s
$ coverage --version
Coverage.py, version 6.1.2a0 with C extension
Full documentation is at https://coverage.readthedocs.io/en/6.1.2a0
$ cat pyproject.toml 
[tool.coverage.run]
branch = true
[tool.coverage.report]
exclude_lines = ['pragma: no cover',
                 'except NotImplementedError:',
                 'raise NotImplementedError',
                 'return NotImplemented']
show_missing = true

[skirpichev]

First example more questionable, but it might be an issue:

$ cat -n test_bug1.py 
     1  def foo(x):
     2      if x == 1:
     3          return x
     4      elif x == 2:
     5          raise NotImplementedError
     6  
     7  
     8  def test_foo():
     9      foo(1)
$ pytest -q --cov test_bug1 test_bug1.py
.                                                                                            [100%]

---------- coverage: platform linux, python 3.10.0-final-0 -----------
Name           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------
test_bug1.py       6      1      2      1    75%   4
----------------------------------------------------------
TOTAL              6      1      2      1    75%

1 passed in 0.07s

IMHO, elif on line 4 should be ignored together with an exception and the line 2 should be marked as partially covered: no 2->exit. Currently it reports (in the html), that there is not jump 2->4.

[skirpichev]

3rd example:

$ cat -n test_bug3.py 
     1  def foo(x):
     2      if False and not all(_ for _ in f):
     3          raise NotImplementedError
     4  
     5  
     6  def test_foo():
     7      foo([1])
$ pytest -q --cov test_bug3 test_bug3.py --cov-branch
.                                                                                         [100%]

---------- coverage: platform linux, python 3.10.0-final-0 -----------
Name           Stmts   Miss Branch BrPart  Cover   Missing
----------------------------------------------------------
test_bug3.py       4      0      2      1    83%   2->exit
----------------------------------------------------------
TOTAL              4      0      2      1    83%

1 passed in 0.07s

[nedbat]

The fix so far is now released as part of coverage 6.1.2.