enhance unittest to show test name and docstring on one line

[ethanfurman]

BPO	47133
Nosy	@gpshead, @stevendaprano, @ethanfurman, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ethanfurman'
closed_at = None
created_at = <Date 2022-03-26.20:56:32.850>
labels = ['easy', 'type-feature', 'tests', '3.11']
title = 'enhance unittest to show test name and docstring on one line'
updated_at = <Date 2022-04-03.20:00:29.618>
user = 'https://github.com/ethanfurman'

bugs.python.org fields:

activity = <Date 2022-04-03.20:00:29.618>
actor = 'gregory.p.smith'
assignee = 'ethan.furman'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2022-03-26.20:56:32.850>
creator = 'ethan.furman'
dependencies = []
files = []
hgrepos = []
issue_num = 47133
keywords = ['easy']
message_count = 9.0
messages = ['416090', '416094', '416097', '416099', '416100', '416103', '416105', '416138', '416642']
nosy_count = 5.0
nosy_names = ['gregory.p.smith', 'steven.daprano', 'ethan.furman', 'serhiy.storchaka', 'itay.yeshaya']
pr_nums = []
priority = 'low'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue47133'
versions = ['Python 3.11']

Linked PRs

• gh-91289: Reformat the text output of unittest #121704

[ethanfurman]

When running unittest with the -v flag, if a test has errors, and has a docstring, the test name is shown on one line, and the docstring is shown on the next line -- and the ERROR word is shown with the docstring.

What this means is that:

• the test name is easily missed by anyone who isn't aware of that (I only recently found out when someone else pointed it out to me)
• grepping for ERROR doesn't reveal the test name, necessitating a search through the testing code to find the test that failed

There are two possible solutions, and the selection of which one should be discussed on python-dev:

• print the name and docstring, and ERROR, all on one line; or
• provide a command-line switch to select that behavior

---

For the experienced developers who see this: please leave the issue for one of the new developers subscribed to core-mentorship. Thank you.

[itayyeshaya]

I would like to work on this.

[serhiy-storchaka]

bpo-14265, bpo-46126.

[stevendaprano]

I don't think this change of behaviour should be accepted without discussion, and I'm not sure why you think Python-Dev is the right place, rather than here.

Messing around with unittest and docstrings has already caused issues in the past:

https://docs.python.org/3/library/unittest.html#unittest.TestCase.shortDescription

so perhaps we shouldn't break what's not broken?

And frankly, I don't see why the current behaviour is a problem, although maybe that's because the way I use unittest is different from the way you use it. Grepping the output for errors? Why run verbose mode if you aren't going to read the whole output? Seems very odd.

Ethan says: "the test name is easily missed by anyone who isn't aware of that"

Surely the solution to that is education, not changing unittest? Now you are aware of it, and the test name is no longer "easily missed".

If it ever was. To me, it is plain as day. Here is some typical output:

[steve ~]$ python3.10 -m unittest -v mathlib/testing/test_utils.py
test_broken_doc (mathlib.testing.test_utils.TestMinmax)
This is a docstring ... ERROR
test_broken_nodoc (mathlib.testing.test_utils.TestMinmax) ... ERROR
test_empty_iterable (mathlib.testing.test_utils.TestMinmax) ... ok
[additional passing tests ommitted for brevity]

======================================================================
ERROR: test_broken_doc (mathlib.testing.test_utils.TestMinmax)
This is a docstring
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/steve/Desktop/mathlib/testing/test_utils.py", line 20, in test_broken_doc
    self.assertEqual(1, x)
NameError: name 'x' is not defined

======================================================================
ERROR: test_broken_nodoc (mathlib.testing.test_utils.TestMinmax)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/steve/Desktop/mathlib/testing/test_utils.py", line 24, in test_broken_nodoc
    self.assertEqual(1, x)
NameError: name 'x' is not defined

---

Ran 10 tests in 0.004s

FAILED (errors=2)

Errors are listed twice, once in the summary, and the second time in the details section showing the traceback. Note that if you just grep for ERROR you get the result you want:

[steve ~]$ python3.10 -m unittest -v mathlib/testing/test_utils.py |& grep ERROR
This is a docstring ... ERROR
test_broken_nodoc (mathlib.testing.test_utils.TestMinmax) ... ERROR
ERROR: test_broken_doc (mathlib.testing.test_utils.TestMinmax)
ERROR: test_broken_nodoc (mathlib.testing.test_utils.TestMinmax)

So I am having trouble seeing what the problem here is.

[ethanfurman]

I proposed a discussion to python-dev to increase the odds that folks with an interest could chime in. Not everyone follows the new-bugs list.

I find having the output on two lines counter-intuitive -- I was expecting one line per test, and indeed my experience seemed to support that (out of hundreds of tests showing on one line, it's easy to miss that the rare verbose error is on two). And I'm not the only one, since another developer also thought there was a problem and spent most of bpo-46126 "fixing" it.

Another reason for posting to python-dev is that bpo-46126 had been active for three months and had a PR merged before another dev mentioned that verbose tests with docstrings just printed on two lines instead of one.

Thank you for pointing out the possible disruption caused by unittest output changes.