gh-86463: Fix default prog in subparsers if usage is used in the main parser

[serhiy-storchaka]

The usage parameter of argparse.ArgumentParser no longer affects the default value of the prog parameter in subparsers.

Previously the full custom usage of the main parser was used as the prog prefix in subparsers.

• Issue: [argparse] Bad error message formatting when using custom usage text #86463

---

📚 Documentation preview 📚: https://cpython-previews--125891.org.readthedocs.build/

[savannahostrowski]

Hmm, I'm a bit worried that this could be a pretty significant breaking change for folks relying on the current behaviour where usage is propagated down into the subparser message. @serhiy-storchaka I'm not sure if you have any intuition here about the impact but would be interested to get your thoughts.

[serhiy-storchaka]

I have found less than a hundred of projects that use usage= and subparsers on GitHub: https://github.com/search?q=ArgumentParser+usage%3D+add_subparsers+language%3APython&type=code (some results may be false positives). Almost all of them are affected by this bug -- they have incorrect usage and error messages in subparsers. Only 2-4 of them have usage that works correctly -- these that only contain the program name with possible ellipsis (such as usage='prog_name' or usage='%(prog)s ...'). This is actually a workaround of the bug. This PR will make such workaround not needed.

I had some doubts, but now my confidence grew.

[savannahostrowski]

Thanks for sharing that GitHub search! At a cursory glance, I don't see that pattern being relied upon today, but there are thousands of results, so it's hard to tell.

That said, another thought I had if we wanted to be conservative here is to not backport the changes to 3.12 and 3.13. That way, it's perhaps even less risky, and we can more deeply communicate the behaviour change as an improvement in the release notes/what's new?

[serhiy-storchaka]

I considered this as a bug fix, but if you insist, we may cancel backporting.

[savannahostrowski]

I considered this as a bug fix, but if you insist, we may cancel backporting.

I think it could be one of those cases where the bug has become expected behaviour. I think we can just denote that this is a behaviour change in 3.14.

[savannahostrowski]

I spent some more time playing around with this and I actually don't know that we should do this. Here's a concrete example to showcase why:

import argparse
parser = argparse.ArgumentParser(description="A version control system CLI", usage="my_vcs")

parser.add_argument("--version", action="version", version="1.0")

subparsers = parser.add_subparsers(dest="command", required=True) 

add_parser = subparsers.add_parser("add", help="Add a file to the repository")
add_parser.add_argument("files", nargs="+", help="List of files to add to staging")

commit_parser = subparsers.add_parser("commit", help="Commit changes to the repository")
commit_parser.add_argument("-m", "--message", required=True, help="Commit message")

commit_group = commit_parser.add_mutually_exclusive_group(required=False)
commit_group.add_argument("--amend", action="store_true", help="Amend the last commit")
commit_group.add_argument("--no-edit", action="store_true", help="Reuse the last commit message")

push_parser = subparsers.add_parser("push", help="Push changes to remote repository")

network_group = push_parser.add_argument_group("Network options")
network_group.add_argument("--dryrun", action="store_true", help="Simulate changes")
network_group.add_argument("--timeout", type=int, default=30, help="Timeout in seconds")

auth_group = push_parser.add_argument_group("Authentication options")
auth_group.add_argument("--username", required=True, help="Username for authentication")
auth_group.add_argument("--password", required=True, help="Password for authentication")

global_group = parser.add_mutually_exclusive_group()
global_group.add_argument("--verbose", action="store_true", help="Verbose output")
global_group.add_argument("--quiet", action="store_true", help="Quiet output")

print(parser.parse_args(['commit', '-h']))

Before this change, I could see the exact command that was run as part of the usage string without any additional intervention:

usage: my_vcs commit [-h] -m MESSAGE [--amend | --no-edit]

But after, the output changes to this, which doesn't seem right:

usage: tester.py commit [-h] -m MESSAGE [--amend | --no-edit]

The comment above the changed line explicitly states that this is intentionally passed down so it seems like this was designed behavior and not a bug. The original issue seems more of a formatting issue, so it looks more correct to me to handle it from that angle rather than removing this behavior. Even if I pass prog in explicitly to the subparser to restore the behavior, we're back to square one with the original issue.

[serhiy-storchaka]

Yes, you can specify a custom usage message which does not broke subparsers. But this is not the case in a vast majority of cases. In 95% or more cases the custom usage message breaks the usage message and error messages in subparsers. Clearly there is an issue and we should fix it.

I initially planned to backport this change and then add a warning if the user specify a custom usage message for the main parser but does not specify 'prog' or 'usage' for subparsers. But then I decided that changing the default behavior together with adding a suggestion in the documentation can be enough. Changes in the documentation alone will not be enough.

The comment you refer to states: "prog defaults to the usage message of this parser, skipping optional arguments and with no "usage:" prefix". But the code cannot skip optional arguments if a custom usage message is used, because a custom usage message replaces the generated usage message in which you can skip optional arguments. Clearly the comment did not mean such case. It is also does not contain all details -- more precisely it should be "...at the moment, not including subparser commands and the following positional arguments". But the custom usage message usually (in vast majority of cases) contains the full line, including subparser commands and options. This is the issue. This is why this code does not work for custom usage message.

[savannahostrowski]

I don't disagree with the necessity of fixing the bug. It's definitely a real issue. I just don't know that I agree that the right fix should be to stop supporting this propagation of the custom usage message down into the subparser. It seems like we should probably try to fix the formatting issues in the usage message or in error messages.

[serhiy-storchaka]

This is the way to fix these issues.

[savannahostrowski]

I still disagree, but I'll trust you on this one if you feel strongly! You've already addressed my concerns about not backporting the change, I don't want to block this any further. 😊

[hugovk]

This was merged with a failing lint check on the CI.

https://github.com/python/cpython/actions/runs/11701457525/job/32587601941

Please could you trim the trailing space here?

-:meth:`~_SubParsersAction.add_parser`, to ensure consistent command prefixes and 
+:meth:`~_SubParsersAction.add_parser`, to ensure consistent command prefixes and

[serhiy-storchaka]

I will not be available in the next 2 hours.

[serhiy-storchaka]

Or maybe not.

[serhiy-storchaka]

#127162.