Closed
Description
Bug report
Bug description:
import tokenize, io
source_code = r'''
f"""{80 * '*'}\n{{test}}{{details}}{{test2}}\n{80 * '*'}"""
'''
tokens = tokenize.generate_tokens(io.StringIO(source_code).readline)
x = tokenize.untokenize((t,s) for t, s, *_ in tokens)
print(x)
Expected:
f"""{80 *'*'}\n{{test}}{{details}}{{test2}}\n{80 *'*'}"""
Got:
f"""{80 *'*'}\n{test}}{{details}}{{test2}}\n{80 *'*'}"""
Note the absence of a second { in the {{ after the \n — but in no other positions.
Unlike some other roundtrip failures of tokenize, some of which are minor infelicities, this one actually creates a syntactically invalid program on roundtrip, which is quite bad. You get a SyntaxError: f-string: single '}' is not allowed
when trying to use the results.
CPython versions tested on:
3.12
Operating systems tested on:
Linux, Windows