Add escaping mode to PyUnicode_FromFormat(): %#c, %#s, %#S, %#U, %#V

[serhiy-storchaka]

Feature or enhancement

Error messages often embed a C string or an object via the s, S, U and V conversions in PyUnicode_FromFormat() and PyErr_Format(). If the string contains non-printable characters, the message can be mangled or misleading: embedded newlines and terminal escape sequences break the output, invisible characters hide the actual content. The R and A conversions escape, but add surrounding quotes and require a Python object.

I propose to add an escaping mode, enabled by the # flag for the c, s, S, U and V conversions. It escapes special and non-printable characters like repr() does, but without adding quotes: \\, \t, \r and \n; other non-printable ASCII characters as \xNN; other non-printable characters as \uNNNN or \UNNNNNNNN. With the additional + flag (%+#s), all non-ASCII characters are escaped as well, so the result is pure ASCII.

For the s and V conversions, bytes which cannot be decoded from UTF-8 are escaped as \xNN, while valid non-ASCII characters are always escaped as \uNNNN or \UNNNNNNNN (e.g. \u009f for valid \xc2\x9f), so they can always be distinguished.

The precision is applied to the string before escaping, the width -- after escaping.

Quotes are deliberately not escaped: the escaped string is intended to be embedded into a larger message without quotes. For quoted output use the R and A conversions.

Related: gh-154610 (escaping attribute names in AttributeError messages).

Linked PRs

• gh-154668: Add escaping mode to PyUnicode_FromFormat() #154669