Declared <https://github.com/python/cpython/blob/main/Lib/xmlrpc/server.py#L622>
Inside
[`do_POST()`](https://github.com/python/cpython/blob/main/Lib/xmlrpc/server.py#L493)
`data: bytes`, where `decode_request_content()` only handles `gzip`
compression.
The `result` is then written to `self.wfile`, which uses `bytes`.
But for
[CGI](https://github.com/python/cpython/blob/main/Lib/xmlrpc/server.py#L636)
`str` is used: [`handle_xmlrpc(self,
request_text)`](https://github.com/python/cpython/blob/main/Lib/xmlrpc/server.py#L642)
calls the argument `request_text`, which is read from `sys.stdin` as
type `str` and then passed to `_marshaled_dispatch()`, which internally calls
[`xmlrpc.client.loads()`](https://github.com/python/cpython/blob/main/Lib/xmlrpc/server.py#L257)
to parse the XML using Expat, which accepts both `str` and `bytes`; the
later defaults to encoding `utf-8`, but other encodings can be
used when explicitly specified:
>>> xmlrpc.client.loads('<params><param><value><string>ä</string></value></param></params>')
(('ä',), None)
>>> xmlrpc.client.loads('<params><param><value><string>ä</string></value></param></params>'.encode("utf-8"))
(('ä',), None)
>>> xmlrpc.client.loads('<params><param><value><string>ä</string></value></param></params>'.encode("iso-8859-1"))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.11/xmlrpc/client.py", line 1029, in loads
p.feed(data)
File "/usr/lib/python3.11/xmlrpc/client.py", line 451, in feed
self._parser.Parse(data, False)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 3, column 15
>>> xmlrpc.client.loads('<?xml version="1.0" encoding="utf-8"?><params><param><value><string>ä</string></value></param></params>'.encode("iso-8859-1"))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.11/xmlrpc/client.py", line 1029, in loads
p.feed(data)
File "/usr/lib/python3.11/xmlrpc/client.py", line 451, in feed
self._parser.Parse(data, False)
xml.parsers.expat.ExpatError: not well-formed (invalid token): line 1, column 68
>>> xmlrpc.client.loads('<?xml version="1.0" encoding="iso-8859-1"?><params><param><value><string>ä</string></value></param></params>'.encode("iso-8859-1"))
(('ä',), None)
Signed-off-by: Philipp Hahn <hahn@univention.de>
Reviewed-By: Jelle Zijlstra <jelle.zijlstra@gmail.com>
In PEP 675, Graham Bleaney and I had specified a list of `LiteralString`-preserving [overloads](https://peps.python.org/pep-0675/#appendix-c-str-methods-that-preserve-literalstring) for `str`. However, we didn't specify an overload for `__getitem__` and didn't give any rationale either. IIRC this was an edge case we didn't want to take a strong decision on unless users wanted it.
Carl Meyer brought this up yesterday, so I think it's worth discussing.
Pro: `my_literal_string[i]` or `my_literal_string[i:j]` should technically be compatible with `LiteralString`, since it is a substring of a literal-derived string.
Con: The main downside is that an attacker might control the indexes and try to access a specific substring from a literal string in the code. For example, they might narrow down the string to `rm foo` or `SELECT *`.
It's true that `join` and other methods could also construct dangerous strings from `LiteralString`s, and we even call that out as an accepted tradeoff in the PEP:
> 4. Trivial functions could be constructed to convert a str to a LiteralString:
>
> def make_literal(s: str) -> LiteralString:
> letters: Dict[str, LiteralString] = {
> "A": "A",
> "B": "B",
> ...
> }
> output: List[LiteralString] = [letters[c] for c in s]
> return "".join(output)
>
> We could mitigate the above using linting, code review, etc., but ultimately a clever, malicious developer attempting to circumvent the protections offered by LiteralString will always succeed. The important thing to remember is that LiteralString is not intended to protect against malicious developers; it is meant to protect against benign developers accidentally using sensitive APIs in a dangerous way (without getting in their way otherwise).
>
> Without LiteralString, the best enforcement tool API authors have is documentation, which is easily ignored and often not seen. With LiteralString, API misuse requires conscious thought and artifacts in the code that reviewers and future developers can notice.
>
> -- [PEP 675 - Appendix B: Limitations](https://peps.python.org/pep-0675/#appendix-b-limitations)
`__getitem__`, however, seems a bit different, because it (and `split`, `zfill`, etc.) accept an index or width that could be used to construct a dangerous query or a humongous string. So, we need to clarify the intent a little.
What was the intent of these overloads? We wanted to forbid "arbitrary user-supplied strings" while allowing methods that preserved literal strings. We were not trying to prevent every possible exploit on the string. Since `__getitem__` forbids arbitrary user-supplied strings and preserves literal strings, I think we should add an overload for it.
Really all I needed for fixing the inheritance was _ssl._SSLContext.
But then I needed all the other stuff in _ssl, and if I was doing that
I wanted to do a thorough job of it.
Motivation was originally related to https://github.com/python/typeshed/issues/3968 ,
but we're well beyond that now, really.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
