Fixes#9799.
Stubtest started failing on our `setuptools` stubs because of a default value added to our stdlib `distutils` stubs. The reason for this is because our `setuptools` stubs erroneously claim here that the `setuptools._distutils.core.Distribution` class is the same as the stdlib `distutils.dist.Distribution` class:
https://github.com/python/typeshed/blob/06755e10ba0d39e7e4c18fcc2663d9da564a71ad/stubs/setuptools/setuptools/_distutils/core.pyi#L3
In actual fact, they're not, and they have different default values for this parameter. But reverting the addition of the default value is the simplest short-term fix for now.
(An alternative fix would be to just merge #9795.)
This argument is forwarded on to `Popen.__init__`, like most of the other arguments. It can be `None`, just like the `executable` parameter for all the other `subprocess` functions:
```pycon
>>> import subprocess
>>> subprocess.check_call(["python", "-c", "1/1"], executable=None)
0
>>> subprocess.check_call(["python", "-c", "1/0"], executable=None)
Traceback (most recent call last):
File "<string>", line 1, in <module>
ZeroDivisionError: division by zero
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\alexw\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', '-c', '1/0']' returned non-zero exit status 1.
```
Continuing work towards #8988.
The first five commits were created using stubdefaulter on various Python versions; the following commits were all created manually by me to fix various problems. The main things this adds that weren't present in #9501 are:
- Defaults in Windows-only modules and Windows-only branches (because I'm running a Windows machine)
- Defaults in non-py311 branches
- Defaults for float parameters
- Defaults for overloads
- Add a few more hooks. These are all very fast, and I've found them useful in other projects:
- Autofixes:
- `trailing-whitespace`: fixes trailing whitespace
- `requirements-txt-fixer`: alphabetises items in `requirements.txt` files
- `end-of-file-fixer`: makes sure every file ends with a single newline character
- `mixed-line-ending`: Makes sure Windows users don't accidentally introduce CRLF line endings into a file that uses LF line endings
- None-autofixes:
- `check-yaml`: loads YAML files to validate syntax
- `check-toml`: loads TOML files to validate syntax
- `check-merge-conflict`: detects merge-conflict strings in files and blocks them from accidentally being committed
- `check-case-conflict`: checks for files with names that would conflict on a case-insensitive filesystem like MacOS HFS+ or Windows FAT; blocks them from being committed.
- Change the bot schedule to quarterly, to reduce noisy PRs
- Change the `black` language target-version to Python 3.10, synching the setting here with the changes that were made to our `pyproject.toml` file in #7538
Some fairly subtle logic was added in #9408, and I'm immediately regretting not having documented it slightly better. This PR adds some more comments to make maintaining this code easier in the future.
The approach is pretty similar to the approach I took in #9382 for regr_test.py: dynamically create virtual environments for testing stubs packages in isolation. However, since mypy_test.py tests all of typeshed's stubs in succession (and since lots of typeshed's stubs packages depend on other typeshed stubs packages), the performance issues with creating a virtual environment for testing each stubs package are even more severe than with regr_test.py.
I mitigate the performance issues associated with dynamically creating virtual environments in two ways:
- Dynamically creating venvs is mostly slow due to I/O bottlenecks. Creating the virtual environments can be sped up dramatically by creating them concurrently in a threadpool. The same goes for pip installing the requirements into the venvs -- though unfortunately, we have to use pip with --no-cache-dir, as the pip cache gets easily corrupted if you try to do concurrent reads and writes to the pip cache.
- If types-pycocotools requires numpy>=1 and types-D3DShot also requires numpy>=1 (for example), there's no need to create 2 virtual environments. The same requirements have been specified, so they can share a virtual environment between them. This means we don't have to create nearly so many virtual environments.
Alex Waygood