Previously, handling of test directories (`@tests` and `test_cases`) was
distributed over multiple files and redundant. This unifies the handling
in the `utils` module. This also fixes some instances where "package"
was used instead of "distribution". And in a few instances paths were
joined by using a `/` in a string, which is incompatible with Windows.
Also move `runtests.py` from `scripts` to `tests`. This is required so that
we can import `utils`, but it's also arguably the better fit. The only
mention of the script is actually in the `tests/README.md` file.
Helps with #11762.
It used to be:
```
-p [{3.11,3.10,3.9,3.8,3.7} ...], --python-version [{3.11,3.10,3.9,3.8,3.7} ...]
Run mypy for certain Python versions (defaults to
sys.version_info[:2])Note that this cannot be specified if
--all is also specified.
```
- Move the logic for running mypy on the test cases from `tests/mypy_test.py` to a separate script, `tests/regr_test.py`.
- Add the necessary logic in order to be able to have test cases for third-party stubs.
- Move logic common to `tests/mypy_test.py` and `tests/regr_test.py` into `tests/colors.py`, and rename `tests/colors.py` to `tests/utils.py`.
- Add a new check to `tests/check_consistent.py`, to enforce the use of `# pyright: reportUnnecessaryTypeIgnoreComment=true` comments in third-party test cases. These are essential if we want to have our tests against false-negatives work with pyright.
- Update the relevant documentation to account for the new test file.
- Add a new job to the `tests.yml` GitHub workflow, to run the new test in CI.
- Add a simple proof-of-concept test case for `requests`, as a regression test for #7998.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Sebastian Rittau <srittau@rittau.biz>
