Previously, handling of test directories (`@tests` and `test_cases`) was
distributed over multiple files and redundant. This unifies the handling
in the `utils` module. This also fixes some instances where "package"
was used instead of "distribution". And in a few instances paths were
joined by using a `/` in a string, which is incompatible with Windows.
Also move `runtests.py` from `scripts` to `tests`. This is required so that
we can import `utils`, but it's also arguably the better fit. The only
mention of the script is actually in the `tests/README.md` file.
Helps with #11762.
If you're reading about this commit from an autogenerated changelog entry, this should have no user-visible impact on how the stubs are interpreted by a type checker; it's just an internal change to how typeshed's tests work.
- Move the logic for running mypy on the test cases from `tests/mypy_test.py` to a separate script, `tests/regr_test.py`.
- Add the necessary logic in order to be able to have test cases for third-party stubs.
- Move logic common to `tests/mypy_test.py` and `tests/regr_test.py` into `tests/colors.py`, and rename `tests/colors.py` to `tests/utils.py`.
- Add a new check to `tests/check_consistent.py`, to enforce the use of `# pyright: reportUnnecessaryTypeIgnoreComment=true` comments in third-party test cases. These are essential if we want to have our tests against false-negatives work with pyright.
- Update the relevant documentation to account for the new test file.
- Add a new job to the `tests.yml` GitHub workflow, to run the new test in CI.
- Add a simple proof-of-concept test case for `requests`, as a regression test for #7998.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Sebastian Rittau <srittau@rittau.biz>
packaging is the standard tool for dealing with versions and
requirements. This allows us to easily support all PEP 508 specs.
packaging also has good error messages, so I don't create our own here.
This lets us get rid of our handrolled half parser and sanity checks.
I kept the whitespace consistency check, since that's just a formatting
lint and isn't testing the correctness of parsing (which we now
definitely do correctly).
- mypy_test and pyright no longer just test the stubs, they now also test other parts of typeshed as well.
- pytype_test.py can now be run on 3.10, meaning the whole test suite can now also be run on 3.10.
- Various test scripts now have from `__future__ import annotations`, meaning they can now only be run on 3.7+.
- Clean up the description of pyright_test.py, which had a slightly confusing wording.
- Also fix the `--dry-run` config option in mypy_test.py, which I accidentally broke in #7746
Move and consolidate venv setup and running isort/black/flake8 into
separate sections and link those sections from the ToC. Also extend
those sections slightly.
Move the tests section into a separate file.
Make venv name in pre-commit match name in CONTRIBUTING.
