I think stubsabot has been a really big success. Let's change it to a daily workflow, so that we don't get an avalanche of PRs every Friday, and so that we can see more quickly if a PR changing stubsabot has accidentally broken the script. (If that _does_ happen, it should be pretty easy to just revert the commit that accidentally broke stubsabot, and go back to a version that works pretty well.)
- Move the logic for running mypy on the test cases from `tests/mypy_test.py` to a separate script, `tests/regr_test.py`.
- Add the necessary logic in order to be able to have test cases for third-party stubs.
- Move logic common to `tests/mypy_test.py` and `tests/regr_test.py` into `tests/colors.py`, and rename `tests/colors.py` to `tests/utils.py`.
- Add a new check to `tests/check_consistent.py`, to enforce the use of `# pyright: reportUnnecessaryTypeIgnoreComment=true` comments in third-party test cases. These are essential if we want to have our tests against false-negatives work with pyright.
- Update the relevant documentation to account for the new test file.
- Add a new job to the `tests.yml` GitHub workflow, to run the new test in CI.
- Add a simple proof-of-concept test case for `requests`, as a regression test for #7998.
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
Co-authored-by: Sebastian Rittau <srittau@rittau.biz>
packaging is the standard tool for dealing with versions and
requirements. This allows us to easily support all PEP 508 specs.
packaging also has good error messages, so I don't create our own here.
This lets us get rid of our handrolled half parser and sanity checks.
I kept the whitespace consistency check, since that's just a formatting
lint and isn't testing the correctness of parsing (which we now
definitely do correctly).
Currently, when a maintainer opens a pull request from a branch in the
typeshed repository (instead of a fork), CI is run twice: once for the
PR and once for the push. This not only wastes CI minutes (and it seems
we're running into some limits lately), but also makes CI output twice
as large and more difficult to process.
The disadvantage is that a PR needs to be opened to run CI.
Alternatively, CI can be triggered manually.
