Add distributions arg to third-party stubtest (#5628)

* Add distributions arg to third-party stubtest

Make stubtest_third_party.py executable

Co-authored-by: Akuli <akuviljanen17@gmail.com>

tests/README.md

@@ -7,7 +7,10 @@ tests typeshed with [mypy](https://github.com/python/mypy/)
 [pyright](https://github.com/microsoft/pyright).
 - `tests/check_consistent.py` checks certain files in typeshed remain
 consistent with each other.
-- `tests/stubtest_stdlib.py` checks stubs against the objects at runtime.
+- `tests/stubtest_stdlib.py` checks standard library stubs against the
+objects at runtime.
+- `tests/stubtest_third_party.py` checks third-party stubs against the
+objects at runtime.
 
 To run the tests, follow the [setup instructions](../CONTRIBUTING.md#preparing-the-environment)
 in the `CONTRIBUTING.md` document.
@@ -103,4 +106,10 @@ Run using
 (.venv3)$ python3 tests/stubtest_third_party.py
 ```
 
-Similar to `stubtest_stdlib.py`, but tests the third party stubs.
+Similar to `stubtest_stdlib.py`, but tests the third party stubs. By default,
+it checks all third-party stubs, but you can provide the distributions to
+check on the command line:
+
+```
+(.venv3)$ python3 tests/stubtest_third_party.py Pillow toml  # check stubs/Pillow and stubs/toml
+```

tests/stubtest_third_party.py

@@ -109,10 +109,15 @@ def main():
     parser = argparse.ArgumentParser()
     parser.add_argument("--num-shards", type=int, default=1)
     parser.add_argument("--shard-index", type=int, default=0)
+    parser.add_argument("dists", metavar="DISTRIBUTION", type=str, nargs=argparse.ZERO_OR_MORE)
     args = parser.parse_args()
 
     typeshed_dir = Path(".").resolve()
-    dists = sorted((typeshed_dir / "stubs").iterdir())
+    if len(args.dists) == 0:
+        dists = sorted((typeshed_dir / "stubs").iterdir())
+    else:
+        dists = [typeshed_dir / "stubs" / d for d in args.dists]
+
     for i, dist in enumerate(dists):
         if i % args.num_shards != args.shard_index:
             continue