Jedi passes pickles to subprocesses which are running the target
version of Python and thus may not be the same as the version
under which Jedi itself is running. In Python 3.13, pathlib is
being refactored to allow for easier extension and has thus moved
most of its internal implementation to a submodule. Unfortunately
this changes the paths of the symbols, causing pickles of those
types to fail to load in earlier versions of Python.
This commit introduces a custom unpickler which accounts for this
move, allowing bi-directional passing of pickles to work.
There was a race condition due to the combination of Python's
object ids being re-usable and Jedi persisting such ids beyond
the real lifeteime of some objects. This could lead to the
subprocess' view of the lifetime of `InferenceState` contexts
getting out of step with that in the parent process and
resulting in errors when removing them. It is also possible
that this could result in erroneous results being reported,
however this was not directly observed.
The race was specifically:
- `InferenceState` A created, gets id 1
- `InferenceStateSubprocess` A' created, uses `InferenceState`
A which it stores as a weakref and an id
- `InferenceStateSubprocess` A' is used, the sub-process learns
about an `InferenceState` with id 1
- `InferenceState` A goes away, `InferenceStateSubprocess` A' is
not yet garbage collected
- `InferenceState` B created, gets id 1
- `InferenceStateSubprocess` B' created, uses `InferenceState` B
which it stores as a weakref and an id
- `InferenceStateSubprocess` B' is used, the sub-process re-uses
its entry for an `InferenceState` with id 1
At this point the order of operations between the two
`InferenceStateSubprocess` instances going away is immaterial --
both will trigger a removal of a state with id 1. As long as B'
doesn't try to use the sub-process again after the first removal
has happened then the second removal will fail.
This commit resolves the race condition by coupling the context
in the subprocess to the corresponding manager class instance
in the parent process, rather than to the consumer `InferenceState`.
See inline comments for further details.
In Python 3.13 the `locals` function now returns a fresh mapping
each time it's called (when called in a function). We thus need
to store a reference to the mapping being used, rather than
re-fetching it each time.
Since we don't actually need to modify the locals within the scope
of the test function itself, it suffices to use our own mapping
here rather than the result of calling `locals`, which fully
isolates this test from the nature of that function.
Fixes https://github.com/davidhalter/jedi/issues/2002
I'm not sure where this was used in the past, however it appears
to be unused now. Removing this simplifies a change I'm about to
make to _InferenceStateProcess.
This removes some of the coupling between the management of the
underlying process and the inference state itself, which intends
to enable changing the origin of the id. This will be useful in
the next commit.
It's not completely clear why this was pinned originally, though
at the time Jedi supported Python 2.7 as well as 3.5-3.8, so that
may have had something to do with it.
Removing this pin now seems to work in CI and unblocks some issues
we're seeing around Python 3.12 (specifically that Django<3.1
implicitly relies on distutils, which is no longer available by
default, and possibly other issues).
`find_module` is deprecated in all supported version of Python and
is slated for removal in the upcoming 3.12. Happily it seems we
can move to the related `find_spec` and just hoist the loader from
the spec which that returns. (This is mostly what current `find_module`
implementations do anyway).
This adds support for targetting Python 3.11 via picking up the
latest grammar from parso while also validating support for running
on 3.11 by adding it to the CI matrix.
This includes updating the ignore comments for things which mypy
now knows about or now complains about, as well as pulling in some
typeshed packages for things outside the standard library.
This ensures that these quoted likely forwards references in
return type annotations behave like their non-quoted equivalents.
I suspect there may be other places which will need similar
adjustments, which may mean that we should push the conversion
a layer closer to the parsing (perhaps in `py__annotations__`?).
One case I know that this doesn't solve (but which likely needs
similar adjustment) is generics in return types of comment-style
annotations. They're less likely and may not be worth supporting
since all supported Python versions can use the in-syntax spelling
for annotations at this point.
This means that passthrough signatures will be found for top level
functions, which is useful both where they're wrappered by
`functools.wraps` or not.
Fixes https://github.com/davidhalter/jedi/issues/1791.
This avoids mypy complaining that we need to provide a generic
argument to Popen, which we cannot acctually do as the implementation
of Popen does not inherit from typing.Generic.
No fixes yet, this just gets the config in place.
Note: I'm assuming that we'll pick up a change to parso such that
it exposes its type stubs here. Otherwise we'll want to tweak the
imports config to ignore those errors.
This reinstates the import check for pydoc_data for now.
Specifically I looked in the following:
- python-3.6.8-embed-amd64.zip: missing pydoc_data
- python-3.7.8-embed-amd64.zip: present
- python-3.8.5-embed-amd64.zip: present
In particular this improves support for detecting usage of various
type annotation usages and adds support for correctly parsing
type: ignore comments which contain a reason tag.
This builds on the approach taken in https://github.com/davidhalter/jedi/pull/1613
but applies it to type vars themselves so that their type var
nature is preserved when a function returns Callable[[T], T] and
the T has an upper bound.
Peter Law