Jedi passes pickles to subprocesses which are running the target
version of Python and thus may not be the same as the version
under which Jedi itself is running. In Python 3.13, pathlib is
being refactored to allow for easier extension and has thus moved
most of its internal implementation to a submodule. Unfortunately
this changes the paths of the symbols, causing pickles of those
types to fail to load in earlier versions of Python.
This commit introduces a custom unpickler which accounts for this
move, allowing bi-directional passing of pickles to work.
There was a race condition due to the combination of Python's
object ids being re-usable and Jedi persisting such ids beyond
the real lifeteime of some objects. This could lead to the
subprocess' view of the lifetime of `InferenceState` contexts
getting out of step with that in the parent process and
resulting in errors when removing them. It is also possible
that this could result in erroneous results being reported,
however this was not directly observed.
The race was specifically:
- `InferenceState` A created, gets id 1
- `InferenceStateSubprocess` A' created, uses `InferenceState`
A which it stores as a weakref and an id
- `InferenceStateSubprocess` A' is used, the sub-process learns
about an `InferenceState` with id 1
- `InferenceState` A goes away, `InferenceStateSubprocess` A' is
not yet garbage collected
- `InferenceState` B created, gets id 1
- `InferenceStateSubprocess` B' created, uses `InferenceState` B
which it stores as a weakref and an id
- `InferenceStateSubprocess` B' is used, the sub-process re-uses
its entry for an `InferenceState` with id 1
At this point the order of operations between the two
`InferenceStateSubprocess` instances going away is immaterial --
both will trigger a removal of a state with id 1. As long as B'
doesn't try to use the sub-process again after the first removal
has happened then the second removal will fail.
This commit resolves the race condition by coupling the context
in the subprocess to the corresponding manager class instance
in the parent process, rather than to the consumer `InferenceState`.
See inline comments for further details.
In Python 3.13 the `locals` function now returns a fresh mapping
each time it's called (when called in a function). We thus need
to store a reference to the mapping being used, rather than
re-fetching it each time.
Since we don't actually need to modify the locals within the scope
of the test function itself, it suffices to use our own mapping
here rather than the result of calling `locals`, which fully
isolates this test from the nature of that function.
Fixes https://github.com/davidhalter/jedi/issues/2002
I'm not sure where this was used in the past, however it appears
to be unused now. Removing this simplifies a change I'm about to
make to _InferenceStateProcess.
This removes some of the coupling between the management of the
underlying process and the inference state itself, which intends
to enable changing the origin of the id. This will be useful in
the next commit.
* Fix#1988
* Fix failing code quality test.
* Fix flake W504 line break after binary operator. Now as formatted by Black.
* Added test to test/completion/pep0484_basic.py
Addressed feedback from Dave
* properties with setters are now reported as 'property' for completion
* code cleanups
* fixed test
* fixed tests
* Revert "fixed test"
This reverts commit a80c955a48.
* code quality cleanup
* so picky
* Revert "Revert "fixed test""
This reverts commit 58dfc5292e.
* updated test per maintainer comments #1983
* removed extra char
It's not completely clear why this was pinned originally, though
at the time Jedi supported Python 2.7 as well as 3.5-3.8, so that
may have had something to do with it.
Removing this pin now seems to work in CI and unblocks some issues
we're seeing around Python 3.12 (specifically that Django<3.1
implicitly relies on distutils, which is no longer available by
default, and possibly other issues).
