Compare commits

..

1 Commits

Author SHA1 Message Date
Dave Halter 354dab9503 Debug 2021-01-02 13:48:54 +01:00
163 changed files with 1229 additions and 3705 deletions
+19 -18
View File
@@ -1,33 +1,34 @@
name: ci
on: [push, pull_request, workflow_dispatch]
on: push
jobs:
tests:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-24.04, windows-2022]
python-version: ["3.14", "3.13", "3.12", "3.11", "3.10"]
environment: ['3.14', '3.13', '3.12', '3.11', '3.10', 'interpreter']
os: [ubuntu-20.04, windows-2019]
python-version: [3.9, 3.8, 3.7, 3.6]
environment: ['3.8', '3.9', '3.7', '3.6', 'interpreter']
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v2
with:
submodules: recursive
- uses: actions/setup-python@v5
- uses: actions/setup-python@v2
if: ${{ matrix.environment != 'interpreter' }}
with:
python-version: ${{ matrix.environment }}
allow-prereleases: true
- uses: actions/setup-python@v5
- uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
allow-prereleases: true
- name: Install dependencies
run: 'pip install .[dev]'
run: 'pip install .[testing]'
- name: Setup tmate session
uses: mxschmitt/action-tmate@v3
- name: Run tests
run: python -m pytest
@@ -35,32 +36,32 @@ jobs:
JEDI_TEST_ENVIRONMENT: ${{ matrix.environment }}
code-quality:
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v2
with:
submodules: recursive
- name: Install dependencies
run: 'pip install .[dev]'
run: 'pip install .[qa]'
- name: Run tests
run: |
python -m flake8 jedi test setup.py
zuban check
python -m flake8 jedi setup.py
python -m mypy jedi sith.py
coverage:
runs-on: ubuntu-24.04
runs-on: ubuntu-20.04
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v2
with:
submodules: recursive
- name: Install dependencies
run: 'pip install .[dev] coverage'
run: 'pip install .[testing] coverage'
- name: Run tests
run: |
-1
View File
@@ -14,4 +14,3 @@ record.json
/.pytest_cache
/.mypy_cache
/venv/
.nvimrc
+1 -20
View File
@@ -1,21 +1,2 @@
version: 2
python:
install:
- method: pip
path: .
extra_requirements:
- docs
submodules:
include: all
sphinx:
configuration: docs/conf.py
build:
os: ubuntu-24.04
tools:
python: "3.14"
apt_packages:
- graphviz
pip_install: true
+1 -6
View File
@@ -1,4 +1,4 @@
Main Authors
Main Authors
------------
- David Halter (@davidhalter) <davidhalter88@gmail.com>
@@ -61,11 +61,6 @@ Code Contributors
- Vladislav Serebrennikov (@endilll)
- Andrii Kolomoiets (@muffinmad)
- Leo Ryu (@Leo-Ryu)
- Joseph Birkner (@josephbirkner)
- Márcio Mazza (@marciomazza)
- Martin Vielsmaier (@moser) <martin@vielsmaier.net>
- TingJia Wu (@WutingjiaX) <wutingjia@bytedance.com>
- Nguyễn Hồng Quân <ng.hong.quan@gmail.com>
And a few more "anonymous" contributors.
-45
View File
@@ -6,52 +6,7 @@ Changelog
Unreleased
++++++++++
0.20.0 (2026-05-02)
+++++++++++++++++++
- Python 3.14 support
- Removed support for Python 3.8 and 3.9
- Upgraded Typeshed
- Better support for Final/ClassVar
- ``__new__`` is now also recognized as a signature and TypeVar inference
- Support for ``Self``
- Support for ``TypeAlias``, generics for ``type[...]`` and ``tuple[...]``
0.19.2 (2024-11-10)
+++++++++++++++++++
- Python 3.13 support
0.19.1 (2023-10-02)
+++++++++++++++++++
- Python 3.12 support (Thanks Peter!)
0.19.0 (2023-07-29)
+++++++++++++++++++
- Python 3.11 support
- Massive improvements in performance for ``Interpreter`` (e.g. IPython) users.
This especially affects ``pandas`` users with large datasets.
- Add ``jedi.settings.allow_unsafe_interpreter_executions`` to make it easier
for IPython users to avoid unsafe executions.
0.18.2 (2022-11-21)
+++++++++++++++++++
- Added dataclass-equivalent for attrs.define
- Find fixtures from Pytest entrypoints; Examples of pytest plugins installed
like this are pytest-django, pytest-sugar and Faker.
- Fixed Project.search, when a venv was involved, which is why for example
`:Pyimport django.db` did not work in some cases in jedi-vim.
- And many smaller bugfixes
0.18.1 (2021-11-17)
+++++++++++++++++++
- Implict namespaces are now a separate types in ``Name().type``
- Python 3.10 support
- Mostly bugfixes
0.18.0 (2020-12-25)
+++++++++++++++++++
+6 -10
View File
@@ -2,9 +2,6 @@
Jedi - an awesome autocompletion, static analysis and refactoring library for Python
####################################################################################
**I released the successor to Jedi: A
Mypy-Compatible Python Language Server Built in Rust** - `Zuban <https://github.com/zubanls/zuban>`_
.. image:: http://isitmaintained.com/badge/open/davidhalter/jedi.svg
:target: https://github.com/davidhalter/jedi/issues
:alt: The percentage of open issues and pull requests
@@ -13,7 +10,7 @@ Mypy-Compatible Python Language Server Built in Rust** - `Zuban <https://github.
:target: https://github.com/davidhalter/jedi/issues
:alt: The resolution time is the median time an issue or pull request stays open.
.. image:: https://github.com/davidhalter/jedi/actions/workflows/ci.yml/badge.svg?branch=master
.. image:: https://github.com/davidhalter/jedi/workflows/ci/badge.svg?branch=master
:target: https://github.com/davidhalter/jedi/actions
:alt: Tests
@@ -45,7 +42,7 @@ Jedi can currently be used with the following editors/projects:
- `GNOME Builder`_ (with support for GObject Introspection)
- Gedit (gedi_)
- wdb_ - Web Debugger
- `Eric IDE`_
- `Eric IDE`_ (Available as a plugin)
- `IPython 6.0.0+ <https://ipython.readthedocs.io/en/stable/whatsnew/version6.html>`_
- `xonsh shell <https://xon.sh/contents.html>`_ has `jedi extension <https://xon.sh/xontribs.html#jedi>`_
@@ -54,8 +51,7 @@ and many more!
There are a few language servers that use Jedi:
- `jedi-language-server <https://github.com/pappasam/jedi-language-server>`_
- `python-language-server <https://github.com/palantir/python-language-server>`_ (currently unmaintained)
- `python-lsp-server <https://github.com/python-lsp/python-lsp-server>`_ (fork from python-language-server)
- `python-language-server <https://github.com/palantir/python-language-server>`_
- `anakin-language-server <https://github.com/muffinmad/anakin-language-server>`_
Here are some pictures taken from jedi-vim_:
@@ -102,7 +98,7 @@ Features and Limitations
Jedi's features are listed here:
`Features <https://jedi.readthedocs.org/en/latest/docs/features.html>`_.
You can run Jedi on Python 3.10+ but it should also
You can run Jedi on Python 3.6+ but it should also
understand code that is older than those versions. Additionally you should be
able to use `Virtualenvs <https://jedi.readthedocs.org/en/latest/docs/api.html#environments>`_
very well.
@@ -183,10 +179,10 @@ The test suite uses ``pytest``::
pip install pytest
If you want to test only a specific Python version (e.g. Python 3.14), it is as
If you want to test only a specific Python version (e.g. Python 3.8), it is as
easy as::
python3.14 -m pytest
python3.8 -m pytest
For more detailed information visit the `testing documentation
<https://jedi.readthedocs.org/en/latest/docs/testing.html>`_.
-9
View File
@@ -1,9 +0,0 @@
# Security Policy
If security issues arise, we will try to fix those as soon as possible.
Due to Jedi's nature, Security Issues will probably be extremely rare, but we will of course treat them seriously.
## Reporting Security Problems
If you need to report a security vulnerability, please send an email to davidhalter88@gmail.com. Typically, I will respond in the next few business days.
+15 -9
View File
@@ -42,7 +42,7 @@ def pytest_addoption(parser):
help="Warnings are treated as errors.")
parser.addoption("--env", action='store',
help="Execute the tests in that environment (e.g. 314 for python3.14).")
help="Execute the tests in that environment (e.g. 39 for python3.9).")
parser.addoption("--interpreter-env", "-I", action='store_true',
help="Don't use subprocesses to guarantee having safe "
"code execution. Useful for debugging.")
@@ -133,13 +133,11 @@ def goto_or_help(request, Script):
@pytest.fixture(scope='session', params=['goto', 'help', 'infer'])
def goto_or_help_or_infer(request, Script):
class GotoOrHelpOrInfer:
def __call__(self, code, *args, **kwargs):
return getattr(Script(code), request.param)(*args, **kwargs)
def do(code, *args, **kwargs):
return getattr(Script(code), request.param)(*args, **kwargs)
type = request.param
return GotoOrHelpOrInfer()
do.type = request.param
return do
@pytest.fixture(scope='session', params=['goto', 'complete', 'help'])
@@ -159,8 +157,16 @@ def jedi_path():
@pytest.fixture()
def skip_pre_python311(environment):
if environment.version_info < (3, 11):
def skip_pre_python38(environment):
if environment.version_info < (3, 8):
# This if is just needed to avoid that tests ever skip way more than
# they should for all Python versions.
pytest.skip()
@pytest.fixture()
def skip_pre_python37(environment):
if environment.version_info < (3, 7):
# This if is just needed to avoid that tests ever skip way more than
# they should for all Python versions.
pytest.skip()
+2 -2
View File
@@ -35,7 +35,7 @@ to write my own version of a completion engine.
The first idea was to execute non-dangerous code. But I soon realized, that
this would not work. So I started to build a static analysis tool.
The biggest problem that I had at the time was that I did not know a thing
about parsers. I did not even know the word static analysis. It turns
about parsers.I did not did not even know the word static analysis. It turns
out they are the foundation of a good static analysis tool. I of course did not
know that and tried to write my own poor version of a parser that I ended up
throwing away two years later.
@@ -53,7 +53,7 @@ quick and is pretty much feature complete.
--------
I will leave you with a small anecdote that happened in 2012, if I remember
I will leave you with a small annectote that happend in 2012, if I remember
correctly. After I explained Guido van Rossum, how some parts of my
auto-completion work, he said:
+1 -1
View File
@@ -107,7 +107,7 @@ Completions
>>> code = '''import json; json.l'''
>>> script = jedi.Script(code, path='example.py')
>>> script
<Script: 'example.py' <SameEnvironment: 3.14.0 in /usr>>
<Script: 'example.py' <SameEnvironment: 3.9.0 in /usr>>
>>> completions = script.complete(1, 19)
>>> completions
[<Completion: load>, <Completion: loads>]
+3 -3
View File
@@ -16,7 +16,7 @@ Jedi's main API calls and features are:
Basic Features
--------------
- Python 3.10+ support
- Python 3.6+ support
- Ignores syntax errors and wrong indentation
- Can deal with complex module / function / class structures
- Great ``virtualenv``/``venv`` support
@@ -57,7 +57,7 @@ Supported Python Features
Limitations
-----------
In general Jedi's limit is quite high, but for very big projects or very
In general Jedi's limit are quite high, but for very big projects or very
complex code, sometimes Jedi intentionally stops type inference, to avoid
hanging for a long time.
@@ -77,7 +77,7 @@ Performance Issues
Importing ``numpy`` can be quite slow sometimes, as well as loading the
builtins the first time. If you want to speed things up, you could preload
libraries in |jedi|, with :func:`.preload_module`. However, once loaded, this
libriaries in |jedi|, with :func:`.preload_module`. However, once loaded, this
should not be a problem anymore. The same is true for huge modules like
``PySide``, ``wx``, ``tensorflow``, ``pandas``, etc.
+1 -1
View File
@@ -38,7 +38,7 @@ using pip::
If you want to install the current development version (master branch)::
sudo pip install -e git+https://github.com/davidhalter/jedi.git#egg=jedi
sudo pip install -e git://github.com/davidhalter/jedi.git#egg=jedi
System-wide installation via a package manager
+2 -2
View File
@@ -7,10 +7,10 @@ The test suite depends on ``pytest``::
pip install pytest
If you want to test only a specific Python version (e.g. Python 3.14), it is as
If you want to test only a specific Python version (e.g. Python 3.8), it is as
easy as::
python3.14 -m pytest
python3.8 -m pytest
Tests are also run automatically on `GitHub Actions
<https://github.com/davidhalter/jedi/actions>`_.
+6 -7
View File
@@ -3,8 +3,8 @@
Using Jedi
==========
|jedi| can be used with a variety of :ref:`plugins <editor-plugins>`,
:ref:`language servers <language-servers>` and other software.
|jedi| is can be used with a variety of :ref:`plugins <editor-plugins>`,
`language servers <language-servers>` and other software.
It is also possible to use |jedi| in the :ref:`Python shell or with IPython
<repl-completion>`.
@@ -13,11 +13,10 @@ Below you can also find a list of :ref:`recipes for type hinting <recipes>`.
.. _language-servers:
Language Servers
----------------
--------------
- `jedi-language-server <https://github.com/pappasam/jedi-language-server>`_
- `python-language-server <https://github.com/palantir/python-language-server>`_ (currently unmaintained)
- `python-lsp-server <https://github.com/python-lsp/python-lsp-server>`_ (fork from python-language-server)
- `python-language-server <https://github.com/palantir/python-language-server>`_
- `anakin-language-server <https://github.com/muffinmad/anakin-language-server>`_
.. _editor-plugins:
@@ -87,7 +86,7 @@ Gedit
Eric IDE
~~~~~~~~
- `Eric IDE`_
- `Eric IDE`_ (Available as a plugin)
Web Debugger
~~~~~~~~~~~~
@@ -97,7 +96,7 @@ Web Debugger
xonsh shell
~~~~~~~~~~~
Jedi is a preinstalled extension in `xonsh shell <https://xon.sh/contents.html>`_.
Jedi is a preinstalled extension in `xonsh shell <https://xon.sh/contents.html>`_.
Run the following command to enable:
::
+1 -1
View File
@@ -27,7 +27,7 @@ ad
load
"""
__version__ = '0.20.0'
__version__ = '0.18.0'
from jedi.api import Script, Interpreter, set_debug_function, preload_module
from jedi import settings
+15 -12
View File
@@ -5,24 +5,27 @@ different Python versions.
import errno
import sys
import pickle
from typing import Any
class Unpickler(pickle.Unpickler):
def find_class(self, module: str, name: str) -> Any:
# Python 3.13 moved pathlib implementation out of __init__.py as part of
# generalising its implementation. Ensure that we support loading
# pickles from 3.13 on older version of Python. Since 3.13 maintained a
# compatible API, pickles from older Python work natively on the newer
# version.
if module == 'pathlib._local':
module = 'pathlib'
return super().find_class(module, name)
def cast_path(string):
"""
Take a bytes or str path and cast it to unicode.
Apparently it is perfectly fine to pass both byte and unicode objects into
the sys.path. This probably means that byte paths are normal at other
places as well.
Since this just really complicates everything and Python 2.7 will be EOL
soon anyway, just go with always strings.
"""
if isinstance(string, bytes):
return str(string, encoding='UTF-8', errors='replace')
return str(string)
def pickle_load(file):
try:
return Unpickler(file).load()
return pickle.load(file)
# Python on Windows don't throw EOF errors for pipes. So reraise them with
# the correct type, which is caught upwards.
except OSError:
+13 -41
View File
@@ -13,6 +13,7 @@ from pathlib import Path
import parso
from parso.python import tree
from jedi._compatibility import cast_path
from jedi.parser_utils import get_executable_nodes
from jedi import debug
from jedi import settings
@@ -99,15 +100,13 @@ class Script:
"""
def __init__(self, code=None, *, path=None, environment=None, project=None):
self._orig_path = path
# An empty path (also empty string) should always result in no path.
if isinstance(path, str):
path = Path(path)
self.path = path.absolute() if path else None
if code is None:
if path is None:
raise ValueError("Must provide at least one of code or path")
# TODO add a better warning than the traceback!
with open(path, 'rb') as f:
code = f.read()
@@ -153,7 +152,7 @@ class Script:
if self.path is None:
file_io = None
else:
file_io = KnownContentFileIO(self.path, self._code)
file_io = KnownContentFileIO(cast_path(self.path), self._code)
if self.path is not None and self.path.suffix == '.pyi':
# We are in a stub file. Try to load the stub properly.
stub_module = load_proper_stub_module(
@@ -206,7 +205,6 @@ class Script:
before magic methods and name mangled names that start with ``__``.
:rtype: list of :class:`.Completion`
"""
self._inference_state.reset_recursion_limitations()
with debug.increase_indent_cm('complete'):
completion = Completion(
self._inference_state, self._get_module_context(), self._code_lines,
@@ -231,7 +229,6 @@ class Script:
:param prefer_stubs: Prefer stubs to Python objects for this method.
:rtype: list of :class:`.Name`
"""
self._inference_state.reset_recursion_limitations()
pos = line, column
leaf = self._module_node.get_name_of_position(pos)
if leaf is None:
@@ -275,7 +272,6 @@ class Script:
:param prefer_stubs: Prefer stubs to Python objects for this method.
:rtype: list of :class:`.Name`
"""
self._inference_state.reset_recursion_limitations()
tree_name = self._module_node.get_name_of_position((line, column))
if tree_name is None:
# Without a name we really just want to jump to the result e.g.
@@ -368,17 +364,10 @@ class Script:
:rtype: list of :class:`.Name`
"""
self._inference_state.reset_recursion_limitations()
definitions = self.goto(line, column, follow_imports=True)
if definitions:
return definitions
leaf = self._module_node.get_leaf_for_position((line, column))
if leaf is not None and leaf.end_pos == (line, column) and leaf.type == 'newline':
next_ = leaf.get_next_leaf()
if next_ is not None and next_.start_pos == leaf.end_pos:
leaf = next_
if leaf is not None and leaf.type in ('keyword', 'operator', 'error_leaf'):
def need_pydoc():
if leaf.value in ('(', ')', '[', ']'):
@@ -410,7 +399,6 @@ class Script:
the current module only.
:rtype: list of :class:`.Name`
"""
self._inference_state.reset_recursion_limitations()
def _references(include_builtins=True, scope='project'):
if scope not in ('project', 'file'):
@@ -445,7 +433,6 @@ class Script:
:rtype: list of :class:`.Signature`
"""
self._inference_state.reset_recursion_limitations()
pos = line, column
call_details = helpers.get_signature_details(self._module_node, pos)
if call_details is None:
@@ -483,7 +470,7 @@ class Script:
module_context = self._get_module_context()
n = leaf.search_ancestor('funcdef', 'classdef')
n = tree.search_ancestor(leaf, 'funcdef', 'classdef')
if n is not None and n.start_pos < pos <= n.children[-1].start_pos:
# This is a bit of a special case. The context of a function/class
# name/param/keyword is always it's parent context, not the
@@ -565,7 +552,6 @@ class Script:
return parso_to_jedi_errors(self._inference_state.grammar, self._module_node)
def _names(self, all_scopes=False, definitions=True, references=False):
self._inference_state.reset_recursion_limitations()
# Set line/column to a random position, because they don't matter.
module_context = self._get_module_context()
defs = [
@@ -594,7 +580,7 @@ class Script:
@validate_line_column
def extract_variable(self, line, column, *, new_name, until_line=None, until_column=None):
"""
Moves an expression to a new statement.
Moves an expression to a new statemenet.
For example if you have the cursor on ``foo`` and provide a
``new_name`` called ``bar``::
@@ -721,8 +707,9 @@ class Interpreter(Script):
:param namespaces: A list of namespace dictionaries such as the one
returned by :func:`globals` and :func:`locals`.
"""
_allow_descriptor_getattr_default = True
def __init__(self, code, namespaces, *, project=None, **kwds):
def __init__(self, code, namespaces, **kwds):
try:
namespaces = [dict(n) for n in namespaces]
except Exception:
@@ -735,32 +722,16 @@ class Interpreter(Script):
if not isinstance(environment, InterpreterEnvironment):
raise TypeError("The environment needs to be an InterpreterEnvironment subclass.")
if project is None:
project = Project(Path.cwd())
super().__init__(code, environment=environment, project=project, **kwds)
super().__init__(code, environment=environment,
project=Project(Path.cwd()), **kwds)
self.namespaces = namespaces
self._inference_state.allow_unsafe_executions = \
settings.allow_unsafe_interpreter_executions
# Dynamic params search is important when we work on functions that are
# called by other pieces of code. However for interpreter completions
# this is not important at all, because the current code is always new
# and will never be called by something.
# Also sometimes this logic goes a bit too far like in
# https://github.com/ipython/ipython/issues/13866, where it takes
# seconds to do a simple completion.
self._inference_state.do_dynamic_params_search = False
self._inference_state.allow_descriptor_getattr = self._allow_descriptor_getattr_default
@cache.memoize_method
def _get_module_context(self):
if self.path is None:
file_io = None
else:
file_io = KnownContentFileIO(self.path, self._code)
tree_module_value = ModuleValue(
self._inference_state, self._module_node,
file_io=file_io,
file_io=KnownContentFileIO(str(self.path), self._code),
string_names=('__main__',),
code_lines=self._code_lines,
)
@@ -779,7 +750,8 @@ def preload_module(*modules):
:param modules: different module names, list of string.
"""
for m in modules:
Script(f"import {m}").infer()
s = "import %s as x; x." % m
Script(s).complete(1, len(s))
def set_debug_function(func_cb=debug.print_to_stdout, warnings=True,
+9 -10
View File
@@ -17,6 +17,8 @@ import re
from pathlib import Path
from typing import Optional
from parso.tree import search_ancestor
from jedi import settings
from jedi import debug
from jedi.inference.utils import unite
@@ -25,7 +27,7 @@ from jedi.inference.compiled.mixed import MixedName
from jedi.inference.names import ImportName, SubModuleName
from jedi.inference.gradual.stub_value import StubModuleValue
from jedi.inference.gradual.conversion import convert_names, convert_values
from jedi.inference.base_value import ValueSet, HasNoContext
from jedi.inference.base_value import ValueSet
from jedi.api.keywords import KeywordName
from jedi.api import completion_cache
from jedi.api.helpers import filter_follow_imports
@@ -35,17 +37,13 @@ def _sort_names_by_start_pos(names):
return sorted(names, key=lambda s: s.start_pos or (0, 0))
def defined_names(inference_state, value):
def defined_names(inference_state, context):
"""
List sub-definitions (e.g., methods in class).
:type scope: Scope
:rtype: list of Name
"""
try:
context = value.as_context()
except HasNoContext:
return []
filter = next(context.get_filters())
names = [name for name in filter.values()]
return [Name(inference_state, n) for n in _sort_names_by_start_pos(names)]
@@ -96,14 +94,15 @@ class BaseName:
@property
def module_path(self) -> Optional[Path]:
"""
Shows the file path of a module. e.g. ``/usr/lib/python3.14/os.py``
Shows the file path of a module. e.g. ``/usr/lib/python3.9/os.py``
"""
module = self._get_module_context()
if module.is_stub() or not module.is_compiled():
# Compiled modules should not return a module path even if they
# have one.
path: Optional[Path] = self._get_module_context().py__file__()
return path
if path is not None:
return path
return None
@@ -507,7 +506,7 @@ class BaseName:
# - param: The parent_context of a param is not its function but
# e.g. the outer class or module.
cls_or_func_node = self._name.tree_name.get_definition()
parent = cls_or_func_node.search_ancestor('funcdef', 'classdef', 'file_input')
parent = search_ancestor(cls_or_func_node, 'funcdef', 'classdef', 'file_input')
context = self._get_module_context().create_value(parent).as_context()
else:
context = self._name.parent_context
@@ -760,7 +759,7 @@ class Name(BaseName):
"""
defs = self._name.infer()
return sorted(
unite(defined_names(self._inference_state, d) for d in defs),
unite(defined_names(self._inference_state, d.as_context()) for d in defs),
key=lambda s: s._name.start_pos or (0, 0)
)
+15 -52
View File
@@ -1,11 +1,10 @@
import re
from textwrap import dedent
from typing import Any
from inspect import Parameter
from parso.python.token import PythonTokenTypes
from parso.python import tree
from parso.tree import Leaf
from parso.tree import search_ancestor, Leaf
from parso import split_lines
from jedi import debug
@@ -19,8 +18,7 @@ from jedi.inference import imports
from jedi.inference.base_value import ValueSet
from jedi.inference.helpers import infer_call_of_leaf, parse_dotted_names
from jedi.inference.context import get_global_filters
from jedi.inference.value import TreeInstance
from jedi.inference.docstring_utils import DocstringModule
from jedi.inference.value import TreeInstance, ModuleValue
from jedi.inference.names import ParamNameWrapper, SubModuleName
from jedi.inference.gradual.conversion import convert_values, convert_names
from jedi.parser_utils import cut_value_at_position
@@ -66,15 +64,12 @@ def _must_be_kwarg(signatures, positional_count, used_kwargs):
return must_be_kwarg
def filter_names(inference_state, completion_names, stack, like_name, fuzzy,
imported_names, cached_name):
def filter_names(inference_state, completion_names, stack, like_name, fuzzy, cached_name):
comp_dct = set()
if settings.case_insensitive_completion:
like_name = like_name.lower()
for name in completion_names:
string = name.string_name
if string in imported_names and string != like_name:
continue
if settings.case_insensitive_completion:
string = string.lower()
if helpers.match(string, like_name, fuzzy=fuzzy):
@@ -142,11 +137,6 @@ class Completion:
self._fuzzy = fuzzy
# Return list of completions in this order:
# - Beginning with what user is typing
# - Public (alphabet)
# - Private ("_xxx")
# - Dunder ("__xxx")
def complete(self):
leaf = self._module_node.get_leaf_for_position(
self._original_position,
@@ -178,19 +168,14 @@ class Completion:
cached_name, completion_names = self._complete_python(leaf)
imported_names = []
if leaf.parent is not None and leaf.parent.type in ['import_as_names', 'dotted_as_names']:
imported_names.extend(extract_imported_names(leaf.parent))
completions = list(filter_names(self._inference_state, completion_names,
self.stack, self._like_name,
self._fuzzy, imported_names, cached_name=cached_name))
self._fuzzy, cached_name=cached_name))
return (
# Removing duplicates mostly to remove False/True/None duplicates.
_remove_duplicates(prefixed_completions, completions)
+ sorted(completions, key=lambda x: (not x.name.startswith(self._like_name),
x.name.startswith('__'),
+ sorted(completions, key=lambda x: (x.name.startswith('__'),
x.name.startswith('_'),
x.name.lower()))
)
@@ -209,6 +194,7 @@ class Completion:
- In args: */**: no completion
- In params (also lambda): no completion before =
"""
grammar = self._inference_state.grammar
self.stack = stack = None
self._position = (
@@ -245,8 +231,8 @@ class Completion:
if previous_leaf is not None:
stmt = previous_leaf
while True:
stmt = stmt.search_ancestor(
'if_stmt', 'for_stmt', 'while_stmt', 'try_stmt',
stmt = search_ancestor(
stmt, 'if_stmt', 'for_stmt', 'while_stmt', 'try_stmt',
'error_node',
)
if stmt is None:
@@ -266,7 +252,7 @@ class Completion:
elif type_ == 'for_stmt':
allowed_transitions.append('else')
completion_names: list[Any] = []
completion_names = []
kwargs_only = False
if any(t in allowed_transitions for t in (PythonTokenTypes.NAME,
@@ -291,12 +277,6 @@ class Completion:
)
elif nonterminals[-1] in ('trailer', 'dotted_name') and nodes[-1] == '.':
dot = self._module_node.get_leaf_for_position(self._position)
if dot.type == "newline":
dot = dot.get_previous_leaf()
if dot.type == "endmarker":
# This is a bit of a weird edge case, maybe we can somehow
# generalize this.
dot = leaf.get_previous_leaf()
cached_name, n = self._complete_trailer(dot.get_previous_leaf())
completion_names += n
elif self._is_parameter_completion():
@@ -359,7 +339,7 @@ class Completion:
stack_node = self.stack[-3]
if stack_node.nonterminal == 'funcdef':
context = get_user_context(self._module_context, self._position)
node = leaf.search_ancestor('error_node', 'funcdef')
node = search_ancestor(leaf, 'error_node', 'funcdef')
if node is not None:
if node.type == 'error_node':
n = node.children[0]
@@ -429,7 +409,7 @@ class Completion:
Autocomplete inherited methods when overriding in child class.
"""
leaf = self._module_node.get_leaf_for_position(self._position, include_prefixes=True)
cls = leaf.search_ancestor('classdef')
cls = tree.search_ancestor(leaf, 'classdef')
if cls is None:
return
@@ -458,7 +438,6 @@ class Completion:
- Having some doctest code that starts with `>>>`
- Having backticks that doesn't have whitespace inside it
"""
def iter_relevant_lines(lines):
include_next_line = False
for l in code_lines:
@@ -483,12 +462,12 @@ class Completion:
def _complete_code_lines(self, code_lines):
module_node = self._inference_state.grammar.parse(''.join(code_lines))
module_value = DocstringModule(
in_module_context=self._module_context,
inference_state=self._inference_state,
module_node=module_node,
module_value = ModuleValue(
self._inference_state,
module_node,
code_lines=code_lines,
)
module_value.parent_context = self._module_context
return Completion(
self._inference_state,
module_value.as_context(),
@@ -681,19 +660,3 @@ def search_in_module(inference_state, module_context, names, wanted_names,
def_ = classes.Name(inference_state, n2)
if not wanted_type or wanted_type == def_.type:
yield def_
def extract_imported_names(node):
imported_names = []
if node.type in ['import_as_names', 'dotted_as_names', 'dotted_as_name', 'import_as_name']:
for index, child in enumerate(node.children):
if child.type == 'name':
if (index > 1 and node.children[index - 1].type == "keyword"
and node.children[index - 1].value == "as"):
continue
imported_names.append(child.value)
elif child.type in ('import_as_name', 'dotted_as_name'):
imported_names.extend(extract_imported_names(child))
return imported_names
+31 -48
View File
@@ -8,7 +8,6 @@ import hashlib
import filecmp
from collections import namedtuple
from shutil import which
from typing import TYPE_CHECKING, Any
from jedi.cache import memoize_method, time_cache
from jedi.inference.compiled.subprocess import CompiledSubprocess, \
@@ -16,13 +15,9 @@ from jedi.inference.compiled.subprocess import CompiledSubprocess, \
import parso
if TYPE_CHECKING:
from jedi.inference import InferenceState
_VersionInfo = namedtuple('VersionInfo', 'major minor micro')
_VersionInfo = namedtuple('VersionInfo', 'major minor micro') # type: ignore[name-match]
_SUPPORTED_PYTHONS = ['3.14', '3.13', '3.12', '3.11', '3.10']
_SUPPORTED_PYTHONS = ['3.9', '3.8', '3.7', '3.6']
_SAFE_PATHS = ['/usr/bin', '/usr/local/bin']
_CONDA_VAR = 'CONDA_PREFIX'
_CURRENT_VERSION = '%s.%s' % (sys.version_info.major, sys.version_info.minor)
@@ -36,9 +31,6 @@ class InvalidPythonEnvironment(Exception):
class _BaseEnvironment:
version_info: Any
executable: Any
@memoize_method
def get_grammar(self):
version_string = '%s.%s' % (self.version_info.major, self.version_info.minor)
@@ -110,10 +102,7 @@ class Environment(_BaseEnvironment):
version = '.'.join(str(i) for i in self.version_info)
return '<%s: %s in %s>' % (self.__class__.__name__, version, self.path)
def get_inference_state_subprocess(
self,
inference_state: 'InferenceState',
) -> InferenceStateSubprocess:
def get_inference_state_subprocess(self, inference_state):
return InferenceStateSubprocess(inference_state, self._get_subprocess())
@memoize_method
@@ -145,10 +134,7 @@ class SameEnvironment(_SameEnvironmentMixin, Environment):
class InterpreterEnvironment(_SameEnvironmentMixin, _BaseEnvironment):
def get_inference_state_subprocess(
self,
inference_state: 'InferenceState',
) -> InferenceStateSameProcess:
def get_inference_state_subprocess(self, inference_state):
return InferenceStateSameProcess(inference_state)
def get_sys_path(self):
@@ -254,7 +240,7 @@ def get_cached_default_environment():
# /path/to/env so we need to fully resolve the paths in order to
# compare them.
if var and os.path.realpath(var) != os.path.realpath(environment.path):
_get_cached_default_environment.clear_cache() # type: ignore[attr-defined]
_get_cached_default_environment.clear_cache()
return _get_cached_default_environment()
return environment
@@ -358,7 +344,7 @@ def get_system_environment(version, *, env_vars=None):
return SameEnvironment()
return Environment(exe)
if sys.platform == "win32":
if os.name == 'nt':
for exe in _get_executables_from_windows_registry(version):
try:
return Environment(exe, env_vars=env_vars)
@@ -386,42 +372,39 @@ def _get_executable_path(path, safe=True):
Returns None if it's not actually a virtual env.
"""
if sys.platform == "win32":
pythons = [os.path.join(path, 'Scripts', 'python.exe'), os.path.join(path, 'python.exe')]
else:
pythons = [os.path.join(path, 'bin', 'python')]
for python in pythons:
if os.path.exists(python):
break
if os.name == 'nt':
python = os.path.join(path, 'Scripts', 'python.exe')
else:
python = os.path.join(path, 'bin', 'python')
if not os.path.exists(python):
raise InvalidPythonEnvironment("%s seems to be missing." % python)
_assert_safe(python, safe)
return python
if sys.platform == "win32":
def _get_executables_from_windows_registry(version):
import winreg
def _get_executables_from_windows_registry(version):
# https://github.com/python/typeshed/pull/3794 adds winreg
import winreg # type: ignore[import]
# TODO: support Python Anaconda.
sub_keys = [
r'SOFTWARE\Python\PythonCore\{version}\InstallPath',
r'SOFTWARE\Wow6432Node\Python\PythonCore\{version}\InstallPath',
r'SOFTWARE\Python\PythonCore\{version}-32\InstallPath',
r'SOFTWARE\Wow6432Node\Python\PythonCore\{version}-32\InstallPath'
]
for root_key in [winreg.HKEY_CURRENT_USER, winreg.HKEY_LOCAL_MACHINE]:
for sub_key in sub_keys:
sub_key = sub_key.format(version=version)
try:
with winreg.OpenKey(root_key, sub_key) as key:
prefix = winreg.QueryValueEx(key, '')[0]
exe = os.path.join(prefix, 'python.exe')
if os.path.isfile(exe):
yield exe
except WindowsError:
pass
# TODO: support Python Anaconda.
sub_keys = [
r'SOFTWARE\Python\PythonCore\{version}\InstallPath',
r'SOFTWARE\Wow6432Node\Python\PythonCore\{version}\InstallPath',
r'SOFTWARE\Python\PythonCore\{version}-32\InstallPath',
r'SOFTWARE\Wow6432Node\Python\PythonCore\{version}-32\InstallPath'
]
for root_key in [winreg.HKEY_CURRENT_USER, winreg.HKEY_LOCAL_MACHINE]:
for sub_key in sub_keys:
sub_key = sub_key.format(version=version)
try:
with winreg.OpenKey(root_key, sub_key) as key:
prefix = winreg.QueryValueEx(key, '')[0]
exe = os.path.join(prefix, 'python.exe')
if os.path.isfile(exe):
yield exe
except WindowsError:
pass
def _assert_safe(executable_path, safe):
+1 -1
View File
@@ -23,7 +23,7 @@ class RefactoringError(_JediError):
Refactorings can fail for various reasons. So if you work with refactorings
like :meth:`.Script.rename`, :meth:`.Script.inline`,
:meth:`.Script.extract_variable` and :meth:`.Script.extract_function`, make
sure to catch these. The descriptions in the errors are usually valuable
sure to catch these. The descriptions in the errors are ususally valuable
for end users.
A typical ``RefactoringError`` would tell the user that inlining is not
+3 -2
View File
@@ -205,6 +205,7 @@ def filter_follow_imports(names, follow_builtin_imports=False):
class CallDetails:
def __init__(self, bracket_leaf, children, position):
['bracket_leaf', 'call_index', 'keyword_name_str']
self.bracket_leaf = bracket_leaf
self._children = children
self._position = position
@@ -280,7 +281,7 @@ class CallDetails:
def count_positional_arguments(self):
count = 0
for star_count, key_start, had_equal in self._list_arguments()[:-1]:
if star_count or key_start:
if star_count:
break
count += 1
return count
@@ -306,7 +307,7 @@ def _iter_arguments(nodes, position):
first = node.children[0]
second = node.children[1]
if second == '=':
if second.start_pos < position and first.type == 'name':
if second.start_pos < position:
yield 0, first.value, True
else:
yield 0, remove_after_pos(first), False
-8
View File
@@ -61,7 +61,6 @@ class MixedModuleContext(ModuleContext):
)
def get_filters(self, until_position=None, origin_scope=None):
yield MergedFilter(
MixedParserTreeFilter(
parent_context=self,
@@ -73,10 +72,3 @@ class MixedModuleContext(ModuleContext):
for mixed_object in self.mixed_values:
yield from mixed_object.get_filters(until_position, origin_scope)
# Now that we have merged the filter for this mixed context we have to
# remove the first entry (which is the module itself), but we want to
# add the other filters like the star imports.
filters = self._value.get_filters(origin_scope)
next(filters, None)
yield from filters
+3 -2
View File
@@ -5,7 +5,8 @@ from typing import Dict, Optional
from jedi.inference.names import AbstractArbitraryName
try:
from pydoc_data import topics
# https://github.com/python/typeshed/pull/4351 adds pydoc_data
from pydoc_data import topics # type: ignore[import]
pydoc_topics: Optional[Dict[str, str]] = topics.topics
except ImportError:
# Python 3.6.8 embeddable does not have pydoc_data.
@@ -41,7 +42,7 @@ def imitate_pydoc(string):
try:
# is a tuple now
label, related = string # type: ignore[misc]
label, related = string
except TypeError:
return ''
+23 -28
View File
@@ -106,16 +106,7 @@ class Project:
with open(self._get_json_path(self._path), 'w') as f:
return json.dump((_SERIALIZER_VERSION, data), f)
def __init__(
self,
path,
*,
environment_path=None,
load_unsafe_extensions=False,
sys_path=None,
added_sys_path=(),
smart_sys_path=True,
) -> None:
def __init__(self, path, **kwargs):
"""
:param path: The base path for this project.
:param environment_path: The Python executable path, typically the path
@@ -134,22 +125,25 @@ class Project:
local directories. Otherwise you will have to rely on your packages
being properly configured on the ``sys.path``.
"""
def py2_comp(path, environment_path=None, load_unsafe_extensions=False,
sys_path=None, added_sys_path=(), smart_sys_path=True):
if isinstance(path, str):
path = Path(path).absolute()
self._path = path
if isinstance(path, str):
path = Path(path).absolute()
self._path = path
self._environment_path = environment_path
if sys_path is not None:
self._environment_path = environment_path
if sys_path is not None:
# Remap potential pathlib.Path entries
sys_path = list(map(str, sys_path))
self._sys_path = sys_path
self._smart_sys_path = smart_sys_path
self._load_unsafe_extensions = load_unsafe_extensions
self._django = False
# Remap potential pathlib.Path entries
sys_path = list(map(str, sys_path))
self._sys_path = sys_path
self._smart_sys_path = smart_sys_path
self._load_unsafe_extensions = load_unsafe_extensions
self._django = False
# Remap potential pathlib.Path entries
self.added_sys_path = list(map(str, added_sys_path))
"""The sys path that is going to be added at the end of the """
self.added_sys_path = list(map(str, added_sys_path))
"""The sys path that is going to be added at the end of the """
py2_comp(path, **kwargs)
@property
def path(self):
@@ -334,8 +328,7 @@ class Project:
)
# 2. Search for identifiers in the project.
for module_context in search_in_file_ios(inference_state, file_ios,
name, complete=complete):
for module_context in search_in_file_ios(inference_state, file_ios, name):
names = get_module_names(module_context.tree_node, all_scopes=all_scopes)
names = [module_context.create_name(n) for n in names]
names = _remove_imports(names)
@@ -352,8 +345,9 @@ class Project:
# 3. Search for modules on sys.path
sys_path = [
p for p in self._get_sys_path(inference_state)
# Exclude the current folder which is handled by recursing the folders.
if p != self._path
# Exclude folders that are handled by recursing of the Python
# folders.
if not p.startswith(str(self._path))
]
names = list(iter_module_names(inference_state, empty_module_context, sys_path))
yield from search_in_module(
@@ -432,6 +426,7 @@ def get_default_project(path=None):
probable_path = dir
if probable_path is not None:
# TODO search for setup.py etc
return Project(probable_path)
if first_no_init_file is not None:
+8 -30
View File
@@ -5,7 +5,6 @@ from typing import Dict, Iterable, Tuple
from parso import split_lines
from jedi.api.exceptions import RefactoringError
from jedi.inference.value.namespace import ImplicitNSName
EXPRESSION_PARTS = (
'or_test and_test not_test comparison '
@@ -43,17 +42,11 @@ class ChangedFile:
if self._from_path is None:
from_p = ''
else:
try:
from_p = self._from_path.relative_to(project_path)
except ValueError: # Happens it the path is not on th project_path
from_p = self._from_path
from_p = self._from_path.relative_to(project_path)
if self._to_path is None:
to_p = ''
else:
try:
to_p = self._to_path.relative_to(project_path)
except ValueError:
to_p = self._to_path
to_p = self._to_path.relative_to(project_path)
diff = difflib.unified_diff(
old_lines, new_lines,
fromfile=str(from_p),
@@ -103,12 +96,7 @@ class Refactoring:
to_path=calculate_to_path(path),
module_node=next(iter(map_)).get_root_node(),
node_to_str_map=map_
)
# We need to use `or`, because the path can be None
for path, map_ in sorted(
self._file_to_node_changes.items(),
key=lambda x: x[0] or Path("")
)
) for path, map_ in sorted(self._file_to_node_changes.items())
}
def get_renames(self) -> Iterable[Tuple[Path, Path]]:
@@ -122,7 +110,7 @@ class Refactoring:
project_path = self._inference_state.project.path
for from_, to in self.get_renames():
text += 'rename from %s\nrename to %s\n' \
% (_try_relative_to(from_, project_path), _try_relative_to(to, project_path))
% (from_.relative_to(project_path), to.relative_to(project_path))
return text + ''.join(f.get_diff() for f in self.get_changed_files().values())
@@ -152,16 +140,13 @@ def rename(inference_state, definitions, new_name):
raise RefactoringError("There is no name under the cursor")
for d in definitions:
# This private access is ok in a way. It's not public to
# protect Jedi users from seeing it.
tree_name = d._name.tree_name
if d.type == 'module' and tree_name is None and d.module_path is not None:
p = Path(d.module_path)
if d.type == 'module' and tree_name is None:
p = None if d.module_path is None else Path(d.module_path)
file_renames.add(_calculate_rename(p, new_name))
elif isinstance(d._name, ImplicitNSName):
for p in d._name._value.py__path__():
file_renames.add(_calculate_rename(Path(p), new_name))
else:
# This private access is ok in a way. It's not public to
# protect Jedi users from seeing it.
if tree_name is not None:
fmap = file_tree_name_map.setdefault(d.module_path, {})
fmap[tree_name] = tree_name.prefix + new_name
@@ -255,10 +240,3 @@ def _remove_indent_of_prefix(prefix):
Removes the last indentation of a prefix, e.g. " \n \n " becomes " \n \n".
"""
return ''.join(split_lines(prefix, keepends=True)[:-1])
def _try_relative_to(path: Path, base: Path) -> Path:
try:
return path.relative_to(base)
except ValueError:
return path
+1 -1
View File
@@ -9,7 +9,7 @@ just use IPython instead::
Then you will be able to use Jedi completer in your Python interpreter::
$ python
Python 3.14.0+ (default, Jul 20 2020, 22:15:08)
Python 3.9.2+ (default, Jul 20 2020, 22:15:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
+2 -5
View File
@@ -36,11 +36,8 @@ def complete_dict(module_context, code_lines, leaf, position, string, fuzzy):
string = cut_value_at_position(leaf, position)
context = module_context.create_context(bracket_leaf)
before_node = before_bracket_leaf = bracket_leaf.get_previous_leaf()
if before_node in (')', ']', '}'):
before_node = before_node.parent
if before_node.type in ('atom', 'trailer', 'name'):
before_bracket_leaf = bracket_leaf.get_previous_leaf()
if before_bracket_leaf.type in ('atom', 'trailer', 'name'):
values = infer_call_of_leaf(context, before_bracket_leaf)
return list(_completions_for_dicts(
module_context.inference_state,
+2 -2
View File
@@ -28,7 +28,7 @@ def clear_time_caches(delete_all: bool = False) -> None:
:param delete_all: Deletes also the cache that is normally not deleted,
like parser cache, which is important for faster parsing.
"""
global _time_caches # noqa: F824
global _time_caches
if delete_all:
for cache in _time_caches.values():
@@ -93,7 +93,7 @@ def time_cache(seconds):
cache[key] = time.time(), result
return result
wrapper.clear_cache = lambda: cache.clear() # type: ignore[attr-defined]
wrapper.clear_cache = lambda: cache.clear()
return wrapper
return decorator
+6 -3
View File
@@ -21,7 +21,7 @@ try:
raise ImportError
else:
# Use colorama for nicer console output.
from colorama import Fore, init # type: ignore[import, unused-ignore]
from colorama import Fore, init # type: ignore[import]
from colorama import initialise
def _lazy_colorama_init(): # noqa: F811
@@ -36,7 +36,7 @@ try:
# pytest resets the stream at the end - causes troubles. Since
# after every output the stream is reset automatically we don't
# need this.
initialise.atexit_done = True # type: ignore[attr-defined]
initialise.atexit_done = True
try:
init(strip=False)
except Exception:
@@ -106,7 +106,10 @@ def dbg(message, *args, color='GREEN'):
debug_function(color, i + 'dbg: ' + message % tuple(repr(a) for a in args))
def warning(message, *args, format=True):
def warning(message, *args, **kwargs):
format = kwargs.pop('format', True)
assert not kwargs
if debug_function and enable_warning:
i = ' ' * _debug_indent
if format:
-3
View File
@@ -1,5 +1,4 @@
import os
from typing import Any
from parso import file_io
@@ -59,8 +58,6 @@ class FolderIO(AbstractFolderIO):
class FileIOFolderMixin:
path: Any
def get_parent_folder(self):
return FolderIO(os.path.dirname(self.path))
+7 -24
View File
@@ -62,8 +62,6 @@ I need to mention now that lazy type inference is really good because it
only *inferes* what needs to be *inferred*. All the statements and modules
that are not used are just being ignored.
"""
from typing import Any
import parso
from jedi.file_io import FileIO
@@ -84,8 +82,6 @@ from jedi.plugins import plugin_manager
class InferenceState:
analysis_modules: "list[Any]"
def __init__(self, project, environment=None, script_path=None):
if environment is None:
environment = project.get_environment()
@@ -94,7 +90,7 @@ class InferenceState:
self.compiled_subprocess = environment.get_inference_state_subprocess(self)
self.grammar = environment.get_grammar()
self.latest_grammar = parso.load_grammar(version='3.13')
self.latest_grammar = parso.load_grammar(version='3.7')
self.memoize_cache = {} # for memoize decorators
self.module_cache = imports.ModuleCache() # does the job of `sys.modules`.
self.stub_module_cache = {} # Dict[Tuple[str, ...], Optional[ModuleValue]]
@@ -103,11 +99,10 @@ class InferenceState:
self.mixed_cache = {} # see `inference.compiled.mixed._create()`
self.analysis = []
self.dynamic_params_depth = 0
self.do_dynamic_params_search = settings.dynamic_params
self.is_analysis = False
self.project = project
self.access_cache = {}
self.allow_unsafe_executions = False
self.allow_descriptor_getattr = False
self.flow_analysis_enabled = True
self.reset_recursion_limitations()
@@ -126,33 +121,19 @@ class InferenceState:
return value_set
# mypy doesn't suppport decorated propeties (https://github.com/python/mypy/issues/1362)
@property
@property # type: ignore[misc]
@inference_state_function_cache()
def builtins_module(self):
module_name = 'builtins'
builtins_module, = self.import_module((module_name,), sys_path=[])
builtins_module, = self.import_module((module_name,), sys_path=())
return builtins_module
@property
@property # type: ignore[misc]
@inference_state_function_cache()
def typing_module(self):
typing_module, = self.import_module(('typing',))
return typing_module
@property
@inference_state_function_cache()
def types_module(self):
typing_module, = self.import_module(('types',))
return typing_module
@inference_state_function_cache()
def typing_tuple(self):
return self.typing_module.py__getattribute__("Tuple")
@inference_state_function_cache()
def typing_type(self):
return self.typing_module.py__getattribute__("Type")
def reset_recursion_limitations(self):
self.recursion_detector = recursion.RecursionDetector()
self.execution_recursion_detector = recursion.ExecutionRecursionDetector(self)
@@ -200,6 +181,8 @@ class InferenceState:
def parse_and_get_code(self, code=None, path=None,
use_latest_grammar=False, file_io=None, **kwargs):
if path is not None:
path = str(path)
if code is None:
if file_io is None:
file_io = FileIO(path)
+1 -4
View File
@@ -1,6 +1,5 @@
import re
from itertools import zip_longest
from typing import Any
from parso.python import tree
@@ -135,7 +134,7 @@ class _AbstractArgumentsMixin:
class AbstractArguments(_AbstractArgumentsMixin):
context = None
argument_node: Any = None
argument_node = None
trailer = None
@@ -165,8 +164,6 @@ def unpack_arglist(arglist):
class TreeArguments(AbstractArguments):
context: Any
def __init__(self, inference_state, context, argument_node, trailer=None):
"""
:param argument_node: May be an argument_node or a list of nodes.
+9 -32
View File
@@ -9,7 +9,6 @@ just one.
from functools import reduce
from operator import add
from itertools import zip_longest
from typing import TYPE_CHECKING, Any
from parso.python.tree import Name
@@ -22,25 +21,8 @@ from jedi.cache import memoize_method
sentinel = object()
if TYPE_CHECKING:
from jedi.inference import InferenceState
class HasNoContext(Exception):
pass
class HelperValueMixin:
parent_context: Any
inference_state: "InferenceState"
name: Any
get_filters: Any
is_stub: Any
py__getattribute__alternatives: Any
py__iter__: Any
py__mro__: Any
_as_context: Any
def get_root_context(self):
value = self
if value.parent_context is None:
@@ -59,7 +41,7 @@ class HelperValueMixin:
arguments = ValuesArguments([ValueSet([value]) for value in value_list])
return self.inference_state.execute(self, arguments)
def execute_annotation(self, context):
def execute_annotation(self):
return self.execute_with_values()
def gather_annotation_classes(self):
@@ -279,7 +261,7 @@ class Value(HelperValueMixin):
return self.parent_context.is_stub()
def _as_context(self):
raise HasNoContext
raise NotImplementedError('Not all values need to be converted to contexts: %s', self)
@property
def name(self):
@@ -311,7 +293,7 @@ class Value(HelperValueMixin):
just the `_T` generic parameter.
`value_set`: represents the actual argument passed to the parameter
we're inferred for, or (for recursive calls) their types. In the
we're inferrined for, or (for recursive calls) their types. In the
above example this would first be the representation of the list
`[1]` and then, when recursing, just of `1`.
"""
@@ -351,16 +333,11 @@ class _ValueWrapperBase(HelperValueMixin):
class LazyValueWrapper(_ValueWrapperBase):
if TYPE_CHECKING:
@property
def _wrapped_value(self) -> Any:
return
else:
@safe_property
@memoize_method
def _wrapped_value(self):
with debug.increase_indent_cm('Resolve lazy value wrapper'):
return self._get_wrapped_value()
@safe_property
@memoize_method
def _wrapped_value(self):
with debug.increase_indent_cm('Resolve lazy value wrapper'):
return self._get_wrapped_value()
def __repr__(self):
return '<%s>' % (self.__class__.__name__)
@@ -518,7 +495,7 @@ class ValueSet:
return ValueSet.from_sets(_getitem(c, *args, **kwargs) for c in self._set)
def try_merge(self, function_name):
value_set = ValueSet([])
value_set = self.__class__([])
for c in self._set:
try:
method = getattr(c, function_name)
+2 -3
View File
@@ -14,9 +14,8 @@ def builtin_from_name(inference_state, string):
else:
filter_ = next(typing_builtins_module.get_filters())
name, = filter_.get(string)
# Most of the time there is only symbol, but sometimes there are different
# sys.version_infos, where there are multiple ones, just use the first one.
return next(iter(name.infer()))
value, = name.infer()
return value
class ExactValue(LazyValueWrapper):
+34 -78
View File
@@ -8,8 +8,6 @@ import warnings
import re
import builtins
import typing
from pathlib import Path
from typing import Optional, Tuple
from jedi.inference.compiled.getattr_static import getattr_static
@@ -40,7 +38,7 @@ NOT_CLASS_TYPES = (
MethodDescriptorType = type(str.replace)
WrapperDescriptorType = type(set.__iter__)
# `object.__subclasshook__` is an already executed descriptor.
object_class_dict = type.__dict__["__dict__"].__get__(object) # type: ignore[index]
object_class_dict = type.__dict__["__dict__"].__get__(object)
ClassMethodDescriptorType = type(object_class_dict['__subclasshook__'])
_sentinel = object()
@@ -147,7 +145,7 @@ class AccessPath:
self.accesses = accesses
def create_access_path(inference_state, obj) -> AccessPath:
def create_access_path(inference_state, obj):
access = create_access(inference_state, obj)
return AccessPath(access.get_access_path_tuples())
@@ -175,16 +173,16 @@ class DirectObjectAccess:
def _create_access(self, obj):
return create_access(self._inference_state, obj)
def _create_access_path(self, obj) -> AccessPath:
def _create_access_path(self, obj):
return create_access_path(self._inference_state, obj)
def py__bool__(self):
return bool(self._obj)
def py__file__(self) -> Optional[Path]:
def py__file__(self):
try:
return Path(self._obj.__file__)
except (AttributeError, TypeError):
return self._obj.__file__
except AttributeError:
return None
def py__doc__(self):
@@ -213,39 +211,18 @@ class DirectObjectAccess:
def py__getitem__all_values(self):
if isinstance(self._obj, dict):
return [self._create_access_path(v) for v in self._obj.values()]
if isinstance(self._obj, (list, tuple)):
return [self._create_access_path(v) for v in self._obj]
return self.py__iter__list()
if self.is_instance():
cls = DirectObjectAccess(self._inference_state, self._obj.__class__)
return cls.py__getitem__all_values()
try:
getitem = self._obj.__getitem__
except AttributeError:
pass
else:
annotation = DirectObjectAccess(self._inference_state, getitem).get_return_annotation()
if annotation is not None:
return [annotation]
return None
def py__simple_getitem__(self, index, *, safe=True):
if safe and type(self._obj) not in ALLOWED_GETITEM_TYPES:
def py__simple_getitem__(self, index):
if type(self._obj) not in ALLOWED_GETITEM_TYPES:
# Get rid of side effects, we won't call custom `__getitem__`s.
return None
return self._create_access_path(self._obj[index])
def py__iter__list(self):
try:
iter_method = self._obj.__iter__
except AttributeError:
if not hasattr(self._obj, '__getitem__'):
return None
else:
p = DirectObjectAccess(self._inference_state, iter_method).get_return_annotation()
if p is not None:
return [p]
if type(self._obj) not in ALLOWED_GETITEM_TYPES:
# Get rid of side effects, we won't call custom `__getitem__`s.
@@ -329,37 +306,33 @@ class DirectObjectAccess:
except TypeError:
return False
def is_allowed_getattr(self, name, safe=True) -> Tuple[bool, bool, Optional[AccessPath]]:
def is_allowed_getattr(self, name, unsafe=False):
# TODO this API is ugly.
if unsafe:
# Unsafe is mostly used to check for __getattr__/__getattribute__.
# getattr_static works for properties, but the underscore methods
# are just ignored (because it's safer and avoids more code
# execution). See also GH #1378.
# Avoid warnings, see comment in the next function.
with warnings.catch_warnings(record=True):
warnings.simplefilter("always")
try:
return hasattr(self._obj, name), False
except Exception:
# Obviously has an attribute (propably a property) that
# gets executed, so just avoid all exceptions here.
return False, False
try:
attr, is_get_descriptor = getattr_static(self._obj, name)
except AttributeError:
if not safe:
# Unsafe is mostly used to check for __getattr__/__getattribute__.
# getattr_static works for properties, but the underscore methods
# are just ignored (because it's safer and avoids more code
# execution). See also GH #1378.
# Avoid warnings, see comment in the next function.
with warnings.catch_warnings(record=True):
warnings.simplefilter("always")
try:
return hasattr(self._obj, name), False, None
except Exception:
# Obviously has an attribute (probably a property) that
# gets executed, so just avoid all exceptions here.
pass
return False, False, None
return False, False
else:
if is_get_descriptor and type(attr) not in ALLOWED_DESCRIPTOR_ACCESS:
if isinstance(attr, property):
if hasattr(attr.fget, '__annotations__'):
a = DirectObjectAccess(self._inference_state, attr.fget)
return True, True, a.get_return_annotation()
# In case of descriptors that have get methods we cannot return
# it's value, because that would mean code execution.
return True, True, None
return True, False, None
return True, True
return True, False
def getattr_paths(self, name, default=_sentinel):
try:
@@ -388,7 +361,7 @@ class DirectObjectAccess:
except AttributeError:
pass
else:
if module is not None and isinstance(module, str):
if module is not None:
try:
__import__(module)
# For some modules like _sqlite3, the __module__ for classes is
@@ -471,23 +444,18 @@ class DirectObjectAccess:
op = _OPERATORS[operator]
return self._create_access_path(op(self._obj, other_access._obj))
def get_annotation_name_and_args(self) -> tuple[str | None, tuple[AccessPath, ...]]:
def get_annotation_name_and_args(self):
"""
Returns Tuple[Optional[str], Tuple[AccessPath, ...]]
"""
name = None
args = ()
if type(self._obj) is typing.Union: # zuban: ignore[comparison-overlap] # TODO zuban
# This is mostly formatted like `int | str` and we therefor need to
# check the type.
args = typing.get_args(self._obj)
name = "Union"
elif safe_getattr(self._obj, '__module__', default='') == 'typing':
# Try regex first (works for most types)
if safe_getattr(self._obj, '__module__', default='') == 'typing':
m = re.match(r'typing.(\w+)\[', repr(self._obj))
if m is not None:
name = m.group(1)
import typing
if sys.version_info >= (3, 8):
args = typing.get_args(self._obj)
else:
@@ -498,18 +466,6 @@ class DirectObjectAccess:
return inspect.isclass(self._obj) and self._obj != type
def _annotation_to_str(self, annotation):
# In Python 3.14+, Union types are displayed as X | Y instead of Union[X, Y]
# We normalize to that for consistency
import typing
origin = typing.get_origin(annotation)
if origin is typing.Union:
# Get the args and format them as Union[...]
args = typing.get_args(annotation)
return ' | '.join(
self._annotation_to_str(arg) if hasattr(arg, '__origin__')
else getattr(arg, '__name__', str(arg))
for arg in args
)
return inspect.formatannotation(annotation)
def get_signature_params(self):
@@ -536,7 +492,7 @@ class DirectObjectAccess:
# the signature. In that case we just want a simple escape for now.
raise ValueError
def get_return_annotation(self) -> Optional[AccessPath]:
def get_return_annotation(self):
try:
o = self._obj.__annotations__.get('return')
except AttributeError:
+3 -6
View File
@@ -39,7 +39,7 @@ def _is_type(obj):
def _shadowed_dict(klass):
dict_attr = type.__dict__["__dict__"] # type: ignore[index]
dict_attr = type.__dict__["__dict__"]
for entry in _static_getmro(klass):
try:
class_dict = dict_attr.__get__(entry)["__dict__"]
@@ -54,7 +54,7 @@ def _shadowed_dict(klass):
def _static_getmro(klass):
mro = type.__dict__['__mro__'].__get__(klass) # type: ignore[index]
mro = type.__dict__['__mro__'].__get__(klass)
if not isinstance(mro, (tuple, list)):
# There are unfortunately no tests for this, I was not able to
# reproduce this in pure Python. However should still solve the issue
@@ -90,10 +90,7 @@ def getattr_static(obj, attr, default=_sentinel):
if not _is_type(obj):
klass = type(obj)
dict_attr = _shadowed_dict(klass)
# In Python 3.15+, __dict__ is a GetSetDescriptorType instead of being _sentinel
if (dict_attr is _sentinel
or type(dict_attr) is types.MemberDescriptorType
or type(dict_attr) is types.GetSetDescriptorType):
if (dict_attr is _sentinel or type(dict_attr) is types.MemberDescriptorType):
instance_result = _check_instance(obj, attr)
else:
klass = obj
+5 -5
View File
@@ -34,7 +34,7 @@ class MixedObject(ValueWrapper):
This combined logic makes it possible to provide more powerful REPL
completion. It allows side effects that are not noticable with the default
parser structure to still be completable.
parser structure to still be completeable.
The biggest difference from CompiledValue to MixedObject is that we are
generally dealing with Python code and not with C code. This will generate
@@ -142,9 +142,9 @@ class MixedObjectFilter(compiled.CompiledValueFilter):
super().__init__(inference_state, compiled_value)
self._tree_value = tree_value
def _create_name(self, *args, **kwargs):
def _create_name(self, name):
return MixedName(
super()._create_name(*args, **kwargs),
super()._create_name(name),
self._tree_value,
)
@@ -187,7 +187,7 @@ def _find_syntax_node_name(inference_state, python_object):
try:
python_object = _get_object_to_check(python_object)
path = inspect.getsourcefile(python_object)
except (OSError, TypeError):
except TypeError:
# The type might not be known (e.g. class_with_dict.__weakref__)
return None
path = None if path is None else Path(path)
@@ -267,7 +267,7 @@ def _find_syntax_node_name(inference_state, python_object):
@inference_state_function_cache()
def _create(inference_state, compiled_value, module_context):
# TODO accessing this is bad, but it probably doesn't matter that much,
# because we're working with interpreters only here.
# because we're working with interpreteters only here.
python_object = compiled_value.access_handle.access._obj
result = _find_syntax_node_name(inference_state, python_object)
if result is None:
+20 -144
View File
@@ -5,26 +5,8 @@ goals:
1. Making it safer - Segfaults and RuntimeErrors as well as stdout/stderr can
be ignored and dealt with.
2. Make it possible to handle different Python versions as well as virtualenvs.
The architecture here is briefly:
- For each Jedi `Environment` there is a corresponding subprocess which
operates within the target environment. If the subprocess dies it is replaced
at this level.
- `CompiledSubprocess` manages exactly one subprocess and handles communication
from the parent side.
- `Listener` runs within the subprocess, processing each request and yielding
results.
- `InterpreterEnvironment` provides an API which matches that of `Environment`,
but runs functionality inline rather than within a subprocess. It is thus
used both directly in places where a subprocess is unnecessary and/or
undesirable and also within subprocesses themselves.
- `InferenceStateSubprocess` (or `InferenceStateSameProcess`) provide high
level access to functionality within the subprocess from within the parent.
Each `InterpreterState` has an instance of one of these, provided by its
environment.
"""
import collections
import os
import sys
import queue
@@ -33,7 +15,6 @@ import traceback
import weakref
from functools import partial
from threading import Thread
from typing import Dict, TYPE_CHECKING, Any
from jedi._compatibility import pickle_dump, pickle_load
from jedi import debug
@@ -43,16 +24,13 @@ from jedi.inference.compiled.access import DirectObjectAccess, AccessPath, \
SignatureParam
from jedi.api.exceptions import InternalError
if TYPE_CHECKING:
from jedi.inference import InferenceState
_MAIN_PATH = os.path.join(os.path.dirname(__file__), '__main__.py')
PICKLE_PROTOCOL = 4
def _GeneralizedPopen(*args, **kwargs):
if sys.platform == "win32":
if os.name == 'nt':
try:
# Was introduced in Python 3.7.
CREATE_NO_WINDOW = subprocess.CREATE_NO_WINDOW
@@ -104,11 +82,10 @@ def _cleanup_process(process, thread):
class _InferenceStateProcess:
get_compiled_method_return: Any
def __init__(self, inference_state: 'InferenceState') -> None:
def __init__(self, inference_state):
self._inference_state_weakref = weakref.ref(inference_state)
self._handles: Dict[int, AccessHandle] = {}
self._inference_state_id = id(inference_state)
self._handles = {}
def get_or_create_access_handle(self, obj):
id_ = id(obj)
@@ -138,49 +115,11 @@ class InferenceStateSameProcess(_InferenceStateProcess):
class InferenceStateSubprocess(_InferenceStateProcess):
"""
API to functionality which will run in a subprocess.
This mediates the interaction between an `InferenceState` and the actual
execution of functionality running within a `CompiledSubprocess`. Available
functions are defined in `.functions`, though should be accessed via
attributes on this class of the same name.
This class is responsible for indicating that the `InferenceState` within
the subprocess can be removed once the corresponding instance in the parent
goes away.
"""
def __init__(
self,
inference_state: 'InferenceState',
compiled_subprocess: 'CompiledSubprocess',
) -> None:
def __init__(self, inference_state, compiled_subprocess):
super().__init__(inference_state)
self._used = False
self._compiled_subprocess = compiled_subprocess
# Opaque id we'll pass to the subprocess to identify the context (an
# `InferenceState`) which should be used for the request. This allows us
# to make subsequent requests which operate on results from previous
# ones, while keeping a single subprocess which can work with several
# contexts in the parent process. Once it is no longer needed(i.e: when
# this class goes away), we also use this id to indicate that the
# subprocess can discard the context.
#
# Note: this id is deliberately coupled to this class (and not to
# `InferenceState`) as this class manages access handle mappings which
# must correspond to those in the subprocess. This approach also avoids
# race conditions from successive `InferenceState`s with the same object
# id (as observed while adding support for Python 3.13).
#
# This value does not need to be the `id()` of this instance, we merely
# need to ensure that it enables the (visible) lifetime of the context
# within the subprocess to match that of this class. We therefore also
# depend on the semantics of `CompiledSubprocess.delete_inference_state`
# for correctness.
self._inference_state_id = id(self)
def __getattr__(self, name):
func = _get_function(name)
@@ -188,7 +127,7 @@ class InferenceStateSubprocess(_InferenceStateProcess):
self._used = True
result = self._compiled_subprocess.run(
self._inference_state_id,
self._inference_state_weakref(),
func,
args=args,
kwargs=kwargs,
@@ -224,23 +163,12 @@ class InferenceStateSubprocess(_InferenceStateProcess):
class CompiledSubprocess:
"""
A subprocess which runs inference within a target environment.
This class manages the interface to a single instance of such a process as
well as the lifecycle of the process itself. See `.__main__` and `Listener`
for the implementation of the subprocess and details of the protocol.
A single live instance of this is maintained by `jedi.api.environment.Environment`,
so that typically a single subprocess is used at a time.
"""
is_crashed = False
def __init__(self, executable, env_vars=None):
self._executable = executable
self._env_vars = env_vars
self._inference_state_deletion_queue = collections.deque()
self._inference_state_deletion_queue = queue.deque()
self._cleanup_callable = lambda: None
def __repr__(self):
@@ -284,18 +212,18 @@ class CompiledSubprocess:
t)
return process
def run(self, inference_state_id, function, args=(), kwargs={}):
def run(self, inference_state, function, args=(), kwargs={}):
# Delete old inference_states.
while True:
try:
delete_id = self._inference_state_deletion_queue.pop()
inference_state_id = self._inference_state_deletion_queue.pop()
except IndexError:
break
else:
self._send(delete_id, None)
self._send(inference_state_id, None)
assert callable(function)
return self._send(inference_state_id, function, args, kwargs)
return self._send(id(inference_state), function, args, kwargs)
def get_sys_path(self):
return self._send(None, functions.get_sys_path, (), {})
@@ -343,65 +271,21 @@ class CompiledSubprocess:
def delete_inference_state(self, inference_state_id):
"""
Indicate that an inference state (in the subprocess) is no longer
needed.
The state corresponding to the given id will become inaccessible and the
id may safely be re-used to refer to a different context.
Note: it is not guaranteed that the corresponding state will actually be
deleted immediately.
Currently we are not deleting inference_state instantly. They only get
deleted once the subprocess is used again. It would probably a better
solution to move all of this into a thread. However, the memory usage
of a single inference_state shouldn't be that high.
"""
# Warning: if changing the semantics of context deletion see the comment
# in `InferenceStateSubprocess.__init__` regarding potential race
# conditions.
# Currently we are not deleting the related state instantly. They only
# get deleted once the subprocess is used again. It would probably a
# better solution to move all of this into a thread. However, the memory
# usage of a single inference_state shouldn't be that high.
# With an argument - the inference_state gets deleted.
self._inference_state_deletion_queue.append(inference_state_id)
class Listener:
"""
Main loop for the subprocess which actually does the inference.
This class runs within the target environment. It listens to instructions
from the parent process, runs inference and returns the results.
The subprocess has a long lifetime and is expected to process several
requests, including for different `InferenceState` instances in the parent.
See `CompiledSubprocess` for the parent half of the system.
Communication is via pickled data sent serially over stdin and stdout.
Stderr is read only if the child process crashes.
The request protocol is a 4-tuple of:
* inference_state_id | None: an opaque identifier of the parent's
`InferenceState`. An `InferenceState` operating over an
`InterpreterEnvironment` is created within this process for each of
these, ensuring that each parent context has a corresponding context
here. This allows context to be persisted between requests. Unless
`None`, the local `InferenceState` will be passed to the given function
as the first positional argument.
* function | None: the function to run. This is expected to be a member of
`.functions`. `None` indicates that the corresponding inference state is
no longer needed and should be dropped.
* args: positional arguments to the `function`. If any of these are
`AccessHandle` instances they will be adapted to the local
`InferenceState` before being passed.
* kwargs: keyword arguments to the `function`. If any of these are
`AccessHandle` instances they will be adapted to the local
`InferenceState` before being passed.
The result protocol is a 3-tuple of either:
* (False, None, function result): if the function returns without error, or
* (True, traceback, exception): if the function raises an exception
"""
def __init__(self):
self._inference_states = {}
# TODO refactor so we don't need to process anymore just handle
# controlling.
self._process = _InferenceStateProcess(Listener)
def _get_inference_state(self, function, inference_state_id):
from jedi.inference import InferenceState
@@ -423,9 +307,6 @@ class Listener:
if inference_state_id is None:
return function(*args, **kwargs)
elif function is None:
# Warning: if changing the semantics of context deletion see the comment
# in `InferenceStateSubprocess.__init__` regarding potential race
# conditions.
del self._inference_states[inference_state_id]
else:
inference_state = self._get_inference_state(function, inference_state_id)
@@ -466,12 +347,7 @@ class Listener:
class AccessHandle:
def __init__(
self,
subprocess: _InferenceStateProcess,
access: DirectObjectAccess,
id_: int,
) -> None:
def __init__(self, subprocess, access, id_):
self.access = access
self._subprocess = subprocess
self.id = id_
@@ -3,6 +3,10 @@ import sys
from importlib.abc import MetaPathFinder
from importlib.machinery import PathFinder
# Remove the first entry, because it's simply a directory entry that equals
# this directory.
del sys.path[0]
def _get_paths():
# Get the path to jedi.
@@ -17,11 +21,11 @@ class _ExactImporter(MetaPathFinder):
def __init__(self, path_dct):
self._path_dct = path_dct
def find_spec(self, fullname, path=None, target=None):
def find_module(self, fullname, path=None):
if path is None and fullname in self._path_dct:
p = self._path_dct[fullname]
spec = PathFinder.find_spec(fullname, path=[p], target=target)
return spec
loader = PathFinder.find_module(fullname, path=[p])
return loader
return None
+22 -45
View File
@@ -2,11 +2,12 @@ import sys
import os
import inspect
import importlib
import warnings
from pathlib import Path
from zipfile import ZipFile
from zipimport import zipimporter, ZipImportError
from zipimport import zipimporter
from importlib.machinery import all_suffixes
from jedi._compatibility import cast_path
from jedi.inference.compiled import access
from jedi import debug
from jedi import parser_utils
@@ -14,7 +15,7 @@ from jedi.file_io import KnownContentFileIO, ZipFileIO
def get_sys_path():
return sys.path
return list(map(cast_path, sys.path))
def load_module(inference_state, **kwargs):
@@ -92,22 +93,15 @@ def _iter_module_names(inference_state, paths):
# Python modules/packages
for path in paths:
try:
dir_entries = ((entry.name, entry.is_dir()) for entry in os.scandir(path))
dirs = os.scandir(path)
except OSError:
try:
zip_import_info = zipimporter(path)
# Unfortunately, there is no public way to access zipimporter's
# private _files member. We therefore have to use a
# custom function to iterate over the files.
dir_entries = _zip_list_subdirectory(
zip_import_info.archive, zip_import_info.prefix)
except ZipImportError:
# The file might not exist or reading it might lead to an error.
debug.warning("Not possible to list directory: %s", path)
continue
for name, is_dir in dir_entries:
# The file might not exist or reading it might lead to an error.
debug.warning("Not possible to list directory: %s", path)
continue
for dir_entry in dirs:
name = dir_entry.name
# First Namespaces then modules/stubs
if is_dir:
if dir_entry.is_dir():
# pycache is obviously not an interesting namespace. Also the
# name must be a valid identifier.
if name != '__pycache__' and name.isidentifier():
@@ -150,18 +144,11 @@ def _find_module(string, path=None, full_name=None, is_global_search=True):
spec = find_spec(string, p)
if spec is not None:
if spec.origin == "frozen":
continue
loader = spec.loader
if loader is None and not spec.has_location:
# This is a namespace package.
full_name = string if not path else full_name
implicit_ns_info = ImplicitNSInfo(
full_name,
spec.submodule_search_locations._path, # type: ignore[union-attr]
)
implicit_ns_info = ImplicitNSInfo(full_name, spec.submodule_search_locations._path)
return implicit_ns_info, True
break
@@ -169,16 +156,17 @@ def _find_module(string, path=None, full_name=None, is_global_search=True):
def _find_module_py33(string, path=None, loader=None, full_name=None, is_global_search=True):
if not loader:
spec = importlib.machinery.PathFinder.find_spec(string, path)
if spec is not None:
loader = spec.loader
loader = loader or importlib.machinery.PathFinder.find_module(string, path)
if loader is None and path is None: # Fallback to find builtins
try:
spec = importlib.util.find_spec(string)
if spec is not None:
loader = spec.loader
with warnings.catch_warnings(record=True):
# Mute "DeprecationWarning: Use importlib.util.find_spec()
# instead." While we should replace that in the future, it's
# probably good to wait until we deprecate Python 3.3, since
# it was added in Python 3.4 and find_loader hasn't been
# removed in 3.6.
loader = importlib.find_loader(string)
except ValueError as e:
# See #491. Importlib might raise a ValueError, to avoid this, we
# just raise an ImportError to fix the issue.
@@ -202,7 +190,7 @@ def _from_loader(loader, string):
except AttributeError:
return None, is_package
else:
module_path = get_filename(string)
module_path = cast_path(get_filename(string))
# To avoid unicode and read bytes, "overwrite" loader.get_source if
# possible.
@@ -224,7 +212,7 @@ def _from_loader(loader, string):
if code is None:
return None, is_package
if isinstance(loader, zipimporter):
return ZipFileIO(module_path, code, Path(loader.archive)), is_package
return ZipFileIO(module_path, code, Path(cast_path(loader.archive))), is_package
return KnownContentFileIO(module_path, code), is_package
@@ -242,17 +230,6 @@ def _get_source(loader, fullname):
name=fullname)
def _zip_list_subdirectory(zip_path, zip_subdir_path):
zip_file = ZipFile(zip_path)
zip_subdir_path = Path(zip_subdir_path)
zip_content_file_paths = zip_file.namelist()
for raw_file_name in zip_content_file_paths:
file_path = Path(raw_file_name)
if file_path.parent == zip_subdir_path:
file_path = file_path.relative_to(zip_subdir_path)
yield file_path.name, raw_file_name.endswith("/")
class ImplicitNSInfo:
"""Stores information returned from an implicit namespace spec"""
def __init__(self, name, paths):
+36 -46
View File
@@ -5,10 +5,10 @@ import re
from functools import partial
from inspect import Parameter
from pathlib import Path
from typing import Optional
from jedi import debug
from jedi.inference.utils import to_list
from jedi._compatibility import cast_path
from jedi.cache import memoize_method
from jedi.inference.filters import AbstractFilter
from jedi.inference.names import AbstractNameDefinition, ValueNameMixin, \
@@ -51,10 +51,11 @@ class CompiledValue(Value):
def py__call__(self, arguments):
return_annotation = self.access_handle.get_return_annotation()
if return_annotation is not None:
# TODO the return annotation may also be a string.
return create_from_access_path(
self.inference_state,
return_annotation
).execute_annotation(arguments.context)
).execute_annotation()
try:
self.access_handle.getattr_paths('__call__')
@@ -162,14 +163,11 @@ class CompiledValue(Value):
def py__simple_getitem__(self, index):
with reraise_getitem_errors(IndexError, KeyError, TypeError):
try:
access = self.access_handle.py__simple_getitem__(
index,
safe=not self.inference_state.allow_unsafe_executions
)
access = self.access_handle.py__simple_getitem__(index)
except AttributeError:
return super().py__simple_getitem__(index)
if access is None:
return super().py__simple_getitem__(index)
return NO_VALUES
return ValueSet([create_from_access_path(self.inference_state, access)])
@@ -241,7 +239,7 @@ class CompiledValue(Value):
except TypeError:
return NO_VALUES
def execute_annotation(self, context):
def execute_annotation(self):
if self.access_handle.get_repr() == 'None':
# None as an annotation doesn't need to be executed.
return ValueSet([self])
@@ -252,9 +250,7 @@ class CompiledValue(Value):
for path in args
]
if name == 'Union':
return ValueSet.from_sets(
arg.execute_annotation(context)
for arg in arguments)
return ValueSet.from_sets(arg.execute_annotation() for arg in arguments)
elif name:
# While with_generics only exists on very specific objects, we
# should probably be fine, because we control all the typing
@@ -262,8 +258,8 @@ class CompiledValue(Value):
return ValueSet([
v.with_generics(arguments)
for v in self.inference_state.typing_module.py__getattribute__(name)
]).execute_annotation(context)
return super().execute_annotation(context)
]).execute_annotation()
return super().execute_annotation()
def negate(self):
return create_from_access_path(self.inference_state, self.access_handle.negate())
@@ -297,7 +293,10 @@ class CompiledModule(CompiledValue):
return CompiledModuleContext(self)
def py__path__(self):
return self.access_handle.py__path__()
paths = self.access_handle.py__path__()
if paths is None:
return None
return map(cast_path, paths)
def is_package(self):
return self.py__path__() is not None
@@ -310,17 +309,19 @@ class CompiledModule(CompiledValue):
return ()
return tuple(name.split('.'))
def py__file__(self) -> Optional[Path]:
return self.access_handle.py__file__() # type: ignore[no-any-return]
def py__file__(self):
path = cast_path(self.access_handle.py__file__())
if path is None:
return None
return Path(path)
class CompiledName(AbstractNameDefinition):
def __init__(self, inference_state, parent_value, name, is_descriptor):
def __init__(self, inference_state, parent_value, name):
self._inference_state = inference_state
self.parent_context = parent_value.as_context()
self._parent_value = parent_value
self.string_name = name
self.is_descriptor = is_descriptor
def py__doc__(self):
return self.infer_compiled_value().py__doc__()
@@ -347,11 +348,6 @@ class CompiledName(AbstractNameDefinition):
@property
def api_type(self):
if self.is_descriptor:
# In case of properties we want to avoid executions as much as
# possible. Since the api_type can be wrong for other reasons
# anyway, we just return instance here.
return "instance"
return self.infer_compiled_value().api_type
def infer(self):
@@ -442,10 +438,9 @@ class CompiledValueFilter(AbstractFilter):
def get(self, name):
access_handle = self.compiled_value.access_handle
safe = not self._inference_state.allow_unsafe_executions
return self._get(
name,
lambda name: access_handle.is_allowed_getattr(name, safe=safe),
lambda name, unsafe: access_handle.is_allowed_getattr(name, unsafe),
lambda name: name in access_handle.dir(),
check_has_attribute=True
)
@@ -454,40 +449,36 @@ class CompiledValueFilter(AbstractFilter):
"""
To remove quite a few access calls we introduced the callback here.
"""
has_attribute, is_descriptor, property_return_annotation = allowed_getattr_callback(
name,
)
if property_return_annotation is not None:
values = create_from_access_path(
self._inference_state,
property_return_annotation
).execute_annotation(None)
if values:
return [CompiledValueName(v, name) for v in values]
if self._inference_state.allow_descriptor_getattr:
pass
has_attribute, is_descriptor = allowed_getattr_callback(
name,
unsafe=self._inference_state.allow_descriptor_getattr
)
if check_has_attribute and not has_attribute:
return []
if (is_descriptor or not has_attribute) \
and not self._inference_state.allow_unsafe_executions:
and not self._inference_state.allow_descriptor_getattr:
return [self._get_cached_name(name, is_empty=True)]
if self.is_instance and not in_dir_callback(name):
return []
return [self._get_cached_name(name, is_descriptor=is_descriptor)]
return [self._get_cached_name(name)]
@memoize_method
def _get_cached_name(self, name, is_empty=False, *, is_descriptor=False):
def _get_cached_name(self, name, is_empty=False):
if is_empty:
return EmptyCompiledName(self._inference_state, name)
else:
return self._create_name(name, is_descriptor=is_descriptor)
return self._create_name(name)
def values(self):
from jedi.inference.compiled import builtin_from_name
names = []
needs_type_completions, dir_infos = self.compiled_value.access_handle.get_dir_infos()
# We could use `safe=False` here as well, especially as a parameter to
# We could use `unsafe` here as well, especially as a parameter to
# get_dir_infos. But this would lead to a lot of property executions
# that are probably not wanted. The drawback for this is that we
# have a different name for `get` and `values`. For `get` we always
@@ -495,7 +486,7 @@ class CompiledValueFilter(AbstractFilter):
for name in dir_infos:
names += self._get(
name,
lambda name: dir_infos[name],
lambda name, unsafe: dir_infos[name],
lambda name: name in dir_infos,
)
@@ -505,12 +496,11 @@ class CompiledValueFilter(AbstractFilter):
names += filter.values()
return names
def _create_name(self, name, is_descriptor):
def _create_name(self, name):
return CompiledName(
self._inference_state,
self.compiled_value,
name,
is_descriptor,
name
)
def __repr__(self):
@@ -594,7 +584,7 @@ def create_from_name(inference_state, compiled_value, name):
value = create_cached_compiled_value(
inference_state,
access_path,
parent_context=None if value is None else value.as_context(), # type: ignore # TODO
parent_context=None if value is None else value.as_context(),
)
return value
@@ -612,7 +602,7 @@ def create_from_access_path(inference_state, access_path):
value = create_cached_compiled_value(
inference_state,
access,
parent_context=None if value is None else value.as_context() # type: ignore # TODO
parent_context=None if value is None else value.as_context()
)
return value
+12 -21
View File
@@ -1,8 +1,7 @@
from abc import abstractmethod
from contextlib import contextmanager
from pathlib import Path
from typing import Optional, Any
from parso.tree import search_ancestor
from parso.python.tree import Name
from jedi.inference.filters import ParserTreeFilter, MergedFilter, \
@@ -16,8 +15,6 @@ from jedi import parser_utils
class AbstractContext:
# Must be defined: inference_state and tree_node and parent_context as an attribute/property
tree_node: Any
parent_context: Any
def __init__(self, inference_state):
self.inference_state = inference_state
@@ -220,13 +217,6 @@ class ValueContext(AbstractContext):
class TreeContextMixin:
tree_node: Any
is_module: Any
get_value: Any
inference_state: Any
is_class: Any
parent_context: Any
def infer_node(self, node):
from jedi.inference.syntax_tree import infer_node
return infer_node(self, node)
@@ -298,7 +288,7 @@ class TreeContextMixin:
def create_name(self, tree_name):
definition = tree_name.get_definition()
if definition and definition.type == 'param' and definition.name == tree_name:
funcdef = definition.search_ancestor('funcdef', 'lambdef')
funcdef = search_ancestor(definition, 'funcdef', 'lambdef')
func = self.create_value(funcdef)
return AnonymousParamName(func, tree_name)
else:
@@ -309,6 +299,7 @@ class TreeContextMixin:
class FunctionContext(TreeContextMixin, ValueContext):
def get_filters(self, until_position=None, origin_scope=None):
yield ParserTreeFilter(
self.inference_state,
parent_context=self,
until_position=until_position,
origin_scope=origin_scope
@@ -316,8 +307,8 @@ class FunctionContext(TreeContextMixin, ValueContext):
class ModuleContext(TreeContextMixin, ValueContext):
def py__file__(self) -> Optional[Path]:
return self._value.py__file__() # type: ignore[no-any-return]
def py__file__(self):
return self._value.py__file__()
def get_filters(self, until_position=None, origin_scope=None):
filters = self._value.get_filters(origin_scope)
@@ -334,7 +325,7 @@ class ModuleContext(TreeContextMixin, ValueContext):
yield from filters
def get_global_filter(self):
return GlobalNameFilter(self)
return GlobalNameFilter(self, self.tree_node)
@property
def string_names(self):
@@ -364,8 +355,8 @@ class NamespaceContext(TreeContextMixin, ValueContext):
def string_names(self):
return self._value.string_names
def py__file__(self) -> Optional[Path]:
return self._value.py__file__() # type: ignore[no-any-return]
def py__file__(self):
return self._value.py__file__()
class ClassContext(TreeContextMixin, ValueContext):
@@ -414,8 +405,8 @@ class CompiledModuleContext(CompiledContext):
def string_names(self):
return self._value.string_names
def py__file__(self) -> Optional[Path]:
return self._value.py__file__() # type: ignore[no-any-return]
def py__file__(self):
return self._value.py__file__()
def _get_global_filters_for_name(context, name_or_none, position):
@@ -423,13 +414,13 @@ def _get_global_filters_for_name(context, name_or_none, position):
# function and get inferred in the value before the function. So
# make sure to exclude the function/class name.
if name_or_none is not None:
ancestor = name_or_none.search_ancestor('funcdef', 'classdef', 'lambdef')
ancestor = search_ancestor(name_or_none, 'funcdef', 'classdef', 'lambdef')
lambdef = None
if ancestor == 'lambdef':
# For lambdas it's even more complicated since parts will
# be inferred later.
lambdef = ancestor
ancestor = name_or_none.search_ancestor('funcdef', 'classdef')
ancestor = search_ancestor(name_or_none, 'funcdef', 'classdef')
if ancestor is not None:
colon = ancestor.children[-2]
if position is not None and position < colon.start_pos:
-21
View File
@@ -1,21 +0,0 @@
from jedi.inference.value import ModuleValue
from jedi.inference.context import ModuleContext
class DocstringModule(ModuleValue):
def __init__(self, in_module_context, **kwargs):
super().__init__(**kwargs)
self._in_module_context = in_module_context
def _as_context(self):
return DocstringModuleContext(self, self._in_module_context)
class DocstringModuleContext(ModuleContext):
def __init__(self, module_value, in_module_context):
super().__init__(module_value)
self._in_module_context = in_module_context
def get_filters(self, until_position=None, origin_scope=None):
yield from super().get_filters(until_position=until_position)
yield from self._in_module_context.get_filters()
+34 -20
View File
@@ -17,10 +17,12 @@ annotations.
import re
import warnings
from textwrap import dedent
from parso import parse, ParserSyntaxError
from jedi import debug
from jedi.common import indent_block
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.base_value import iterator_to_value_set, ValueSet, \
NO_VALUES
@@ -48,7 +50,7 @@ def _get_numpy_doc_string_cls():
global _numpy_doc_string_cache
if isinstance(_numpy_doc_string_cache, (ImportError, SyntaxError)):
raise _numpy_doc_string_cache
from numpydoc.docscrape import NumpyDocString # type: ignore[import, unused-ignore]
from numpydoc.docscrape import NumpyDocString # type: ignore[import]
_numpy_doc_string_cache = NumpyDocString
return _numpy_doc_string_cache
@@ -109,7 +111,7 @@ def _expand_typestr(type_str):
yield type_str.split('of')[0]
# Check if type has is a set of valid literal values eg: {'C', 'F', 'A'}
elif type_str.startswith('{'):
node = parse(type_str, version='3.13').children[0]
node = parse(type_str, version='3.7').children[0]
if node.type == 'atom':
for leaf in getattr(node.children[1], "children", []):
if leaf.type == 'number':
@@ -180,40 +182,52 @@ def _strip_rst_role(type_str):
def _infer_for_statement_string(module_context, string):
code = dedent("""
def pseudo_docstring_stuff():
'''
Create a pseudo function for docstring statements.
Need this docstring so that if the below part is not valid Python this
is still a function.
'''
{}
""")
if string is None:
return []
potential_imports = re.findall(r'((?:\w+\.)*\w+)\.', string)
# Try to import module part in dotted name.
# (e.g., 'threading' in 'threading.Thread').
imports = "\n".join(f"import {p}" for p in potential_imports)
string = f'{imports}\n{string}'
for element in re.findall(r'((?:\w+\.)*\w+)\.', string):
# Try to import module part in dotted name.
# (e.g., 'threading' in 'threading.Thread').
string = 'import %s\n' % element + string
debug.dbg('Parse docstring code %s', string, color='BLUE')
grammar = module_context.inference_state.grammar
try:
module = grammar.parse(string, error_recovery=False)
module = grammar.parse(code.format(indent_block(string)), error_recovery=False)
except ParserSyntaxError:
return []
try:
# It's not the last item, because that's an end marker.
stmt = module.children[-2]
funcdef = next(module.iter_funcdefs())
# First pick suite, then simple_stmt and then the node,
# which is also not the last item, because there's a newline.
stmt = funcdef.children[-1].children[-1].children[-2]
except (AttributeError, IndexError):
return []
if stmt.type not in ('name', 'atom', 'atom_expr'):
return []
# Here we basically use a fake module that also uses the filters in
# the actual module.
from jedi.inference.docstring_utils import DocstringModule
m = DocstringModule(
in_module_context=module_context,
inference_state=module_context.inference_state,
module_node=module,
code_lines=[],
from jedi.inference.value import FunctionValue
function_value = FunctionValue(
module_context.inference_state,
module_context,
funcdef
)
return list(_execute_types_in_stmt(m.as_context(), stmt))
func_execution_context = function_value.as_context()
# Use the module of the param.
# TODO this module is not the module of the param in case of a function
# call. In that case it's the module of the function call.
# stuffed with content from a function call.
return list(_execute_types_in_stmt(func_execution_context, stmt))
def _execute_types_in_stmt(module_context, stmt):
@@ -246,7 +260,7 @@ def _execute_array_values(inference_state, array):
cls = FakeTuple if array.array_type == 'tuple' else FakeList
return {cls(inference_state, values)}
else:
return array.execute_annotation(None)
return array.execute_annotation()
@inference_state_method_cache()
+5 -5
View File
@@ -66,11 +66,11 @@ def dynamic_param_lookup(function_value, param_index):
have to look for all calls to ``func`` to find out what ``foo`` possibly
is.
"""
if not function_value.inference_state.do_dynamic_params_search:
return NO_VALUES
funcdef = function_value.tree_node
if not settings.dynamic_params:
return NO_VALUES
path = function_value.get_root_context().py__file__()
if path is not None and is_stdlib_path(path):
# We don't want to search for references in the stdlib. Usually people
@@ -109,7 +109,7 @@ def _search_function_arguments(module_context, funcdef, string_name):
if string_name == '__init__':
cls = get_parent_scope(funcdef)
if cls.type == 'classdef':
string_name = cls.name.value # type: ignore[union-attr]
string_name = cls.name.value
compare_node = cls
found_arguments = False
@@ -203,7 +203,7 @@ def _check_name_for_execution(inference_state, context, compare_node, name, trai
# Here we're trying to find decorators by checking the first
# parameter. It's not very generic though. Should find a better
# solution that also applies to nested decorators.
param_names = value.parent_context.get_param_names() # type: ignore[attr-defined]
param_names = value.parent_context.get_param_names()
if len(param_names) != 1:
continue
values = param_names[0].infer()
+23 -48
View File
@@ -3,21 +3,22 @@ Filters are objects that you can use to filter names in different scopes. They
are needed for name resolution.
"""
from abc import abstractmethod
from typing import MutableMapping, Type, Any
from typing import List, MutableMapping, Type
import weakref
from parso.tree import search_ancestor
from parso.python.tree import Name, UsedNamesMapping
from jedi.inference import flow_analysis
from jedi.inference.base_value import ValueSet, ValueWrapper, \
LazyValueWrapper
from jedi.parser_utils import get_cached_parent_scope, get_parso_cache_node
from jedi.parser_utils import get_cached_parent_scope
from jedi.inference.utils import to_list
from jedi.inference.names import TreeNameDefinition, ParamName, \
AnonymousParamName, AbstractNameDefinition, NameWrapper
_definition_name_cache: 'MutableMapping[UsedNamesMapping, dict[str, tuple[Name, ...]]]' \
= weakref.WeakKeyDictionary()
_definition_name_cache: MutableMapping[UsedNamesMapping, List[Name]]
_definition_name_cache = weakref.WeakKeyDictionary()
class AbstractFilter:
@@ -53,15 +54,11 @@ class FilterWrapper:
return self.wrap_names(self._wrapped_filter.values())
def _get_definition_names(parso_cache_node, used_names, name_key):
if parso_cache_node is None:
names = used_names.get(name_key, ())
return tuple(name for name in names if name.is_definition(include_setitem=True))
def _get_definition_names(used_names, name_key):
try:
for_module = _definition_name_cache[parso_cache_node]
for_module = _definition_name_cache[used_names]
except KeyError:
for_module = _definition_name_cache[parso_cache_node] = {}
for_module = _definition_name_cache[used_names] = {}
try:
return for_module[name_key]
@@ -73,40 +70,18 @@ def _get_definition_names(parso_cache_node, used_names, name_key):
return result
class _AbstractUsedNamesFilter(AbstractFilter):
class AbstractUsedNamesFilter(AbstractFilter):
name_class = TreeNameDefinition
def __init__(self, parent_context, node_context=None):
if node_context is None:
node_context = parent_context
self._node_context = node_context
self._parser_scope = node_context.tree_node
module_context = node_context.get_root_context()
# It is quite hacky that we have to use that. This is for caching
# certain things with a WeakKeyDictionary. However, parso intentionally
# uses slots (to save memory) and therefore we end up with having to
# have a weak reference to the object that caches the tree.
#
# Previously we have tried to solve this by using a weak reference onto
# used_names. However that also does not work, because it has a
# reference from the module, which itself is referenced by any node
# through parents.
path = module_context.py__file__()
if path is None:
# If the path is None, there is no guarantee that parso caches it.
self._parso_cache_node = None
else:
self._parso_cache_node = get_parso_cache_node(
module_context.inference_state.latest_grammar
if module_context.is_stub() else module_context.inference_state.grammar,
path
)
self._used_names = module_context.tree_node.get_used_names()
def __init__(self, parent_context, parser_scope):
self._parser_scope = parser_scope
self._module_node = self._parser_scope.get_root_node()
self._used_names = self._module_node.get_used_names()
self.parent_context = parent_context
def get(self, name):
return self._convert_names(self._filter(
_get_definition_names(self._parso_cache_node, self._used_names, name),
_get_definition_names(self._used_names, name),
))
def _convert_names(self, names):
@@ -117,7 +92,7 @@ class _AbstractUsedNamesFilter(AbstractFilter):
name
for name_key in self._used_names
for name in self._filter(
_get_definition_names(self._parso_cache_node, self._used_names, name_key),
_get_definition_names(self._used_names, name_key),
)
)
@@ -125,7 +100,7 @@ class _AbstractUsedNamesFilter(AbstractFilter):
return '<%s: %s>' % (self.__class__.__name__, self.parent_context)
class ParserTreeFilter(_AbstractUsedNamesFilter):
class ParserTreeFilter(AbstractUsedNamesFilter):
def __init__(self, parent_context, node_context=None, until_position=None,
origin_scope=None):
"""
@@ -134,7 +109,10 @@ class ParserTreeFilter(_AbstractUsedNamesFilter):
value, but for some type inference it's important to have a local
value of the other classes.
"""
super().__init__(parent_context, node_context)
if node_context is None:
node_context = parent_context
super().__init__(parent_context, node_context.tree_node)
self._node_context = node_context
self._origin_scope = origin_scope
self._until_position = until_position
@@ -148,7 +126,7 @@ class ParserTreeFilter(_AbstractUsedNamesFilter):
if parent.type == 'trailer':
return False
base_node = parent if parent.type in ('classdef', 'funcdef') else name
return get_cached_parent_scope(self._parso_cache_node, base_node) == self._parser_scope
return get_cached_parent_scope(self._used_names, base_node) == self._parser_scope
def _check_flows(self, names):
for name in sorted(names, key=lambda name: name.start_pos, reverse=True):
@@ -180,7 +158,7 @@ class _FunctionExecutionFilter(ParserTreeFilter):
@to_list
def _convert_names(self, names):
for name in names:
param = name.search_ancestor('param')
param = search_ancestor(name, 'param')
# Here we don't need to check if the param is a default/annotation,
# because those are not definitions and never make it to this
# point.
@@ -204,7 +182,7 @@ class AnonymousFunctionExecutionFilter(_FunctionExecutionFilter):
return AnonymousParamName(self._function_value, name)
class GlobalNameFilter(_AbstractUsedNamesFilter):
class GlobalNameFilter(AbstractUsedNamesFilter):
def get(self, name):
try:
names = self._used_names[name]
@@ -346,9 +324,6 @@ class _OverwriteMeta(type):
class _AttributeOverwriteMixin:
overwritten_methods: Any
_wrapped_value: Any
def get_filters(self, *args, **kwargs):
yield SpecialMethodFilter(self, self.overwritten_methods, self._wrapped_value)
yield from self._wrapped_value.get_filters(*args, **kwargs)
+2 -1
View File
@@ -15,6 +15,7 @@ Unfortunately every other thing is being ignored (e.g. a == '' would be easy to
check for -> a is a string). There's big potential in these checks.
"""
from parso.tree import search_ancestor
from parso.python.tree import Name
from jedi import settings
@@ -75,7 +76,7 @@ def check_flow_information(value, flow, search_name, pos):
])
for name in names:
ass = name.search_ancestor('assert_stmt')
ass = search_ancestor(name, 'assert_stmt')
if ass is not None:
result = _check_isinstance_type(value, ass.assertion, search_name)
if result is not None:
+18 -54
View File
@@ -32,20 +32,17 @@ def infer_annotation(context, annotation):
Also checks for forward references (strings)
"""
value_set = context.infer_node(annotation)
if len(value_set) == 0:
debug.warning(
"Inferred typing index %s should lead to 1 object, not %s" % (annotation, value_set))
if len(value_set) != 1:
debug.warning("Inferred typing index %s should lead to 1 object, "
" not %s" % (annotation, value_set))
return value_set
strings_removed = NO_VALUES
for part in value_set:
if is_string(part):
result = _get_forward_reference_node(context, part.get_safe_value())
if result is not None:
strings_removed |= context.infer_node(result)
continue
strings_removed |= ValueSet([part])
return strings_removed
inferred_value = list(value_set)[0]
if is_string(inferred_value):
result = _get_forward_reference_node(context, inferred_value.get_safe_value())
if result is not None:
return context.infer_node(result)
return value_set
def _infer_annotation_string(context, string, index=None):
@@ -199,43 +196,13 @@ def py__annotations__(funcdef):
return dct
def resolve_forward_references(context, all_annotations):
def resolve(node):
if node is None or node.type != 'string':
return node
node = _get_forward_reference_node(
context,
context.inference_state.compiled_subprocess.safe_literal_eval(
node.value,
),
)
if node is None:
# There was a string, but it's not a valid annotation
return None
# The forward reference tree has an additional root node ('eval_input')
# that we don't want. Extract the node we do want, that is equivalent to
# the nodes returned by `py__annotations__` for a non-quoted node.
node = node.children[0]
return node
return {name: resolve(node) for name, node in all_annotations.items()}
@inference_state_method_cache()
def infer_return_types(function, arguments):
"""
Infers the type of a function's return value,
according to type annotations.
"""
context = function.get_default_param_context()
all_annotations = resolve_forward_references(
context,
py__annotations__(function.tree_node),
)
all_annotations = py__annotations__(function.tree_node)
annotation = all_annotations.get("return", None)
if annotation is None:
# If there is no Python 3-type annotation, look for an annotation
@@ -250,14 +217,15 @@ def infer_return_types(function, arguments):
return NO_VALUES
return _infer_annotation_string(
context,
function.get_default_param_context(),
match.group(1).strip()
).execute_annotation(context)
).execute_annotation()
context = function.get_default_param_context()
unknown_type_vars = find_unknown_type_vars(context, annotation)
annotation_values = infer_annotation(context, annotation)
if not unknown_type_vars:
return annotation_values.execute_annotation(context)
return annotation_values.execute_annotation()
type_var_dict = infer_type_vars_for_execution(function, arguments, all_annotations)
@@ -265,7 +233,7 @@ def infer_return_types(function, arguments):
ann.define_generics(type_var_dict)
if isinstance(ann, (DefineGenericBaseClass, TypeVar)) else ValueSet({ann})
for ann in annotation_values
).execute_annotation(context)
).execute_annotation()
def infer_type_vars_for_execution(function, arguments, annotation_dict):
@@ -318,7 +286,7 @@ def infer_return_for_callable(arguments, param_values, result_values):
if isinstance(v, (DefineGenericBaseClass, TypeVar))
else ValueSet({v})
for v in result_values
).execute_annotation(arguments.context)
).execute_annotation()
def _infer_type_vars_for_callable(arguments, lazy_params):
@@ -394,7 +362,7 @@ def merge_pairwise_generics(annotation_value, annotated_argument_class):
for annotation_generics_set, actual_generic_set in zip(annotation_generics, actual_generics):
merge_type_var_dicts(
type_var_dict,
annotation_generics_set.infer_type_vars(actual_generic_set.execute_annotation(None)),
annotation_generics_set.infer_type_vars(actual_generic_set.execute_annotation()),
)
return type_var_dict
@@ -405,10 +373,6 @@ def find_type_from_comment_hint_for(context, node, name):
def find_type_from_comment_hint_with(context, node, name):
if len(node.children) > 4:
# In case there are multiple with_items, we do not want a type hint for
# now.
return []
assert len(node.children[1].children) == 3, \
"Can only be here when children[1] is 'foo() as f'"
varlist = node.children[1].children[2]
@@ -441,7 +405,7 @@ def _find_type_from_comment_hint(context, node, varlist, name):
return []
return _infer_annotation_string(
context, match.group(1).strip(), index
).execute_annotation(context)
).execute_annotation()
def find_unknown_type_vars(context, node):
+4 -4
View File
@@ -195,7 +195,7 @@ class GenericClass(DefineGenericBaseClass, ClassMixin):
@to_list
def py__bases__(self):
for base in self._wrapped_value.py__bases__(): # type: ignore[attr-defined]
for base in self._wrapped_value.py__bases__():
yield _LazyGenericBaseClass(self, base, self._generics_manager)
def _create_instance_with_generics(self, generics_manager):
@@ -306,7 +306,7 @@ class _GenericInstanceWrapper(ValueWrapper):
if cls.py__name__() == 'Generator':
generics = cls.get_generics()
try:
return generics[2].execute_annotation(None)
return generics[2].execute_annotation()
except IndexError:
pass
elif cls.py__name__() == 'Iterator':
@@ -384,7 +384,7 @@ class BaseTypingValue(LazyValueWrapper):
return _PseudoTreeNameClass(self.parent_context, self._tree_name)
def get_signatures(self):
return self._wrapped_value.get_signatures() # type: ignore[attr-defined]
return self._wrapped_value.get_signatures()
def __repr__(self):
return '%s(%s)' % (self.__class__.__name__, self._tree_name.value)
@@ -427,7 +427,7 @@ class BaseTypingInstance(LazyValueWrapper):
return ValueName(self, self._tree_name)
def _get_wrapped_value(self):
object_, = builtin_from_name(self.inference_state, 'object').execute_annotation(None)
object_, = builtin_from_name(self.inference_state, 'object').execute_annotation()
return object_
def __repr__(self):
+1 -10
View File
@@ -2,7 +2,6 @@
This module is about generics, like the `int` in `List[int]`. It's not about
the Generic class.
"""
from abc import abstractmethod
from jedi import debug
from jedi.cache import memoize_method
@@ -25,17 +24,9 @@ def _resolve_forward_references(context, value_set):
class _AbstractGenericManager:
@abstractmethod
def __getitem__(self, index):
raise NotImplementedError
@abstractmethod
def to_tuple(self):
raise NotImplementedError
def get_index_and_execute(self, index):
try:
return self[index].execute_annotation(None)
return self[index].execute_annotation()
except IndexError:
debug.warning('No param #%s found for annotation %s', index, self)
return NO_VALUES
-2
View File
@@ -86,8 +86,6 @@ class StubFilter(ParserTreeFilter):
# Imports in stub files are only public if they have an "as"
# export.
definition = name.get_definition()
if definition is None:
return False
if definition.type in ('import_from', 'import_name'):
if name.parent.type not in ('import_as_name', 'dotted_as_name'):
return False
+3 -3
View File
@@ -100,8 +100,8 @@ class TypeVar(BaseTypingValue):
return found
return ValueSet({self})
def execute_annotation(self, context):
return self._get_classes().execute_annotation(context)
def execute_annotation(self):
return self._get_classes().execute_annotation()
def infer_type_vars(self, value_set):
def iterate():
@@ -123,5 +123,5 @@ class TypeWrapper(ValueWrapper):
super().__init__(wrapped_value)
self._original_value = original_value
def execute_annotation(self, context):
def execute_annotation(self):
return ValueSet({self._original_value})
+18 -3
View File
@@ -1,4 +1,5 @@
import os
import re
from functools import wraps
from collections import namedtuple
from typing import Dict, Mapping, Tuple
@@ -6,6 +7,7 @@ from pathlib import Path
from jedi import settings
from jedi.file_io import FileIO
from jedi._compatibility import cast_path
from jedi.parser_utils import get_cached_code_lines
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.inference.gradual.stub_value import TypingModuleWrapper, StubModuleValue
@@ -42,6 +44,7 @@ def _create_stub_map(directory_path_info):
return
for entry in listed:
entry = cast_path(entry)
path = os.path.join(directory_path_info.path, entry)
if os.path.isdir(path):
init = os.path.join(path, '__init__.pyi')
@@ -57,8 +60,19 @@ def _create_stub_map(directory_path_info):
def _get_typeshed_directories(version_info):
yield PathInfo(str(TYPESHED_PATH.joinpath("stdlib")), False)
yield PathInfo(str(TYPESHED_PATH.joinpath("stubs")), True)
check_version_list = ['2and3', '3']
for base in ['stdlib', 'third_party']:
base_path = TYPESHED_PATH.joinpath(base)
base_list = os.listdir(base_path)
for base_list_entry in base_list:
match = re.match(r'(\d+)\.(\d+)$', base_list_entry)
if match is not None:
if match.group(1) == '3' and int(match.group(2)) <= version_info.minor:
check_version_list.append(base_list_entry)
for check_version in check_version_list:
is_third_party = base != 'stdlib'
yield PathInfo(str(base_path.joinpath(check_version)), is_third_party)
_version_cache: Dict[Tuple[int, int], Mapping[str, PathInfo]] = {}
@@ -155,6 +169,7 @@ def _try_to_load_stub(inference_state, import_names, python_value_set,
if len(import_names) == 1:
# foo-stubs
for p in sys_path:
p = cast_path(p)
init = os.path.join(p, *import_names) + '-stubs' + os.path.sep + '__init__.pyi'
m = _try_to_load_stub_from_file(
inference_state,
@@ -281,7 +296,7 @@ def parse_stub_module(inference_state, file_io):
def create_stub_module(inference_state, grammar, python_value_set,
stub_module_node, file_io, import_names):
if import_names in [('typing',), ('typing_extensions',)]:
if import_names == ('typing',):
module_cls = TypingModuleWrapper
else:
module_cls = StubModuleValue
+15 -37
View File
@@ -6,7 +6,6 @@ values.
This file deals with all the typing.py cases.
"""
import itertools
from typing import Any
from jedi import debug
from jedi.inference.compiled import builtin_from_name, create_simple_object
@@ -33,8 +32,7 @@ _TYPE_ALIAS_TYPES = {
'DefaultDict': 'collections.defaultdict',
'Deque': 'collections.deque',
}
_PROXY_TYPES = ['Optional', 'Union', 'ClassVar', 'Annotated', 'Final']
IGNORE_ANNOTATION_PARTS = ['ClassVar', 'Annotated', 'Final']
_PROXY_TYPES = 'Optional Union ClassVar'.split()
class TypingModuleName(NameWrapper):
@@ -83,9 +81,6 @@ class TypingModuleName(NameWrapper):
elif name == 'cast':
cast_fn, = self._wrapped_name.infer()
yield CastFunction.create_cached(inference_state, cast_fn)
elif name == 'Self':
yield SelfClass.create_cached(
inference_state, self.parent_context, self.tree_name)
elif name == 'TypedDict':
# TODO doesn't even exist in typeshed/typing.py, yet. But will be
# added soon.
@@ -103,24 +98,24 @@ class TypingModuleFilterWrapper(FilterWrapper):
class ProxyWithGenerics(BaseTypingClassWithGenerics):
def execute_annotation(self, context):
def execute_annotation(self):
string_name = self._tree_name.value
if string_name == 'Union':
# This is kind of a special case, because we have Unions (in Jedi
# ValueSets).
return self.gather_annotation_classes().execute_annotation(context)
return self.gather_annotation_classes().execute_annotation()
elif string_name == 'Optional':
# Optional is basically just saying it's either None or the actual
# type.
return self.gather_annotation_classes().execute_annotation(context) \
return self.gather_annotation_classes().execute_annotation() \
| ValueSet([builtin_from_name(self.inference_state, 'None')])
elif string_name == 'Type':
# The type is actually already given in the index_value
return self._generics_manager[0]
elif string_name in IGNORE_ANNOTATION_PARTS:
elif string_name == 'ClassVar':
# For now don't do anything here, ClassVars are always used.
return self._generics_manager[0].execute_annotation(context)
return self._generics_manager[0].execute_annotation()
mapped = {
'Tuple': Tuple,
@@ -191,8 +186,6 @@ class ProxyTypingValue(BaseTypingValue):
class _TypingClassMixin(ClassMixin):
_tree_name: Any
def py__bases__(self):
return [LazyKnownValues(
self.inference_state.builtins_module.py__getattribute__('object')
@@ -220,17 +213,17 @@ class TypingClassWithGenerics(ProxyWithGenerics, _TypingClassMixin):
# This is basically a trick to avoid extra code: We execute the
# incoming classes to be able to use the normal code for type
# var inference.
value_set.execute_annotation(None),
value_set.execute_annotation(),
)
elif annotation_name == 'Callable':
if len(annotation_generics) == 2:
return annotation_generics[1].infer_type_vars(
value_set.execute_annotation(None),
value_set.execute_annotation(),
)
elif annotation_name == 'Tuple':
tuple_annotation, = self.execute_annotation(None)
tuple_annotation, = self.execute_annotation()
return tuple_annotation.infer_type_vars(value_set)
return type_var_dict
@@ -301,9 +294,6 @@ class Callable(BaseTypingInstance):
from jedi.inference.gradual.annotation import infer_return_for_callable
return infer_return_for_callable(arguments, param_values, result_values)
def py__get__(self, instance, class_value):
return ValueSet([self])
class Tuple(BaseTypingInstance):
def _is_homogenous(self):
@@ -326,7 +316,7 @@ class Tuple(BaseTypingInstance):
yield LazyKnownValues(self._generics_manager.get_index_and_execute(0))
else:
for v in self._generics_manager.to_tuple():
yield LazyKnownValues(v.execute_annotation(None))
yield LazyKnownValues(v.execute_annotation())
def py__getitem__(self, index_value_set, contextualized_node):
if self._is_homogenous():
@@ -334,11 +324,11 @@ class Tuple(BaseTypingInstance):
return ValueSet.from_sets(
self._generics_manager.to_tuple()
).execute_annotation(None)
).execute_annotation()
def _get_wrapped_value(self):
tuple_, = self.inference_state.builtins_module \
.py__getattribute__('tuple').execute_annotation(None)
.py__getattribute__('tuple').execute_annotation()
return tuple_
@property
@@ -395,20 +385,11 @@ class Protocol(BaseTypingInstance):
class AnyClass(BaseTypingValue):
def execute_annotation(self, context):
def execute_annotation(self):
debug.warning('Used Any - returned no results')
return NO_VALUES
class SelfClass(BaseTypingValue):
def execute_annotation(self, context):
debug.warning('Used Self')
if context is not None:
# Execute the class of Self
return context.get_value().execute_annotation(None)
return NO_VALUES
class OverloadFunction(BaseTypingValue):
@repack_with_argument_clinic('func, /')
def py__call__(self, func_value_set):
@@ -443,21 +424,18 @@ class NewType(Value):
return c
def py__call__(self, arguments):
return self._type_value_set.execute_annotation(arguments.context)
return self._type_value_set.execute_annotation()
@property
def name(self):
from jedi.inference.compiled.value import CompiledValueName
return CompiledValueName(self, 'NewType')
def __repr__(self) -> str:
return '<NewType: %s>%s' % (self.tree_node, self._type_value_set)
class CastFunction(ValueWrapper):
@repack_with_argument_clinic('type, object, /')
def py__call__(self, type_value_set, object_value_set):
return type_value_set.execute_annotation(None)
return type_value_set.execute_annotation()
class TypedDictClass(BaseTypingValue):
+1 -1
View File
@@ -19,7 +19,7 @@ def load_proper_stub_module(inference_state, grammar, file_io, import_names, mod
# /[...]/stdlib/3/os/__init__.pyi -> stdlib/3/os/__init__
rest = relative_path.with_suffix('')
# Remove the stdlib/3 or third_party/3.6 part
import_names = rest.parts[1:]
import_names = rest.parts[2:]
if rest.name == '__init__':
import_names = import_names[:-1]
+2 -2
View File
@@ -5,12 +5,12 @@ import os
from itertools import chain
from contextlib import contextmanager
from parso import tree
from parso.python import tree
def is_stdlib_path(path):
# Python standard library paths look like this:
# /usr/lib/python3.14/...
# /usr/lib/python3.9/...
# TODO The implementation below is probably incorrect and not complete.
parts = path.parts
if 'dist-packages' in parts or 'site-packages' in parts:
+23 -15
View File
@@ -12,6 +12,7 @@ import os
from pathlib import Path
from parso.python import tree
from parso.tree import search_ancestor
from jedi import debug
from jedi import settings
@@ -94,7 +95,7 @@ def goto_import(context, tree_name):
def _prepare_infer_import(module_context, tree_name):
import_node = tree_name.search_ancestor('import_name', 'import_from')
import_node = search_ancestor(tree_name, 'import_name', 'import_from')
import_path = import_node.get_path_for_name(tree_name)
from_import_name = None
try:
@@ -370,16 +371,16 @@ def import_module_by_names(inference_state, import_names, sys_path=None,
i.value if isinstance(i, tree.Name) else i
for i in import_names
)
base = [None]
value_set = [None]
for i, name in enumerate(import_names):
base = value_set = ValueSet.from_sets([
value_set = ValueSet.from_sets([
import_module(
inference_state,
str_import_names[:i+1],
parent_module_value,
sys_path,
prefer_stubs=prefer_stubs, # type: ignore[call-arg]
) for parent_module_value in base
prefer_stubs=prefer_stubs,
) for parent_module_value in value_set
])
if not value_set:
message = 'No module named ' + '.'.join(str_import_names)
@@ -421,13 +422,20 @@ def import_module(inference_state, import_names, parent_module_value, sys_path):
# The module might not be a package.
return NO_VALUES
file_io_or_ns, is_pkg = inference_state.compiled_subprocess.get_module_info(
string=import_names[-1],
path=paths,
full_name=module_name,
is_global_search=False,
)
if is_pkg is None:
for path in paths:
# At the moment we are only using one path. So this is
# not important to be correct.
if not isinstance(path, list):
path = [path]
file_io_or_ns, is_pkg = inference_state.compiled_subprocess.get_module_info(
string=import_names[-1],
path=path,
full_name=module_name,
is_global_search=False,
)
if is_pkg is not None:
break
else:
return NO_VALUES
if isinstance(file_io_or_ns, ImplicitNSInfo):
@@ -474,12 +482,12 @@ def _load_python_module(inference_state, file_io,
)
def _load_builtin_module(inference_state, import_names, sys_path):
def _load_builtin_module(inference_state, import_names=None, sys_path=None):
project = inference_state.project
if sys_path is None:
sys_path = inference_state.get_sys_path()
if not project._load_unsafe_extensions:
safe_paths = set(project._get_base_sys_path(inference_state))
safe_paths = project._get_base_sys_path(inference_state)
sys_path = [p for p in sys_path if p in safe_paths]
dotted_name = '.'.join(import_names)
@@ -548,7 +556,7 @@ def load_namespace_from_path(inference_state, folder_io):
def follow_error_node_imports_if_possible(context, name):
error_node = name.search_ancestor('error_node')
error_node = tree.search_ancestor(name, 'error_node')
if error_node is not None:
# Get the first command start of a started simple_stmt. The error
# node is sometimes a small_stmt and sometimes a simple_stmt. Check
+20 -36
View File
@@ -1,8 +1,11 @@
from abc import abstractmethod
from inspect import Parameter
from typing import Optional, Tuple, Any
from typing import Optional, Tuple
from parso.tree import search_ancestor
from jedi.parser_utils import find_statement_documentation, clean_scope_docstring
from jedi.inference.utils import unite
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.inference.cache import inference_state_method_cache
from jedi.inference import docstrings
@@ -36,6 +39,7 @@ class AbstractNameDefinition:
def infer(self):
raise NotImplementedError
@abstractmethod
def goto(self):
# Typically names are already definitions and therefore a goto on that
# name will always result on itself.
@@ -103,15 +107,12 @@ class AbstractArbitraryName(AbstractNameDefinition):
class AbstractTreeName(AbstractNameDefinition):
tree_name: Any
parent_context: Any
def __init__(self, parent_context, tree_name):
self.parent_context = parent_context
self.tree_name = tree_name
def get_qualified_names(self, include_module_names=False):
import_node = self.tree_name.search_ancestor('import_name', 'import_from')
import_node = search_ancestor(self.tree_name, 'import_name', 'import_from')
# For import nodes we cannot just have names, because it's very unclear
# how they would look like. For now we just ignore them in most cases.
# In case of level == 1, it works always, because it's like a submodule
@@ -195,23 +196,24 @@ class AbstractTreeName(AbstractNameDefinition):
new_dotted = deep_ast_copy(par)
new_dotted.children[index - 1:] = []
values = context.infer_node(new_dotted)
return [
n
return unite(
value.goto(name, name_context=context)
for value in values
for n in value.goto(name, name_context=context)
]
)
if node_type == 'trailer' and par.children[0] == '.':
values = infer_call_of_leaf(context, name, cut_own_trailer=True)
return values.goto(name, name_context=context)
else:
stmt = name.search_ancestor('expr_stmt', 'lambdef') or name
stmt = search_ancestor(
name, 'expr_stmt', 'lambdef'
) or name
if stmt.type == 'lambdef':
stmt = name
return context.goto(name, position=stmt.start_pos)
def is_import(self):
imp = self.tree_name.search_ancestor('import_from', 'import_name')
imp = search_ancestor(self.tree_name, 'import_from', 'import_name')
return imp is not None
@property
@@ -224,9 +226,6 @@ class AbstractTreeName(AbstractNameDefinition):
class ValueNameMixin:
_value: Any
parent_context: Any
def infer(self):
return ValueSet([self._value])
@@ -245,11 +244,11 @@ class ValueNameMixin:
def get_root_context(self):
if self.parent_context is None: # A module
return self._value.as_context()
return super().get_root_context() # type: ignore
return super().get_root_context()
def get_defining_qualified_value(self):
context = self.parent_context
if context is not None and (context.is_module() or context.is_class()):
if context.is_module() or context.is_class():
return self.parent_context.get_value() # Might be None
return None
@@ -342,12 +341,6 @@ class TreeNameDefinition(AbstractTreeName):
def py__doc__(self):
api_type = self.api_type
if api_type in ('function', 'class', 'property'):
if self.parent_context.get_root_context().is_stub():
from jedi.inference.gradual.conversion import convert_names
names = convert_names([self], prefer_stub_to_compiled=False)
if self not in names:
return _merge_name_docs(names)
# Make sure the names are not TreeNameDefinitions anymore.
return clean_scope_docstring(self.tree_name.get_definition())
@@ -362,16 +355,14 @@ class TreeNameDefinition(AbstractTreeName):
class _ParamMixin:
get_kind: Any
def maybe_positional_argument(self, include_star=True):
options: list[int] = [Parameter.POSITIONAL_ONLY, Parameter.POSITIONAL_OR_KEYWORD]
options = [Parameter.POSITIONAL_ONLY, Parameter.POSITIONAL_OR_KEYWORD]
if include_star:
options.append(Parameter.VAR_POSITIONAL)
return self.get_kind() in options
def maybe_keyword_argument(self, include_stars=True):
options: list[int] = [Parameter.KEYWORD_ONLY, Parameter.POSITIONAL_OR_KEYWORD]
options = [Parameter.KEYWORD_ONLY, Parameter.POSITIONAL_OR_KEYWORD]
if include_stars:
options.append(Parameter.VAR_KEYWORD)
return self.get_kind() in options
@@ -417,9 +408,6 @@ class ParamNameInterface(_ParamMixin):
return 2
return 0
def infer_default(self):
return NO_VALUES
class BaseTreeParamName(ParamNameInterface, AbstractTreeName):
annotation_node = None
@@ -454,7 +442,7 @@ class _ActualTreeParamName(BaseTreeParamName):
self.function_value = function_value
def _get_param_node(self):
return self.tree_name.search_ancestor('param')
return search_ancestor(self.tree_name, 'param')
@property
def annotation_node(self):
@@ -466,7 +454,7 @@ class _ActualTreeParamName(BaseTreeParamName):
self.function_value, self._get_param_node(),
ignore_stars=ignore_stars)
if execute_annotation:
values = values.execute_annotation(self.function_value.get_default_param_context())
values = values.execute_annotation()
return values
def infer_default(self):
@@ -636,10 +624,6 @@ class NameWrapper:
class StubNameMixin:
api_type: str
tree_name: Any
infer: Any
def py__doc__(self):
from jedi.inference.gradual.conversion import convert_names
# Stubs are not complicated and we can just follow simple statements
@@ -651,7 +635,7 @@ class StubNameMixin:
names = convert_names(names, prefer_stub_to_compiled=False)
if self in names:
return super().py__doc__() # type: ignore
return super().py__doc__()
else:
# We have signatures ourselves in stubs, so don't use signatures
# from the implementation.
+1 -1
View File
@@ -12,7 +12,7 @@ count the function calls.
Settings
~~~~~~~~~~
Recursion settings are important if you don't want extremely
Recursion settings are important if you don't want extremly
recursive python code to go absolutely crazy.
The default values are based on experiments while completing the |jedi| library
+17 -33
View File
@@ -180,34 +180,26 @@ def _check_fs(inference_state, file_io, regex):
return m.as_context()
def gitignored_paths(folder_io, file_io):
ignored_paths_abs = set()
ignored_paths_rel = set()
def gitignored_lines(folder_io, file_io):
ignored_paths = set()
ignored_names = set()
for l in file_io.read().splitlines():
if not l or l.startswith(b'#') or l.startswith(b'!') or b'*' in l:
if not l or l.startswith(b'#'):
continue
p = l.decode('utf-8', 'ignore').rstrip('/')
if '/' in p:
name = p.lstrip('/')
ignored_paths_abs.add(os.path.join(folder_io.path, name))
p = l.decode('utf-8', 'ignore')
if p.startswith('/'):
name = p[1:]
if name.endswith(os.path.sep):
name = name[:-1]
ignored_paths.add(os.path.join(folder_io.path, name))
else:
name = p
ignored_paths_rel.add((folder_io.path, name))
return ignored_paths_abs, ignored_paths_rel
def expand_relative_ignore_paths(folder_io, relative_paths):
curr_path = folder_io.path
return {os.path.join(curr_path, p[1]) for p in relative_paths if curr_path.startswith(p[0])}
ignored_names.add(p)
return ignored_paths, ignored_names
def recurse_find_python_folders_and_files(folder_io, except_paths=()):
except_paths = set(except_paths)
except_paths_relative = set()
for root_folder_io, folder_ios, file_ios in folder_io.walk():
# Delete folders that we don't want to iterate over.
for file_io in file_ios:
@@ -217,21 +209,14 @@ def recurse_find_python_folders_and_files(folder_io, except_paths=()):
yield None, file_io
if path.name == '.gitignore':
ignored_paths_abs, ignored_paths_rel = gitignored_paths(
root_folder_io, file_io
)
except_paths |= ignored_paths_abs
except_paths_relative |= ignored_paths_rel
except_paths_relative_expanded = expand_relative_ignore_paths(
root_folder_io, except_paths_relative
)
ignored_paths, ignored_names = \
gitignored_lines(root_folder_io, file_io)
except_paths |= ignored_paths
folder_ios[:] = [
folder_io
for folder_io in folder_ios
if folder_io.path not in except_paths
and folder_io.path not in except_paths_relative_expanded
and folder_io.get_base_name() not in _IGNORE_FOLDERS
]
for folder_io in folder_ios:
@@ -297,13 +282,12 @@ def get_module_contexts_containing_name(inference_state, module_contexts, name,
limit_reduction=limit_reduction)
def search_in_file_ios(inference_state, file_io_iterator, name,
limit_reduction=1, complete=False):
def search_in_file_ios(inference_state, file_io_iterator, name, limit_reduction=1):
parse_limit = _PARSED_FILE_LIMIT / limit_reduction
open_limit = _OPENED_FILE_LIMIT / limit_reduction
file_io_count = 0
parsed_file_count = 0
regex = re.compile(r'\b' + re.escape(name) + (r'' if complete else r'\b'))
regex = re.compile(r'\b' + re.escape(name) + r'\b')
for file_io in file_io_iterator:
file_io_count += 1
m = _check_fs(inference_state, file_io, regex)
-7
View File
@@ -1,5 +1,4 @@
from inspect import Parameter
from typing import Any
from jedi.cache import memoize_method
from jedi import debug
@@ -7,10 +6,6 @@ from jedi import parser_utils
class _SignatureMixin:
get_param_names: Any
name: Any
annotation_string: Any
def to_string(self):
def param_strings():
is_positional = False
@@ -41,8 +36,6 @@ class _SignatureMixin:
class AbstractSignature(_SignatureMixin):
_function_value: Any
def __init__(self, value, is_bound=False):
self.value = value
self.is_bound = is_bound
+3 -6
View File
@@ -18,14 +18,11 @@ from jedi.inference.helpers import is_big_annoying_library
def _iter_nodes_for_param(param_name):
from parso.python.tree import search_ancestor
from jedi.inference.arguments import TreeArguments
execution_context = param_name.parent_context
# Walk up the parso tree to get the FunctionNode we want. We use the parso
# tree rather than going via the execution context so that we're agnostic of
# the specific scope we're evaluating within (i.e: module or function,
# etc.).
function_node = param_name.tree_name.search_ancestor('funcdef', 'lambdef')
function_node = execution_context.tree_node
module_node = function_node.get_root_node()
start = function_node.children[-1].start_pos
end = function_node.children[-1].end_pos
@@ -35,7 +32,7 @@ def _iter_nodes_for_param(param_name):
argument = name.parent
if argument.type == 'argument' \
and argument.children[0] == '*' * param_name.star_count:
trailer = argument.search_ancestor('trailer')
trailer = search_ancestor(argument, 'trailer')
if trailer is not None: # Make sure we're in a function
context = execution_context.create_context(trailer)
if _goes_to_param_name(param_name, context, name):
+15 -65
View File
@@ -2,7 +2,6 @@
Functions inferring the syntax tree.
"""
import copy
import itertools
from parso.python import tree
@@ -30,8 +29,6 @@ from jedi.inference.names import TreeNameDefinition
from jedi.inference.context import CompForContext
from jedi.inference.value.decorator import Decoratee
from jedi.plugins import plugin_manager
from jedi.inference.gradual.typing import ProxyTypingValue, IGNORE_ANNOTATION_PARTS
from jedi.inference.gradual.type_var import TypeVar
operator_to_magic_method = {
'+': '__add__',
@@ -91,7 +88,6 @@ def infer_node(context, element):
if isinstance(context, CompForContext):
return _infer_node(context, element)
name_dicts = [{}]
if_stmt = element
while if_stmt is not None:
if_stmt = if_stmt.parent
@@ -107,6 +103,7 @@ def infer_node(context, element):
if predefined_if_name_dict is None and if_stmt \
and if_stmt.type == 'if_stmt' and context.inference_state.is_analysis:
if_stmt_test = if_stmt.children[1]
name_dicts = [{}]
# If we already did a check, we don't want to do it again -> If
# value.predefined_names is filled, we stop.
# We don't want to check the if stmt itself, it's just about
@@ -240,7 +237,7 @@ def _infer_node(context, element):
return context.infer_node(element.children[0])
elif typ == 'annassign':
return annotation.infer_annotation(context, element.children[1]) \
.execute_annotation(context)
.execute_annotation()
elif typ == 'yield_expr':
if len(element.children) and element.children[1].type == 'yield_arg':
# Implies that it's a yield from.
@@ -253,8 +250,6 @@ def _infer_node(context, element):
return NO_VALUES
elif typ == 'namedexpr_test':
return context.infer_node(element.children[2])
elif typ == 'star_expr':
return NO_VALUES
else:
return infer_or_test(context, element)
@@ -292,7 +287,7 @@ def infer_atom(context, atom):
state = context.inference_state
if atom.type == 'name':
# This is the first global lookup.
stmt = atom.search_ancestor('expr_stmt', 'lambdef', 'if_stmt') or atom
stmt = tree.search_ancestor(atom, 'expr_stmt', 'lambdef', 'if_stmt') or atom
if stmt.type == 'if_stmt':
if not any(n.start_pos <= atom.start_pos < n.end_pos for n in stmt.get_test_nodes()):
stmt = atom
@@ -333,8 +328,8 @@ def infer_atom(context, atom):
c = atom.children
# Parentheses without commas are not tuples.
if c[0] == '(' and not len(c) == 2 \
and not (c[1].type == 'testlist_comp'
and len(c[1].children) > 1):
and not(c[1].type == 'testlist_comp'
and len(c[1].children) > 1):
return context.infer_node(c[1])
try:
@@ -438,7 +433,7 @@ def _infer_expr_stmt(context, stmt, seek_name=None):
else:
operator = copy.copy(first_operator)
operator.value = operator.value[:-1]
for_stmt = stmt.search_ancestor('for_stmt')
for_stmt = tree.search_ancestor(stmt, 'for_stmt')
if for_stmt is not None and for_stmt.type == 'for_stmt' and value_set \
and parser_utils.for_stmt_defines_one_name(for_stmt):
# Iterate through result and add the values, that's possible
@@ -497,10 +492,8 @@ def infer_factor(value_set, operator):
elif operator == 'not':
b = value.py__bool__()
if b is None: # Uncertainty.
yield list(value.inference_state.builtins_module.py__getattribute__('bool')
.execute_annotation(None)).pop()
else:
yield compiled.create_simple_object(value.inference_state, not b)
return
yield compiled.create_simple_object(value.inference_state, not b)
else:
yield value
@@ -522,20 +515,10 @@ def _literals_to_types(inference_state, result):
def _infer_comparison(context, left_values, operator, right_values):
state = context.inference_state
if isinstance(operator, str):
operator_str = operator
else:
operator_str = str(operator.value)
if not left_values or not right_values:
# illegal slices e.g. cause left/right_result to be None
result = (left_values or NO_VALUES) | (right_values or NO_VALUES)
return _literals_to_types(state, result)
elif operator_str == "|" and all(
value.is_class() or value.is_compiled() or isinstance(value, TypeVar)
for value in itertools.chain(left_values, right_values)
):
# ^^^ A naive hack for PEP 604
return ValueSet.from_sets((left_values, right_values))
else:
# I don't think there's a reasonable chance that a string
# operation is still correct, once we pass something like six
@@ -551,7 +534,7 @@ def _infer_comparison(context, left_values, operator, right_values):
def _is_annotation_name(name):
ancestor = name.search_ancestor('param', 'funcdef', 'expr_stmt')
ancestor = tree.search_ancestor(name, 'param', 'funcdef', 'expr_stmt')
if ancestor is None:
return False
@@ -651,9 +634,7 @@ def _infer_comparison_part(inference_state, context, left, operator, right):
_bool_to_value(inference_state, False)
])
elif str_operator in ('in', 'not in'):
return inference_state.builtins_module.py__getattribute__('bool').execute_annotation(
context
)
return NO_VALUES
def check(obj):
"""Checks if a Jedi object is either a float or an int."""
@@ -703,29 +684,11 @@ def tree_name_to_values(inference_state, context, tree_name):
if expr_stmt.type == "expr_stmt" and expr_stmt.children[1].type == "annassign":
correct_scope = parser_utils.get_parent_scope(name) == context.tree_node
ann_assign = expr_stmt.children[1]
first = ann_assign.children[1]
code = first.get_code()
if correct_scope and not (code.endswith(".TypeAlias")
or code.strip() == "TypeAlias"):
if (
(first.type == 'name')
and (ann_assign.children[1].value == tree_name.value)
and context.parent_context
):
context = context.parent_context
found = annotation.infer_annotation(
if correct_scope:
found_annotation = True
value_set |= annotation.infer_annotation(
context, expr_stmt.children[1].children[1]
)
set_found_annotation = True
if len(found) == 1:
first = next(iter(found))
set_found_annotation = not (
isinstance(first, ProxyTypingValue)
and first.name.string_name in IGNORE_ANNOTATION_PARTS
)
found_annotation = set_found_annotation
value_set |= found.execute_annotation(context)
).execute_annotation()
if found_annotation:
return value_set
@@ -775,15 +738,8 @@ def tree_name_to_values(inference_state, context, tree_name):
types = infer_expr_stmt(context, node, tree_name)
elif typ == 'with_stmt':
value_managers = context.infer_node(node.get_test_node_from_name(tree_name))
if node.parent.type == 'async_stmt':
# In the case of `async with` statements, we need to
# first get the coroutine from the `__aenter__` method,
# then "unwrap" via the `__await__` method
enter_methods = value_managers.py__getattribute__('__aenter__')
coro = enter_methods.execute_with_values()
return coro.py__await__().py__stop_iteration_returns()
enter_methods = value_managers.py__getattribute__('__enter__')
return enter_methods.execute_annotation(context)
return enter_methods.execute_with_values()
elif typ in ('import_from', 'import_name'):
types = imports.infer_import(context, tree_name)
elif typ in ('funcdef', 'classdef'):
@@ -869,16 +825,10 @@ def check_tuple_assignments(name, value_set):
# For no star unpacking is not possible.
return NO_VALUES
i = 0
lazy_value = None
while i <= index:
try:
lazy_value = next(iterated)
except StopIteration:
# A desperate attempt to fix inference for tuples from an
# iterator.
if lazy_value is not None:
return lazy_value.infer()
# We could do this with the default param in next. But this
# would allow this loop to run for a very long time if the
# index number is high. Therefore break if the loop is
+2 -1
View File
@@ -186,6 +186,7 @@ def _get_buildout_script_paths(search_path: Path):
directory that look like python files.
:param search_path: absolute path to the module.
:type search_path: str
"""
project_root = _get_parent_dir_with_file(search_path, 'buildout.cfg')
if not project_root:
@@ -204,7 +205,7 @@ def _get_buildout_script_paths(search_path: Path):
except (UnicodeDecodeError, IOError) as e:
# Probably a binary file; permission error or race cond. because
# file got deleted. Ignore it.
debug.warning(str(e))
debug.warning(e)
continue
+1
View File
@@ -74,6 +74,7 @@ class PushBackIterator:
def __init__(self, iterator):
self.pushes = []
self.iterator = iterator
self.current = None
def push_back(self, value):
self.pushes.append(value)
+1 -1
View File
@@ -16,7 +16,7 @@ settings will stop this process.
It is important to note that:
1. Array modifications work only in the current module.
1. Array modfications work only in the current module.
2. Jedi only checks Array additions; ``list.pop``, etc are ignored.
"""
from jedi import debug
+8 -13
View File
@@ -1,4 +1,4 @@
from typing import Any
from parso.python import tree
from jedi import debug
from jedi.inference.cache import inference_state_method_cache, CachedMetaClass
@@ -55,10 +55,6 @@ class FunctionAndClassBase(TreeValue):
class FunctionMixin:
api_type = 'function'
tree_node: Any
py__class__: Any
as_context: Any
get_signature_functions: Any
def get_filters(self, origin_scope=None):
cls = self.py__class__()
@@ -266,8 +262,8 @@ class BaseFunctionExecutionContext(ValueContext, TreeContextMixin):
@recursion.execution_recursion_decorator(default=iter([]))
def get_yield_lazy_values(self, is_async=False):
# TODO: if is_async, wrap yield statements in Awaitable/async_generator_asend
for_parents = [(y, y.search_ancestor('for_stmt', 'funcdef',
'while_stmt', 'if_stmt'))
for_parents = [(y, tree.search_ancestor(y, 'for_stmt', 'funcdef',
'while_stmt', 'if_stmt'))
for y in get_yield_exprs(self.inference_state, self.tree_node)]
# Calculate if the yields are placed within the same for loop.
@@ -338,18 +334,17 @@ class BaseFunctionExecutionContext(ValueContext, TreeContextMixin):
return ValueSet(
GenericClass(c, TupleGenericManager(generics))
for c in async_generator_classes
).execute_annotation(None)
).execute_annotation()
else:
async_classes = inference_state.types_module.py__getattribute__('CoroutineType')
async_classes = inference_state.typing_module.py__getattribute__('Coroutine')
return_values = self.get_return_values()
# Only the first generic is relevant.
generics = (NO_VALUES, NO_VALUES, return_values.py__class__())
generics = (return_values.py__class__(), NO_VALUES, NO_VALUES)
return ValueSet(
GenericClass(c, TupleGenericManager(generics)) for c in async_classes
).execute_annotation(None)
).execute_annotation()
else:
# If there are annotations, prefer them over anything else.
if self.is_generator() and not self.infer_annotations():
if self.is_generator():
return ValueSet([iterable.Generator(inference_state, self)])
else:
return self.get_return_values()
+9 -12
View File
@@ -1,5 +1,6 @@
from abc import abstractproperty
from typing import Any
from parso.tree import search_ancestor
from jedi import debug
from jedi import settings
@@ -17,7 +18,7 @@ from jedi.inference.arguments import ValuesArguments, TreeArgumentsWrapper
from jedi.inference.value.function import \
FunctionValue, FunctionMixin, OverloadedFunctionValue, \
BaseFunctionExecutionContext, FunctionExecutionContext, FunctionNameInClass
from jedi.inference.value.klass import ClassFilter, init_or_new_func
from jedi.inference.value.klass import ClassFilter
from jedi.inference.value.dynamic_arrays import get_dynamic_array_instance
from jedi.parser_utils import function_is_staticmethod, function_is_classmethod
@@ -155,9 +156,8 @@ class AbstractInstanceValue(Value):
return super().py__iter__(contextualized_node)
def iterate():
yield LazyKnownValues(
self.execute_function_slots(iter_slot_names).py__next__(contextualized_node).infer()
)
for generator in self.execute_function_slots(iter_slot_names):
yield from generator.py__next__(contextualized_node)
return iterate()
def __repr__(self):
@@ -189,9 +189,6 @@ class CompiledInstance(AbstractInstanceValue):
class _BaseTreeInstance(AbstractInstanceValue):
get_defined_names: Any
_arguments: Any
@property
def array_type(self):
name = self.class_value.py__name__()
@@ -232,7 +229,7 @@ class _BaseTreeInstance(AbstractInstanceValue):
new = node
while True:
func_node = new
new = new.search_ancestor('funcdef', 'classdef')
new = search_ancestor(new, 'funcdef', 'classdef')
if class_context.tree_node is new:
func = FunctionValue.from_context(class_context, func_node)
bound_method = BoundMethod(self, class_context, func)
@@ -327,7 +324,7 @@ class TreeInstance(_BaseTreeInstance):
infer_type_vars_for_execution
args = InstanceArguments(self, self._arguments)
for signature in init_or_new_func(self.class_value).get_signatures():
for signature in self.class_value.py__getattribute__('__init__').get_signatures():
# Just take the first result, it should always be one, because we
# control the typeshed code.
funcdef = signature.value.tree_node
@@ -501,13 +498,13 @@ class SelfName(TreeNameDefinition):
return self._instance
def infer(self):
stmt = self.tree_name.search_ancestor('expr_stmt')
stmt = search_ancestor(self.tree_name, 'expr_stmt')
if stmt is not None:
if stmt.children[1].type == "annassign":
from jedi.inference.gradual.annotation import infer_annotation
values = infer_annotation(
self.parent_context, stmt.children[1].children[1]
).execute_annotation(None)
).execute_annotation()
if values:
return values
return super().infer()
+14 -32
View File
@@ -2,8 +2,6 @@
Contains all classes and functions to deal with lists, dicts, generators and
iterators in general.
"""
from typing import Any
from jedi.inference import compiled
from jedi.inference import analysis
from jedi.inference.lazy_value import LazyKnownValue, LazyKnownValues, \
@@ -22,9 +20,6 @@ from jedi.inference.value.dynamic_arrays import check_array_additions
class IterableMixin:
py__iter__: Any
inference_state: Any
def py__next__(self, contextualized_node=None):
return self.py__iter__(contextualized_node)
@@ -44,11 +39,11 @@ class GeneratorBase(LazyAttributeOverwrite, IterableMixin):
array_type = None
def _get_wrapped_value(self):
instance, = self._get_cls().execute_annotation(None)
instance, = self._get_cls().execute_annotation()
return instance
def _get_cls(self):
generator, = self.inference_state.types_module.py__getattribute__('GeneratorType')
generator, = self.inference_state.typing_module.py__getattribute__('Generator')
return generator
def py__bool__(self):
@@ -133,12 +128,6 @@ def comprehension_from_atom(inference_state, value, atom):
class ComprehensionMixin:
_defining_context: Any
_entry_node: Any
array_type: Any
_value_node: Any
_sync_comp_for_node: Any
@inference_state_method_cache()
def _get_comp_for_context(self, parent_context, comp_for):
return CompForContext(parent_context, comp_for)
@@ -187,8 +176,6 @@ class ComprehensionMixin:
class _DictMixin:
get_mapping_item_values: Any
def _get_generics(self):
return tuple(c_set.py__class__() for c_set in self.get_mapping_item_values())
@@ -214,7 +201,7 @@ class Sequence(LazyAttributeOverwrite, IterableMixin):
c, = GenericClass(
klass,
TupleGenericManager(self._cached_generics())
).execute_annotation(None)
).execute_annotation()
return c
def py__bool__(self):
@@ -233,7 +220,7 @@ class Sequence(LazyAttributeOverwrite, IterableMixin):
class _BaseComprehension(ComprehensionMixin):
def __init__(self, inference_state, defining_context, sync_comp_for_node, entry_node):
assert sync_comp_for_node.type == 'sync_comp_for'
super().__init__(inference_state) # type: ignore[call-arg]
super().__init__(inference_state)
self._defining_context = defining_context
self._sync_comp_for_node = sync_comp_for_node
self._entry_node = entry_node
@@ -261,9 +248,6 @@ class GeneratorComprehension(_BaseComprehension, GeneratorBase):
class _DictKeyMixin:
_dict_keys: Any
_dict_values: Any
# TODO merge with _DictMixin?
def get_mapping_item_values(self):
return self._dict_keys(), self._dict_values()
@@ -358,8 +342,6 @@ class SequenceLiteralValue(Sequence):
else:
with reraise_getitem_errors(TypeError, KeyError, IndexError):
node = self.get_tree_entries()[index]
if node == ':' or node.type == 'subscript':
return NO_VALUES
return self._defining_context.infer_node(node)
def py__iter__(self, contextualized_node=None):
@@ -425,6 +407,16 @@ class SequenceLiteralValue(Sequence):
else:
return [array_node]
def exact_key_items(self):
"""
Returns a generator of tuples like dict.items(), where the key is
resolved (as a string) and the values are still lazy values.
"""
for key_node, value in self.get_tree_entries():
for key in self._defining_context.infer_node(key_node):
if is_string(key):
yield key.get_safe_value(), LazyTreeValue(self._defining_context, value)
def __repr__(self):
return "<%s of %s>" % (self.__class__.__name__, self.atom)
@@ -480,16 +472,6 @@ class DictLiteralValue(_DictMixin, SequenceLiteralValue, _DictKeyMixin):
return ValueSet([FakeList(self.inference_state, lazy_values)])
def exact_key_items(self):
"""
Returns a generator of tuples like dict.items(), where the key is
resolved (as a string) and the values are still lazy values.
"""
for key_node, value in self.get_tree_entries():
for key in self._defining_context.infer_node(key_node):
if is_string(key):
yield key.get_safe_value(), LazyTreeValue(self._defining_context, value)
def _dict_values(self):
return ValueSet.from_sets(
self._defining_context.infer_node(v)
+21 -344
View File
@@ -36,10 +36,6 @@ py__doc__() Returns the docstring for a value.
====================================== ========================================
"""
from __future__ import annotations
from typing import List, Optional, Tuple, TYPE_CHECKING, Any
from jedi import debug
from jedi.parser_utils import get_cached_parent_scope, expr_is_dotted, \
function_is_property
@@ -51,18 +47,11 @@ from jedi.inference.filters import ParserTreeFilter
from jedi.inference.names import TreeNameDefinition, ValueName
from jedi.inference.arguments import unpack_arglist, ValuesArguments
from jedi.inference.base_value import ValueSet, iterator_to_value_set, \
NO_VALUES, ValueWrapper
NO_VALUES
from jedi.inference.context import ClassContext
from jedi.inference.value.function import FunctionAndClassBase, FunctionMixin
from jedi.inference.value.decorator import Decoratee
from jedi.inference.value.function import FunctionAndClassBase
from jedi.inference.gradual.generics import LazyGenericManager, TupleGenericManager
from jedi.plugins import plugin_manager
from inspect import Parameter
from jedi.inference.names import BaseTreeParamName
from jedi.inference.signature import AbstractSignature
if TYPE_CHECKING:
from jedi.inference import InferenceState
class ClassName(TreeNameDefinition):
@@ -89,8 +78,6 @@ class ClassName(TreeNameDefinition):
type_ = super().api_type
if type_ == 'function':
definition = self.tree_name.get_definition()
if definition is None:
return type_
if function_is_property(definition):
# This essentially checks if there is an @property before
# the function. @property could be something different, but
@@ -127,10 +114,25 @@ class ClassFilter(ParserTreeFilter):
while node is not None:
if node == self._parser_scope or node == self.parent_context:
return True
node = get_cached_parent_scope(self._parso_cache_node, node)
node = get_cached_parent_scope(self._used_names, node)
return False
def _access_possible(self, name):
# Filter for ClassVar variables
# TODO this is not properly done, yet. It just checks for the string
# ClassVar in the annotation, which can be quite imprecise. If we
# wanted to do this correct, we would have to infer the ClassVar.
if not self._is_instance:
expr_stmt = name.get_definition()
if expr_stmt is not None and expr_stmt.type == 'expr_stmt':
annassign = expr_stmt.children[1]
if annassign.type == 'annassign':
# If there is an =, the variable is obviously also
# defined on the class.
if 'ClassVar' not in annassign.children[1].get_code() \
and '=' not in annassign.children:
return False
# Filter for name mangling of private variables like __foo
return not name.value.startswith('__') or name.value.endswith('__') \
or self._equals_origin_scope()
@@ -140,75 +142,7 @@ class ClassFilter(ParserTreeFilter):
return [name for name in names if self._access_possible(name)]
def init_param_value(arg_nodes) -> Optional[bool]:
"""
Returns:
- ``True`` if ``@dataclass(init=True)``
- ``False`` if ``@dataclass(init=False)``
- ``None`` if not specified ``@dataclass()``
"""
for arg_node in arg_nodes:
if (
arg_node.type == "argument"
and arg_node.children[0].value == "init"
):
if arg_node.children[2].value == "False":
return False
elif arg_node.children[2].value == "True":
return True
return None
def get_dataclass_param_names(cls) -> List[DataclassParamName]:
"""
``cls`` is a :class:`ClassMixin`. The type is only documented as mypy would
complain that some fields are missing.
.. code:: python
@dataclass
class A:
a: int
b: str = "toto"
For the previous example, the param names would be ``a`` and ``b``.
"""
param_names = []
filter_ = cls.as_context().get_global_filter()
for name in sorted(filter_.values(), key=lambda name: name.start_pos):
d = name.tree_name.get_definition()
annassign = d.children[1]
if d.type == 'expr_stmt' and annassign.type == 'annassign':
node = annassign.children[1]
if node.type == "atom_expr" and node.children[0].value == "ClassVar":
continue
if len(annassign.children) < 4:
default = None
else:
default = annassign.children[3]
param_names.append(DataclassParamName(
parent_context=cls.parent_context,
tree_name=name.tree_name,
annotation_node=annassign.children[1],
default_node=default,
))
return param_names
class ClassMixin:
tree_node: Any
parent_context: Any
inference_state: InferenceState
py__bases__: Any
get_metaclasses: Any
get_metaclass_filters: Any
get_metaclass_signatures: Any
list_type_vars: Any
def is_class(self):
return True
@@ -285,6 +219,7 @@ class ClassMixin:
if not is_instance and include_type_when_class:
from jedi.inference.compiled import builtin_from_name
type_ = builtin_from_name(self.inference_state, 'type')
assert isinstance(type_, ClassValue)
if type_ != self:
# We are not using execute_with_values here, because the
# plugin function for type would get executed instead of an
@@ -299,73 +234,6 @@ class ClassMixin:
assert x is not None
yield x
def _has_dataclass_transform_metaclasses(self) -> Tuple[bool, Optional[bool]]:
for meta in self.get_metaclasses(): # type: ignore[attr-defined]
if (
isinstance(meta, Decoratee)
# Internal leakage :|
and isinstance(meta._wrapped_value, DataclassTransformer)
):
return True, meta._wrapped_value.init_mode_from_new()
return False, None
def _get_dataclass_transform_signatures(self) -> List[DataclassSignature]:
"""
Returns: A non-empty list if the class has dataclass semantics else an
empty list.
The dataclass-like semantics will be assumed for any class that directly
or indirectly derives from the decorated class or uses the decorated
class as a metaclass.
"""
param_names = []
is_dataclass_transform = False
default_init_mode: Optional[bool] = None
for cls in reversed(list(self.py__mro__())):
if not is_dataclass_transform:
# If dataclass_transform is applied to a class, dataclass-like semantics
# will be assumed for any class that directly or indirectly derives from
# the decorated class or uses the decorated class as a metaclass.
if (
isinstance(cls, DataclassTransformer)
and cls.init_mode_from_init_subclass
):
is_dataclass_transform = True
default_init_mode = cls.init_mode_from_init_subclass
elif (
# Some object like CompiledValues would not be compatible
isinstance(cls, ClassMixin)
):
is_dataclass_transform, default_init_mode = (
cls._has_dataclass_transform_metaclasses()
)
# Attributes on the decorated class and its base classes are not
# considered to be fields.
if is_dataclass_transform:
continue
# All inherited classes behave like dataclass semantics
if (
is_dataclass_transform
and isinstance(cls, ClassValue)
and (
cls.init_param_mode()
or (cls.init_param_mode() is None and default_init_mode)
)
):
param_names.extend(
get_dataclass_param_names(cls)
)
if is_dataclass_transform:
return [DataclassSignature(cls, param_names)]
else:
return []
def get_signatures(self):
# Since calling staticmethod without a function is illegal, the Jedi
# plugin doesn't return anything. Therefore call directly and get what
@@ -376,14 +244,8 @@ class ClassMixin:
if sigs:
return sigs
args = ValuesArguments([])
instance = self.py__call__(args)
init_funcs = init_or_new_func(instance)
dataclass_sigs = self._get_dataclass_transform_signatures()
if dataclass_sigs:
return dataclass_sigs
else:
return [sig.bind(self) for sig in init_funcs.get_signatures()]
init_funcs = self.py__call__(args).py__getattribute__('__init__')
return [sig.bind(self) for sig in init_funcs.get_signatures()]
def _as_context(self):
return ClassContext(self)
@@ -470,175 +332,6 @@ class ClassMixin:
return ValueSet({self})
def init_or_new_func(value):
init_funcs = value.py__getattribute__('__init__')
if len(init_funcs) == 1:
init = next(iter(init_funcs))
try:
class_context = init.class_context
except AttributeError:
pass
else:
# In the case where we are on object.__init__, we try to use
# __new__.
if class_context.get_root_context().is_builtins_module() \
and init.class_context.name.string_name == "object":
return value.py__getattribute__('__new__')
return init_funcs
class DataclassParamName(BaseTreeParamName):
"""
Represent a field declaration on a class with dataclass semantics.
"""
def __init__(self, parent_context, tree_name, annotation_node, default_node):
super().__init__(parent_context, tree_name)
self.annotation_node = annotation_node
self.default_node = default_node
def get_kind(self):
return Parameter.POSITIONAL_OR_KEYWORD
def infer(self):
if self.annotation_node is None:
return NO_VALUES
else:
return self.parent_context.infer_node(self.annotation_node)
class DataclassSignature(AbstractSignature):
"""
It represents the ``__init__`` signature of a class with dataclass semantics.
.. code:: python
"""
def __init__(self, value, param_names):
super().__init__(value)
self._param_names = param_names
def get_param_names(self, resolve_stars=False):
return self._param_names
class DataclassDecorator(ValueWrapper, FunctionMixin):
"""
A dataclass(-like) decorator with custom parameters.
.. code:: python
@dataclass(init=True) # this
class A: ...
@dataclass_transform
def create_model(*, init=False): pass
@create_model(init=False) # or this
class B: ...
"""
def __init__(self, function, arguments, default_init: bool = True):
"""
Args:
function: Decoratee | function
arguments: The parameters to the dataclass function decorator
default_init: Boolean to indicate the default init value
"""
super().__init__(function)
argument_init = self._init_param_value(arguments)
self.init_param_mode = (
argument_init if argument_init is not None else default_init
)
def _init_param_value(self, arguments) -> Optional[bool]:
if not arguments.argument_node:
return None
arg_nodes = (
arguments.argument_node.children
if arguments.argument_node.type == "arglist"
else [arguments.argument_node]
)
return init_param_value(arg_nodes)
class DataclassTransformer(ValueWrapper, ClassMixin):
"""
A class decorated with the ``dataclass_transform`` decorator. dataclass-like
semantics will be assumed for any class that directly or indirectly derives
from the decorated class or uses the decorated class as a metaclass.
Attributes on the decorated class and its base classes are not considered to
be fields.
"""
def __init__(self, wrapped_value):
super().__init__(wrapped_value)
def init_mode_from_new(self) -> bool:
"""Default value if missing is ``True``"""
new_methods = self._wrapped_value.py__getattribute__("__new__")
if not new_methods:
return True
new_method = list(new_methods)[0]
for param in new_method.get_param_names():
if (
param.string_name == "init"
and param.default_node
and param.default_node.type == "keyword"
):
if param.default_node.value == "False":
return False
elif param.default_node.value == "True":
return True
return True
@property
def init_mode_from_init_subclass(self) -> Optional[bool]:
# def __init_subclass__(cls) -> None: ... is hardcoded in the typeshed
# so the extra parameters can not be inferred.
return True
class DataclassWrapper(ValueWrapper, ClassMixin):
"""
A class with dataclass semantics from a decorator. The init parameters are
only from the current class and parent classes decorated where the ``init``
parameter was ``True``.
.. code:: python
@dataclass
class A: ... # this
@dataclass_transform
def create_model(): pass
@create_model()
class B: ... # or this
"""
def __init__(
self, wrapped_value, should_generate_init: bool
):
super().__init__(wrapped_value)
self.should_generate_init = should_generate_init
def get_signatures(self):
param_names = []
for cls in reversed(list(self.py__mro__())):
if (
isinstance(cls, DataclassWrapper)
and cls.should_generate_init
):
param_names.extend(get_dataclass_param_names(cls))
return [DataclassSignature(cls, param_names)]
class ClassValue(ClassMixin, FunctionAndClassBase, metaclass=CachedMetaClass):
api_type = 'class'
@@ -705,22 +398,6 @@ class ClassValue(ClassMixin, FunctionAndClassBase, metaclass=CachedMetaClass):
return values
return NO_VALUES
def init_param_mode(self) -> Optional[bool]:
"""
It returns ``True`` if ``class X(init=False):`` else ``False``.
"""
bases_arguments = self._get_bases_arguments()
if bases_arguments is None:
return None
if bases_arguments.argument_node.type != "arglist":
# If it is not inheriting from the base model and having
# extra parameters, then init behavior is not changed.
return None
return init_param_value(bases_arguments.argument_node.children)
@plugin_manager.decorate()
def get_metaclass_signatures(self, metaclasses):
return []
+6 -18
View File
@@ -1,6 +1,6 @@
import os
from pathlib import Path
from typing import Optional, TYPE_CHECKING, Any
from typing import Optional
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.names import AbstractNameDefinition, ModuleName
@@ -13,9 +13,6 @@ from jedi.inference.compiled import create_simple_object
from jedi.inference.base_value import ValueSet
from jedi.inference.context import ModuleContext
if TYPE_CHECKING:
from jedi.inference import InferenceState
class _ModuleAttributeName(AbstractNameDefinition):
"""
@@ -38,11 +35,6 @@ class _ModuleAttributeName(AbstractNameDefinition):
class SubModuleDictMixin:
inference_state: "InferenceState"
is_package: Any
py__path__: Any
as_context: Any
@inference_state_method_cache()
def sub_modules_dict(self):
"""
@@ -65,10 +57,6 @@ class SubModuleDictMixin:
class ModuleMixin(SubModuleDictMixin):
_module_name_class = ModuleName
tree_node: Any
string_names: Any
sub_modules_dict: Any
py__file__: Any
def get_filters(self, origin_scope=None):
yield MergedFilter(
@@ -76,7 +64,7 @@ class ModuleMixin(SubModuleDictMixin):
parent_context=self.as_context(),
origin_scope=origin_scope
),
GlobalNameFilter(self.as_context()),
GlobalNameFilter(self.as_context(), self.tree_node),
)
yield DictFilter(self.sub_modules_dict())
yield DictFilter(self._module_attributes_dict())
@@ -92,7 +80,7 @@ class ModuleMixin(SubModuleDictMixin):
def is_stub(self):
return False
@property
@property # type: ignore[misc]
@inference_state_method_cache()
def name(self):
return self._module_name_class(self, self.string_names[-1])
@@ -150,7 +138,7 @@ class ModuleValue(ModuleMixin, TreeValue):
api_type = 'module'
def __init__(self, inference_state, module_node, code_lines, file_io=None,
string_names=None, is_package=False) -> None:
string_names=None, is_package=False):
super().__init__(
inference_state,
parent_context=None,
@@ -160,8 +148,8 @@ class ModuleValue(ModuleMixin, TreeValue):
if file_io is None:
self._path: Optional[Path] = None
else:
self._path = file_io.path
self.string_names: Optional[tuple[str, ...]] = string_names
self._path = Path(file_io.path)
self.string_names = string_names # Optional[Tuple[str, ...]]
self.code_lines = code_lines
self._is_package = is_package
+2 -5
View File
@@ -1,6 +1,3 @@
from pathlib import Path
from typing import Optional
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.filters import DictFilter
from jedi.inference.names import ValueNameMixin, AbstractNameDefinition
@@ -38,13 +35,13 @@ class ImplicitNamespaceValue(Value, SubModuleDictMixin):
def get_qualified_names(self):
return ()
@property
@property # type: ignore[misc]
@inference_state_method_cache()
def name(self):
string_name = self.py__package__()[-1]
return ImplicitNSName(self, string_name)
def py__file__(self) -> Optional[Path]:
def py__file__(self):
return None
def py__package__(self):
+10 -28
View File
@@ -216,14 +216,11 @@ def is_scope(node):
def _get_parent_scope_cache(func):
cache = WeakKeyDictionary()
def wrapper(parso_cache_node, node, include_flows=False):
if parso_cache_node is None:
return func(node, include_flows)
def wrapper(used_names, node, include_flows=False):
try:
for_module = cache[parso_cache_node]
for_module = cache[used_names]
except KeyError:
for_module = cache[parso_cache_node] = {}
for_module = cache[used_names] = {}
try:
return for_module[node]
@@ -259,7 +256,7 @@ def get_parent_scope(node, include_flows=False):
# the if, but the parent of the if.
if not (scope.type == 'if_stmt'
and any(n.start_pos <= node.start_pos < n.end_pos
for n in scope.get_test_nodes())): # type: ignore[attr-defined]
for n in scope.get_test_nodes())):
return scope
scope = scope.parent
@@ -273,18 +270,7 @@ def get_cached_code_lines(grammar, path):
Basically access the cached code lines in parso. This is not the nicest way
to do this, but we avoid splitting all the lines again.
"""
return get_parso_cache_node(grammar, path).lines
def get_parso_cache_node(grammar, path):
"""
This is of course not public. But as long as I control parso, this
shouldn't be a problem. ~ Dave
The reason for this is mostly caching. This is obviously also a sign of a
broken caching architecture.
"""
return parser_cache[grammar._hashed][path]
return parser_cache[grammar._hashed][path].lines
def cut_value_at_position(leaf, position):
@@ -320,7 +306,7 @@ def expr_is_dotted(node):
return node.type == 'name'
def _function_is_x_method(decorator_checker):
def _function_is_x_method(*method_names):
def wrapper(function_node):
"""
This is a heuristic. It will not hold ALL the times, but it will be
@@ -330,16 +316,12 @@ def _function_is_x_method(decorator_checker):
"""
for decorator in function_node.get_decorators():
dotted_name = decorator.children[1]
if decorator_checker(dotted_name.get_code()):
if dotted_name.get_code() in method_names:
return True
return False
return wrapper
function_is_staticmethod = _function_is_x_method(lambda m: m == "staticmethod")
function_is_classmethod = _function_is_x_method(lambda m: m == "classmethod")
function_is_property = _function_is_x_method(
lambda m: m == "property"
or m == "cached_property"
or (m.endswith(".setter"))
)
function_is_staticmethod = _function_is_x_method('staticmethod')
function_is_classmethod = _function_is_x_method('classmethod')
function_is_property = _function_is_x_method('property', 'cached_property')
+3 -4
View File
@@ -2,7 +2,6 @@
Module is used to infer Django model fields.
"""
from inspect import Parameter
from typing import Any
from jedi import debug
from jedi.inference.cache import inference_state_function_cache
@@ -46,7 +45,7 @@ _FILTER_LIKE_METHODS = ('create', 'filter', 'exclude', 'update', 'get',
def _get_deferred_attributes(inference_state):
return inference_state.import_module(
('django', 'db', 'models', 'query_utils')
).py__getattribute__('DeferredAttribute').execute_annotation(None)
).py__getattribute__('DeferredAttribute').execute_annotation()
def _infer_scalar_field(inference_state, field_name, field_tree_instance, is_instance):
@@ -130,7 +129,7 @@ def _create_manager_for(cls, manager_cls='BaseManager'):
for m in managers:
if m.is_class_mixin():
generics_manager = TupleGenericManager((ValueSet([cls]),))
for c in GenericClass(m, generics_manager).execute_annotation(None):
for c in GenericClass(m, generics_manager).execute_annotation():
return c
return None
@@ -141,7 +140,7 @@ def _new_dict_filter(cls, is_instance):
include_metaclasses=False,
include_type_when_class=False)
)
dct: dict[str, Any] = {
dct = {
name.string_name: DjangoModelName(cls, name, is_instance)
for filter_ in reversed(filters)
for name in filter_.values()
+15 -86
View File
@@ -1,9 +1,8 @@
import sys
from typing import List
from pathlib import Path
from parso.tree import search_ancestor
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.imports import goto_import, load_module_from_path
from jedi.inference.imports import load_module_from_path
from jedi.inference.filters import ParserTreeFilter
from jedi.inference.base_value import NO_VALUES, ValueSet
from jedi.inference.helpers import infer_call_of_leaf
@@ -32,15 +31,7 @@ def execute(callback):
def infer_anonymous_param(func):
def get_returns(value):
if value.tree_node.annotation is not None:
result = value.execute_with_values()
if any(v.name.get_qualified_names(include_module_names=True)
== ('typing', 'Generator')
for v in result):
return ValueSet.from_sets(
v.py__getattribute__('__next__').execute_annotation(None)
for v in result
)
return result
return value.execute_with_values()
# In pytest we need to differentiate between generators and normal
# returns.
@@ -52,9 +43,6 @@ def infer_anonymous_param(func):
return function_context.get_return_values()
def wrapper(param_name):
# parameters with an annotation do not need special handling
if param_name.annotation_node:
return func(param_name)
is_pytest_param, param_name_is_function_name = \
_is_a_pytest_param_and_inherited(param_name)
if is_pytest_param:
@@ -119,7 +107,7 @@ def _is_a_pytest_param_and_inherited(param_name):
This is a heuristic and will work in most cases.
"""
funcdef = param_name.tree_name.search_ancestor('funcdef')
funcdef = search_ancestor(param_name.tree_name, 'funcdef')
if funcdef is None: # A lambda
return False, False
decorators = funcdef.get_decorators()
@@ -132,22 +120,6 @@ def _is_pytest_func(func_name, decorator_nodes):
or any('fixture' in n.get_code() for n in decorator_nodes)
def _find_pytest_plugin_modules() -> List[List[str]]:
"""
Finds pytest plugin modules hooked by setuptools entry points
See https://docs.pytest.org/en/stable/how-to/writing_plugins.html#setuptools-entry-points
"""
from importlib.metadata import entry_points
if sys.version_info >= (3, 10):
pytest_entry_points = entry_points(group="pytest11")
else:
pytest_entry_points = entry_points().get("pytest11", ())
return [ep.module.split(".") for ep in pytest_entry_points]
@inference_state_method_cache()
def _iter_pytest_modules(module_context, skip_own_module=False):
if not skip_own_module:
@@ -157,74 +129,32 @@ def _iter_pytest_modules(module_context, skip_own_module=False):
if file_io is not None:
folder = file_io.get_parent_folder()
sys_path = module_context.inference_state.get_sys_path()
# prevent an infinite loop when reaching the root of the current drive
last_folder = None
while any(folder.path.startswith(p) for p in sys_path):
file_io = folder.get_file_io('conftest.py')
if Path(file_io.path) != module_context.py__file__():
try:
m = load_module_from_path(module_context.inference_state, file_io)
conftest_module = m.as_context()
yield conftest_module
plugins_list = m.tree_node.get_used_names().get("pytest_plugins")
if plugins_list:
name = conftest_module.create_name(plugins_list[0])
yield from _load_pytest_plugins(module_context, name)
yield m.as_context()
except FileNotFoundError:
pass
folder = folder.get_parent_folder()
# prevent an infinite for loop if the same parent folder is return twice
if last_folder is not None and folder.path == last_folder.path: # type: ignore # TODO
break
last_folder = folder # keep track of the last found parent name
for names in _PYTEST_FIXTURE_MODULES + _find_pytest_plugin_modules():
for names in _PYTEST_FIXTURE_MODULES:
for module_value in module_context.inference_state.import_module(names):
yield module_value.as_context()
def _load_pytest_plugins(module_context, name):
from jedi.inference.helpers import get_str_or_none
for inferred in name.infer():
for seq_value in inferred.py__iter__():
for value in seq_value.infer():
fq_name = get_str_or_none(value)
if fq_name:
names = fq_name.split(".")
for module_value in module_context.inference_state.import_module(names):
yield module_value.as_context()
class FixtureFilter(ParserTreeFilter):
def _filter(self, names):
for name in super()._filter(names):
# look for fixture definitions of imported names
if name.parent.type == "import_from":
imported_names = goto_import(self.parent_context, name)
if any(
self._is_fixture(iname.parent_context, iname.tree_name)
for iname in imported_names
# discard imports of whole modules, that have no tree_name
if iname.tree_name
):
funcdef = name.parent
# Class fixtures are not supported
if funcdef.type == 'funcdef':
decorated = funcdef.parent
if decorated.type == 'decorated' and self._is_fixture(decorated):
yield name
elif self._is_fixture(self.parent_context, name):
yield name
def _is_fixture(self, context, name):
funcdef = name.parent
# Class fixtures are not supported
if funcdef.type != "funcdef":
return False
decorated = funcdef.parent
if decorated.type != "decorated":
return False
def _is_fixture(self, decorated):
decorators = decorated.children[0]
if decorators.type == 'decorators':
decorators = decorators.children
@@ -241,12 +171,11 @@ class FixtureFilter(ParserTreeFilter):
last_leaf = last_trailer.get_last_leaf()
if last_leaf == ')':
values = infer_call_of_leaf(
context, last_leaf, cut_own_trailer=True
)
self.parent_context, last_leaf, cut_own_trailer=True)
else:
values = context.infer_node(dotted_name)
values = self.parent_context.infer_node(dotted_name)
else:
values = context.infer_node(dotted_name)
values = self.parent_context.infer_node(dotted_name)
for value in values:
if value.name.get_qualified_names(include_module_names=True) \
== ('_pytest', 'fixtures', 'fixture'):
+64 -142
View File
@@ -11,6 +11,7 @@ compiled module that returns the types for C-builtins.
"""
import parso
import os
from inspect import Parameter
from jedi import debug
from jedi.inference.utils import safe_property
@@ -24,20 +25,15 @@ from jedi.inference.value.instance import \
from jedi.inference.base_value import ContextualizedNode, \
NO_VALUES, ValueSet, ValueWrapper, LazyValueWrapper
from jedi.inference.value import ClassValue, ModuleValue
from jedi.inference.value.decorator import Decoratee
from jedi.inference.value.klass import (
DataclassWrapper,
DataclassDecorator,
DataclassTransformer,
)
from jedi.inference.value.klass import ClassMixin
from jedi.inference.value.function import FunctionMixin
from jedi.inference.value import iterable
from jedi.inference.lazy_value import LazyTreeValue, LazyKnownValue, \
LazyKnownValues
from jedi.inference.names import ValueName
from jedi.inference.names import ValueName, BaseTreeParamName
from jedi.inference.filters import AttributeOverwrite, publish_method, \
ParserTreeFilter, DictFilter
from jedi.inference.signature import SignatureWrapper
from jedi.inference.signature import AbstractSignature, SignatureWrapper
# Copied from Python 3.6's stdlib.
@@ -134,7 +130,7 @@ def execute(callback):
except KeyError:
pass
else:
return func(value, arguments=arguments, callback=call) # type: ignore
return func(value, arguments=arguments, callback=call)
return call()
return wrapper
@@ -595,103 +591,65 @@ def _random_choice(sequences):
def _dataclass(value, arguments, callback):
"""
Decorator entry points for dataclass.
1. dataclass decorator declaration with parameters
2. dataclass semantics on a class from a dataclass(-like) decorator
"""
for c in _follow_param(value.inference_state, arguments, 0):
if c.is_class():
# Declare dataclass semantics on a class from a dataclass decorator
should_generate_init = (
# Customized decorator, init may be disabled
value.init_param_mode
if isinstance(value, DataclassDecorator)
# Bare dataclass decorator, always with init mode
else True
)
return ValueSet([DataclassWrapper(c, should_generate_init)])
return ValueSet([DataclassWrapper(c)])
else:
# @dataclass(init=False)
# dataclass decorator customization
return ValueSet(
[
DataclassDecorator(
value,
arguments=arguments,
default_init=True,
)
]
)
return NO_VALUES
def _dataclass_transform(value, arguments, callback):
"""
Decorator entry points for dataclass_transform.
1. dataclass-like decorator instantiation from a dataclass_transform decorator
2. dataclass_transform decorator declaration with parameters
3. dataclass-like decorator declaration with parameters
4. dataclass-like semantics on a class from a dataclass-like decorator
"""
for c in _follow_param(value.inference_state, arguments, 0):
if c.is_class():
is_dataclass_transform = (
value.name.string_name == "dataclass_transform"
# The decorator function from dataclass_transform acting as the
# dataclass decorator.
and not isinstance(value, Decoratee)
# The decorator function from dataclass_transform acting as the
# dataclass decorator with customized parameters
and not isinstance(value, DataclassDecorator)
)
if is_dataclass_transform:
# Declare base class
return ValueSet([DataclassTransformer(c)])
else:
# Declare dataclass-like semantics on a class from a
# dataclass-like decorator
should_generate_init = value.init_param_mode
return ValueSet([DataclassWrapper(c, should_generate_init)])
elif c.is_function():
# dataclass-like decorator instantiation:
# @dataclass_transform
# def create_model()
return ValueSet(
[
DataclassDecorator(
value,
arguments=arguments,
default_init=True,
)
]
)
elif (
# @dataclass_transform
# def create_model(): pass
# @create_model(init=...)
isinstance(value, Decoratee)
):
# dataclass (or like) decorator customization
return ValueSet(
[
DataclassDecorator(
value,
arguments=arguments,
default_init=value._wrapped_value.init_param_mode,
)
]
)
else:
# dataclass_transform decorator with parameters; nothing impactful
return ValueSet([value])
return NO_VALUES
class DataclassWrapper(ValueWrapper, ClassMixin):
def get_signatures(self):
param_names = []
for cls in reversed(list(self.py__mro__())):
if isinstance(cls, DataclassWrapper):
filter_ = cls.as_context().get_global_filter()
# .values ordering is not guaranteed, at least not in
# Python < 3.6, when dicts where not ordered, which is an
# implementation detail anyway.
for name in sorted(filter_.values(), key=lambda name: name.start_pos):
d = name.tree_name.get_definition()
annassign = d.children[1]
if d.type == 'expr_stmt' and annassign.type == 'annassign':
if len(annassign.children) < 4:
default = None
else:
default = annassign.children[3]
param_names.append(DataclassParamName(
parent_context=cls.parent_context,
tree_name=name.tree_name,
annotation_node=annassign.children[1],
default_node=default,
))
return [DataclassSignature(cls, param_names)]
class DataclassSignature(AbstractSignature):
def __init__(self, value, param_names):
super().__init__(value)
self._param_names = param_names
def get_param_names(self, resolve_stars=False):
return self._param_names
class DataclassParamName(BaseTreeParamName):
def __init__(self, parent_context, tree_name, annotation_node, default_node):
super().__init__(parent_context, tree_name)
self.annotation_node = annotation_node
self.default_node = default_node
def get_kind(self):
return Parameter.POSITIONAL_OR_KEYWORD
def infer(self):
if self.annotation_node is None:
return NO_VALUES
else:
return self.parent_context.infer_node(self.annotation_node)
class ItemGetterCallable(ValueWrapper):
def __init__(self, instance, args_value_set):
super().__init__(instance)
@@ -789,13 +747,6 @@ def _os_path_join(args_set, callback):
return callback()
_path_overrides = {
'dirname': _create_string_input_function(os.path.dirname),
'abspath': _create_string_input_function(os.path.abspath),
'relpath': _create_string_input_function(os.path.relpath),
'join': _os_path_join,
}
_implemented = {
'builtins': {
'getattr': builtins_getattr,
@@ -847,19 +798,17 @@ _implemented = {
# runtime_checkable doesn't really change anything and is just
# adding logs for infering stuff, so we can safely ignore it.
'runtime_checkable': lambda value, arguments, callback: NO_VALUES,
# Python 3.11+
'dataclass_transform': _dataclass_transform,
},
'typing_extensions': {
# Python <3.11
'dataclass_transform': _dataclass_transform,
},
'dataclasses': {
# For now this works at least better than Jedi trying to understand it.
'dataclass': _dataclass
},
'posixpath': _path_overrides,
'ntpath': _path_overrides,
'os.path': {
'dirname': _create_string_input_function(os.path.dirname),
'abspath': _create_string_input_function(os.path.abspath),
'relpath': _create_string_input_function(os.path.relpath),
'join': _os_path_join,
}
}
@@ -909,38 +858,11 @@ class EnumInstance(LazyValueWrapper):
yield f
# Make sure tuple[...] behaves like Tuple[...]
class TupleClassWrapper(ValueWrapper):
def py__getitem__(self, index_value_set, contextualized_node):
return self.inference_state.typing_tuple().py__getitem__(
index_value_set,
contextualized_node,
)
# Make sure type[...] behaves like Type[...]
class TypeClassWrapper(ValueWrapper):
def py__getitem__(self, index_value_set, contextualized_node):
return self.inference_state.typing_type().py__getitem__(
index_value_set,
contextualized_node,
)
def tree_name_to_values(func):
def wrapper(inference_state, context, tree_name):
if tree_name.value == 'sep' \
and context.is_module() and context.py__name__() in ('posixpath', 'ntpath'):
if tree_name.value == 'sep' and context.is_module() and context.py__name__() == 'os.path':
return ValueSet({
compiled.create_simple_object(inference_state, os.path.sep),
})
if tree_name.value == 'tuple' \
and context.is_module() and context.py__name__() == 'builtins':
tup, = func(inference_state, context, tree_name)
return ValueSet([TupleClassWrapper(tup)])
if tree_name.value == 'type' \
and context.is_module() and context.py__name__() == 'builtins':
tup, = func(inference_state, context, tree_name)
return ValueSet([TypeClassWrapper(tup)])
return func(inference_state, context, tree_name)
return wrapper
-9
View File
@@ -143,15 +143,6 @@ This improves autocompletion for libraries that use ``setattr`` or
``globals()`` modifications a lot.
"""
allow_unsafe_interpreter_executions = True
"""
Controls whether descriptors are evaluated when using an Interpreter. This is
something you might want to control when using Jedi from a Repl (e.g. IPython)
Generally this setting allows Jedi to execute __getitem__ and descriptors like
`property`.
"""
# ----------------
# Caching Validity
# ----------------
+1 -1
View File
@@ -2,7 +2,7 @@
Utilities for end-users.
"""
import __main__
import __main__ # type: ignore[import]
from collections import namedtuple
import logging
import traceback
-14
View File
@@ -1,14 +0,0 @@
[tool.zuban]
strict = true
enable_error_code = ["ignore-without-code"]
# Revert some --strict specific flags:
allow_untyped_calls = true
allow_untyped_defs = true
allow_incomplete_defs = true
allow_untyped_globals = true
untyped_strict_optional = false
implicit_reexport = true
# Exclude our copies of external stubs
exclude = "^jedi/third_party|^test/(completion|refactor|static_analysis|examples)/"
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env python
"""
Profile a piece of Python code with ``cProfile`` that uses the diff parser.
Usage:
profile.py <file> [-d] [-s <sort>]
profile.py -h | --help
Options:
-h --help Show this screen.
-d --debug Enable Jedi internal debugging.
-s <sort> Sort the profile results, e.g. cumtime, name [default: time].
"""
import cProfile
from docopt import docopt
from jedi.parser.python import load_grammar
from jedi.parser.diff import DiffParser
from jedi.parser.python import ParserWithRecovery
from jedi.common import splitlines
import jedi
def run(parser, lines):
diff_parser = DiffParser(parser)
diff_parser.update(lines)
# Make sure used_names is loaded
parser.module.used_names
def main(args):
if args['--debug']:
jedi.set_debug_function(notices=True)
with open(args['<file>']) as f:
code = f.read()
grammar = load_grammar()
parser = ParserWithRecovery(grammar, code)
# Make sure used_names is loaded
parser.module.used_names
code = code + '\na\n' # Add something so the diff parser needs to run.
lines = splitlines(code, keepends=True)
cProfile.runctx('run(parser, lines)', globals(), locals(), sort=args['-s'])
if __name__ == '__main__':
args = docopt(__doc__)
main(args)
+1 -1
View File
@@ -13,7 +13,7 @@ Note: This requires the psutil library, available on PyPI.
import time
import sys
import os
import psutil # type: ignore[import-untyped]
import psutil
sys.path.insert(0, os.path.abspath(os.path.dirname(__file__) + '/..'))
import jedi
+1 -1
View File
@@ -56,7 +56,7 @@ def main(args):
run(code, i, infer=infer)
if args['--precision']:
pstats.f8 = f8 # type: ignore[attr-defined] # TODO this does not seem to exist?!
pstats.f8 = f8
jedi.set_debug_function(notices=args['--debug'])
if args['--omit']:
+2 -2
View File
@@ -17,11 +17,11 @@ import sys
try:
import urllib.request as urllib2
except ImportError:
import urllib2 # type: ignore[import-not-found, no-redef]
import urllib2
import gc
from os.path import abspath, dirname
import objgraph # type: ignore[import-untyped]
import objgraph
sys.path.insert(0, dirname(dirname(abspath(__file__))))
import jedi
+28 -7
View File
@@ -21,13 +21,34 @@ per-file-ignores =
jedi/__init__.py:F401
jedi/inference/compiled/__init__.py:F401
jedi/inference/value/__init__.py:F401
exclude =
.tox/*
jedi/third_party/*
test/completion/*
test/examples/*
test/refactor/*
test/static_analysis/*
exclude = jedi/third_party/* .tox/*
[pycodestyle]
max-line-length = 100
[mypy]
# Ensure generics are explicit about what they are (e.g: `List[str]` rather than
# just `List`)
disallow_any_generics = True
disallow_subclassing_any = True
# Avoid creating future gotchas emerging from bad typing
warn_redundant_casts = True
warn_unused_ignores = True
warn_return_any = True
warn_unused_configs = True
warn_unreachable = True
# Require values to be explicitly re-exported; this makes things easier for
# Flake8 too and avoids accidentally importing thing from the "wrong" place
# (which helps avoid circular imports)
implicit_reexport = False
strict_equality = True
[mypy-jedi,jedi.inference.compiled,jedi.inference.value,parso]
# Various __init__.py files which contain re-exports we want to implicitly make.
implicit_reexport = True
+13 -52
View File
@@ -1,5 +1,4 @@
#!/usr/bin/env python
from typing import cast
from setuptools import setup, find_packages
from setuptools.depends import get_module_constant
@@ -10,7 +9,7 @@ __AUTHOR__ = 'David Halter'
__AUTHOR_EMAIL__ = 'davidhalter88@gmail.com'
# Get the version from within jedi. It's defined in exactly one place now.
version = cast(str, get_module_constant("jedi", "__version__"))
version = get_module_constant("jedi", "__version__")
readme = open('README.rst').read() + '\n\n' + open('CHANGELOG.rst').read()
@@ -28,61 +27,24 @@ setup(name='jedi',
maintainer=__AUTHOR__,
maintainer_email=__AUTHOR_EMAIL__,
url='https://github.com/davidhalter/jedi',
project_urls={
"Documentation": 'https://jedi.readthedocs.io/en/latest/',
},
license='MIT',
keywords='python completion refactoring vim',
long_description=readme,
packages=find_packages(exclude=['test', 'test.*']),
python_requires='>=3.10',
# Python 3.13 grammars are added to parso in 0.8.4
install_requires=['parso>=0.8.6,<0.9.0'],
python_requires='>=3.6',
install_requires=['parso>=0.8.0,<0.9.0'],
extras_require={
'dev': [
'pytest<9.0.0',
'testing': [
'pytest<6.0.0',
# docopt for sith doctests
'docopt',
# coloroma for colored debug output
'colorama',
'Django',
'attrs',
'typing_extensions',
# latest version on 2025-06-16
'flake8==7.1.2',
'zuban==0.7.0',
# Arbitrary pins, latest at the time of pinning
'types-setuptools==80.9.0.20250529',
'Django<3.1', # For now pin this.
],
'docs': [
# Just pin all of these.
'alabaster==1.0.0',
'babel==2.18.0',
'certifi==2026.4.22',
'charset-normalizer==3.4.7',
'docutils==0.22.4',
'idna==3.13',
'imagesize==2.0.0',
'iniconfig==2.3.0',
'Jinja2==3.1.6',
'MarkupSafe==3.0.3',
'packaging==26.2',
'pluggy==1.6.0',
'Pygments==2.20.0',
'pytest==9.0.3',
'requests==2.33.1',
'roman-numerals==4.1.0',
'snowballstemmer==3.0.1',
'Sphinx==9.1.0',
'sphinx_rtd_theme==3.1.0',
'sphinxcontrib-applehelp==2.0.0',
'sphinxcontrib-devhelp==2.0.0',
'sphinxcontrib-htmlhelp==2.1.0',
'sphinxcontrib-jquery==4.1',
'sphinxcontrib-jsmath==1.0.1',
'sphinxcontrib-qthelp==2.0.0',
'sphinxcontrib-serializinghtml==2.0.0',
'urllib3==2.6.3',
'qa': [
'flake8==3.8.3',
'mypy==0.782',
],
},
package_data={'jedi': ['*.pyi', 'third_party/typeshed/LICENSE',
@@ -95,11 +57,10 @@ setup(name='jedi',
'License :: OSI Approved :: MIT License',
'Operating System :: OS Independent',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.10',
'Programming Language :: Python :: 3.11',
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Topic :: Software Development :: Libraries :: Python Modules',
'Topic :: Text Editors :: Integrated Development Environments (IDE)',
'Topic :: Utilities',
+3 -1
View File
@@ -35,6 +35,7 @@ Usage:
Options:
-h --help Show this screen.
--record=<file> Exceptions are recorded in here [default: record.json].
-f, --fs-cache By default, file system cache is off for reproducibility.
-n, --maxtries=<nr> Maximum of random tries [default: 100]
-d, --debug Jedi print debugging when an error is raised.
-s Shows the path/line numbers of every completion before it starts.
@@ -43,7 +44,7 @@ Options:
--pudb Launch pudb when error is raised.
"""
from docopt import docopt # type: ignore[import, unused-ignore]
from docopt import docopt # type: ignore[import]
import json
import os
@@ -186,6 +187,7 @@ def main(arguments):
'pudb' if arguments['--pudb'] else None
record = arguments['--record']
jedi.settings.use_filesystem_cache = arguments['--fs-cache']
if arguments['--debug']:
jedi.set_debug_function()
+5 -33
View File
@@ -44,8 +44,6 @@ b[int():]
#? list()
b[:]
#? int()
b[:, :-1]
#? 3
b[:]
@@ -69,20 +67,6 @@ class _StrangeSlice():
#? slice()
_StrangeSlice()[1:2]
for x in b[:]:
#? int()
x
for x in b[:, :-1]:
#?
x
class Foo:
def __getitem__(self, item):
return item
#?
Foo()[:, :-1][0]
# -----------------
# iterable multiplication
@@ -207,16 +191,16 @@ C().a
(f, g) = (1,)
#? int()
f
#? int()
g
#? []
g.
(f, g, h) = (1,'')
#? int()
f
#? str()
g
#? str()
h
#? []
h.
(f1, g1) = 1
#? []
@@ -311,13 +295,9 @@ for x in {1: 3.0, '': 1j}:
dict().values().__iter__
d = dict(a=3, b='')
x, y, z = d.values()
x, = d.values()
#? int() str()
x
#? int() str()
y
#? int() str()
z
#? int()
d['a']
#? int() str() None
@@ -531,11 +511,3 @@ lc = [x for a, *x in [(1, '', 1.0)]]
lc[0][0]
#?
lc[0][1]
xy = (1,)
x, y = *xy, None
# whatever it is should not crash
#?
x
+7 -19
View File
@@ -26,6 +26,11 @@ async def y():
x().__await__().__next
return 2
async def x2():
async with open('asdf') as f:
#? ['readlines']
f.readlines
class A():
@staticmethod
async def b(c=1, d=2):
@@ -47,6 +52,8 @@ async def awaitable_test():
#? str()
foo
# python >= 3.6
async def asgen():
yield 1
await asyncio.sleep(0)
@@ -98,22 +105,3 @@ async def f():
f = await C().async_for_classmethod()
#? C()
f
class AsyncCtxMgr:
def some_method():
pass
async def __aenter__(self):
return self
async def __aexit__(self, *args):
pass
async def asyncctxmgr():
async with AsyncCtxMgr() as acm:
#? AsyncCtxMgr()
acm
#? ['some_method']
acm.som
+3 -19
View File
@@ -232,14 +232,13 @@ def a():
#?
# str literals in comment """ upper
# python >= 3.11
def completion_in_comment():
#? ['Exception', 'ExceptionGroup']
#? ['Exception']
# might fail because the comment is not a leaf: Exception
pass
some_word
#? ['Exception', 'ExceptionGroup']
#? ['Exception']
# Very simple comment completion: Exception
# Commment after it
@@ -389,8 +388,7 @@ with open('') as f:
#? ['closed']
f.closed
for line in f:
# TODO this is wrong
#? bytes()
#? str() bytes()
line
with open('') as f1, open('') as f2:
@@ -415,10 +413,6 @@ with Foo() as f3:
with Foo() as f3:
f3
with open("a"), open("b") as bfile:
#? ['flush']
bfile.flush
# -----------------
# Avoiding multiple definitions
# -----------------
@@ -426,13 +420,3 @@ with open("a"), open("b") as bfile:
some_array = ['', '']
#! ['def upper']
some_array[some_not_defined_index].upper
# -----------------
# operator
# -----------------
#? bool()
res = 'f' in 'foo'; res
#? bool()
res = not {}; res
+2 -3
View File
@@ -31,15 +31,14 @@ if x:
#? ['else']
else
# python >= 3.11
try:
pass
#? ['except', 'Exception', 'ExceptionGroup']
#? ['except', 'Exception']
except
try:
pass
#? 6 ['except', 'Exception', 'ExceptionGroup']
#? 6 ['except', 'Exception']
except AttributeError:
pass
#? ['finally']
+4 -10
View File
@@ -23,13 +23,7 @@ def inheritance_fixture():
@pytest.fixture
def capsysbinary(capsysbinary):
#? ['close']
capsysbinary.clos
return capsysbinary
# used when fixtures are defined in multiple files
pytest_plugins = [
"completion.fixture_module",
]
def testdir(testdir):
#? ['chdir']
testdir.chdir
return testdir
+4 -2
View File
@@ -21,9 +21,11 @@ class Y(X):
#? []
def __doc__
#? []
def __class__
# This might or might not be what we wanted, currently properties are also
# used like this. IMO this is not wanted ~dave.
#? ['__class__']
def __class__
#? []
__class__
-1
View File
@@ -5,7 +5,6 @@ import uuid
from django.db import models
from django.contrib.auth.models import User
from django.db.models.query_utils import DeferredAttribute
from django.db.models.manager import BaseManager
class TagManager(models.Manager):
-7
View File
@@ -284,13 +284,6 @@ def doctest_with_space():
import_issu
"""
def doctest_issue_github_1748():
"""From GitHub #1748
#? 10 []
This. Al
"""
pass
def docstring_rst_identifiers():
"""
-6
View File
@@ -1,6 +0,0 @@
# Exists only for completion/pytest.py
import pytest
@pytest.fixture
def my_module_fixture():
return 1.0
+3 -2
View File
@@ -1,4 +1,5 @@
# python >= 3.11
# python >= 3.6
class Foo:
bar = 1
@@ -14,7 +15,7 @@ Fr'{Foo.bar'
Fr'{Foo.bar
#? ['bar']
Fr'{Foo.bar
#? ['Exception', 'ExceptionGroup']
#? ['Exception']
F"{Excepti
#? 8 Foo
-5
View File
@@ -309,8 +309,3 @@ def annotation2() -> Iterator[float]:
next(annotation1())
#? float()
next(annotation2())
# annotations should override generator inference
#? float()
annotation1()
+1 -2
View File
@@ -2,8 +2,7 @@
#? ['raise']
raise
# python >= 3.11
#? ['Exception', 'ExceptionGroup']
#? ['Exception']
except
#? []

Some files were not shown because too many files have changed in this diff Show More