VimPlug/jedi-fork
1
0
Fork 0
forked from VimPlug/jedi
Code Pull Requests Activity

Merge branch 'refactoring'

Browse Source
This commit is contained in:
Dave Halter
2019-08-24 14:38:45 +02:00
parent 8beea77bc8 9290b7291b
commit c94bce315a
158 changed files with 3957 additions and 3840 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.coveragerc
+1 -1
View File
@@ -1,7 +1,7 @@
[run]
omit =
jedi/_compatibility.py
jedi/evaluate/compiled/subprocess/__main__.py
jedi/inference/compiled/subprocess/__main__.py
jedi/__main__.py
# For now this is not being used.
jedi/refactoring.py
CHANGELOG.rst
+3 -3
View File
@@ -131,12 +131,12 @@ New APIs:
++++++++++++++++++
- The import logic has been rewritten to look more like Python's. There is now
an ``Evaluator.modules`` import cache, which resembles ``sys.modules``.
an ``InferState.modules`` import cache, which resembles ``sys.modules``.
- Integrated the parser of 2to3. This will make refactoring possible. It will
also be possible to check for error messages (like compiling an AST would give)
in the future.
- With the new parser, the evaluation also completely changed. It's now simpler
and more readable.
- With the new parser, the type inference also completely changed. It's now
simpler and more readable.
- Completely rewritten REPL completion.
- Added ``jedi.names``, a command to do static analysis. Thanks to that
sourcegraph guys for sponsoring this!
conftest.py
+1 -1
View File
@@ -12,7 +12,7 @@ from jedi._compatibility import py_version
collect_ignore = [
'setup.py',
'__main__.py',
'jedi/evaluate/compiled/subprocess/__main__.py',
'jedi/inference/compiled/subprocess/__main__.py',
'build/',
'test/examples',
]
docs/docs/development.rst
+27 -27
View File
@@ -47,12 +47,12 @@ The Jedi Core
The core of Jedi consists of three parts:
- :ref:`Parser <parser>`
- :ref:`Python code evaluation <evaluate>`
- :ref:`Python type inference <inference>`
- :ref:`API <dev-api>`
Most people are probably interested in :ref:`code evaluation <evaluate>`,
Most people are probably interested in :ref:`type inference <inference>`,
because that's where all the magic happens. I need to introduce the :ref:`parser
<parser>` first, because :mod:`jedi.evaluate` uses it extensively.
<parser>` first, because :mod:`jedi.inference` uses it extensively.
.. _parser:
@@ -66,32 +66,32 @@ The parser creates a syntax tree that |jedi| analyses and tries to understand.
The grammar that this parsers uses is very similar to the official Python
`grammar files <https://docs.python.org/3/reference/grammar.html>`_.
.. _evaluate:
.. _inference:
Evaluation of python code (evaluate/__init__.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Type inference of python code (inference/__init__.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: jedi.evaluate
.. automodule:: jedi.inference
Evaluation Contexts (evaluate/base_context.py)
Inference Values (inference/base_value.py)
++++++++++++++++++++++++++++++++++++++++++++++++++++++
.. automodule:: jedi.evaluate.base_context
.. automodule:: jedi.inference.base_value
.. inheritance-diagram::
jedi.evaluate.context.instance.TreeInstance
jedi.evaluate.context.klass.ClassContext
jedi.evaluate.context.function.FunctionContext
jedi.evaluate.context.function.FunctionExecutionContext
jedi.inference.value.instance.TreeInstance
jedi.inference.value.klass.Classvalue
jedi.inference.value.function.FunctionValue
jedi.inference.value.function.FunctionExecutionContext
:parts: 1
.. _name_resolution:
Name resolution (evaluate/finder.py)
Name resolution (inference/finder.py)
++++++++++++++++++++++++++++++++++++
.. automodule:: jedi.evaluate.finder
.. automodule:: jedi.inference.finder
.. _dev-api:
@@ -124,33 +124,33 @@ without some features.
.. _iterables:
Iterables & Dynamic Arrays (evaluate/context/iterable.py)
Iterables & Dynamic Arrays (inference/value/iterable.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To understand Python on a deeper level, |jedi| needs to understand some of the
dynamic features of Python like lists that are filled after creation:
.. automodule:: jedi.evaluate.context.iterable
.. automodule:: jedi.inference.value.iterable
.. _dynamic:
Parameter completion (evaluate/dynamic.py)
Parameter completion (inference/dynamic.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: jedi.evaluate.dynamic
.. automodule:: jedi.inference.dynamic
.. _docstrings:
Docstrings (evaluate/docstrings.py)
Docstrings (inference/docstrings.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: jedi.evaluate.docstrings
.. automodule:: jedi.inference.docstrings
.. _refactoring:
Refactoring (evaluate/refactoring.py)
Refactoring (inference/refactoring.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: jedi.refactoring
@@ -169,18 +169,18 @@ Imports & Modules
.. _builtin:
Compiled Modules (evaluate/compiled.py)
Compiled Modules (inference/compiled.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: jedi.evaluate.compiled
.. automodule:: jedi.inference.compiled
.. _imports:
Imports (evaluate/imports.py)
Imports (inference/imports.py)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: jedi.evaluate.imports
.. automodule:: jedi.inference.imports
.. _caching-recursions:
@@ -204,7 +204,7 @@ Caching (cache.py)
Recursions (recursion.py)
~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: jedi.evaluate.recursion
.. automodule:: jedi.inference.recursion
.. _dev-helpers:
docs/docs/features.rst
+1 -2
View File
@@ -67,7 +67,6 @@ Will probably never be implemented:
- metaclasses (how could an auto-completion ever support this)
- ``setattr()``, ``__import__()``
- writing to some dicts: ``globals()``, ``locals()``, ``object.__dict__``
- evaluating ``if`` / ``while`` / ``del``
Caveats
@@ -84,7 +83,7 @@ etc.
**Security**
Security is an important issue for |jedi|. Therefore no Python code is
executed. As long as you write pure Python, everything is evaluated
executed. As long as you write pure Python, everything is inferred
statically. But: If you use builtin modules (``c_builtin``) there is no other
option than to execute those modules. However: Execute isn't that critical (as
e.g. in pythoncomplete, which used to execute *every* import!), because it
jedi/api/__init__.py
+67 -63
View File
@@ -28,19 +28,19 @@ from jedi.api import helpers
from jedi.api.completion import Completion
from jedi.api.environment import InterpreterEnvironment
from jedi.api.project import get_default_project, Project
from jedi.evaluate import Evaluator
from jedi.evaluate import imports
from jedi.evaluate import usages
from jedi.evaluate.arguments import try_iter_content
from jedi.evaluate.helpers import get_module_names, evaluate_call_of_leaf
from jedi.evaluate.sys_path import transform_path_to_dotted
from jedi.evaluate.names import TreeNameDefinition, ParamName
from jedi.evaluate.syntax_tree import tree_name_to_contexts
from jedi.evaluate.context import ModuleContext
from jedi.evaluate.base_context import ContextSet
from jedi.evaluate.context.iterable import unpack_tuple_to_dict
from jedi.evaluate.gradual.conversion import convert_names, convert_contexts
from jedi.evaluate.gradual.utils import load_proper_stub_module
from jedi.inference import InferenceState
from jedi.inference import imports
from jedi.inference import usages
from jedi.inference.arguments import try_iter_content
from jedi.inference.helpers import get_module_names, infer_call_of_leaf
from jedi.inference.sys_path import transform_path_to_dotted
from jedi.inference.names import TreeNameDefinition, ParamName
from jedi.inference.syntax_tree import tree_name_to_values
from jedi.inference.value import ModuleValue
from jedi.inference.base_value import ValueSet
from jedi.inference.value.iterable import unpack_tuple_to_dict
from jedi.inference.gradual.conversion import convert_names, convert_values
from jedi.inference.gradual.utils import load_proper_stub_module
# Jedi uses lots and lots of recursion. By setting this a little bit higher, we
# can remove some "maximum recursion depth" errors.
@@ -62,7 +62,7 @@ class Script(object):
- if `sys_path` parameter is ``None`` and ``VIRTUAL_ENV`` environment
variable is defined, ``sys.path`` for the specified environment will be
guessed (see :func:`jedi.evaluate.sys_path.get_venv_path`) and used for
guessed (see :func:`jedi.inference.sys_path.get_venv_path`) and used for
the script;
- otherwise ``sys.path`` will match that of |jedi|.
@@ -111,11 +111,11 @@ class Script(object):
# TODO deprecate and remove sys_path from the Script API.
if sys_path is not None:
project._sys_path = sys_path
self._evaluator = Evaluator(
self._inference_state = InferenceState(
project, environment=environment, script_path=self.path
)
debug.speed('init')
self._module_node, source = self._evaluator.parse_and_get_code(
self._module_node, source = self._inference_state.parse_and_get_code(
code=source,
path=self.path,
encoding=encoding,
@@ -156,7 +156,7 @@ class Script(object):
is_package = False
if self.path is not None:
import_names, is_p = transform_path_to_dotted(
self._evaluator.get_sys_path(add_parent_paths=False),
self._inference_state.get_sys_path(add_parent_paths=False),
self.path
)
if import_names is not None:
@@ -170,7 +170,7 @@ class Script(object):
if self.path is not None and self.path.endswith('.pyi'):
# We are in a stub file. Try to load the stub properly.
stub_module = load_proper_stub_module(
self._evaluator,
self._inference_state,
file_io,
names,
self._module_node
@@ -181,22 +181,25 @@ class Script(object):
if names is None:
names = ('__main__',)
module = ModuleContext(
self._evaluator, self._module_node, file_io,
module = ModuleValue(
self._inference_state, self._module_node, file_io,
string_names=names,
code_lines=self._code_lines,
is_package=is_package,
)
if names[0] not in ('builtins', '__builtin__', 'typing'):
# These modules are essential for Jedi, so don't overwrite them.
self._evaluator.module_cache.add(names, ContextSet([module]))
self._inference_state.module_cache.add(names, ValueSet([module]))
return module
def _get_module_context(self):
return self._get_module().as_context()
def __repr__(self):
return '<%s: %s %r>' % (
self.__class__.__name__,
repr(self._orig_path),
self._evaluator.environment,
self._inference_state.environment,
)
def completions(self):
@@ -209,7 +212,7 @@ class Script(object):
"""
with debug.increase_indent_cm('completions'):
completion = Completion(
self._evaluator, self._get_module(), self._code_lines,
self._inference_state, self._get_module_context(), self._code_lines,
self._pos, self.call_signatures
)
return completion.completions()
@@ -239,16 +242,16 @@ class Script(object):
if leaf is None:
return []
context = self._evaluator.create_context(self._get_module(), leaf)
context = self._get_module_context().create_context(leaf)
contexts = helpers.evaluate_goto_definition(self._evaluator, context, leaf)
contexts = convert_contexts(
contexts,
values = helpers.infer_goto_definition(self._inference_state, context, leaf)
values = convert_values(
values,
only_stubs=only_stubs,
prefer_stubs=prefer_stubs,
)
defs = [classes.Definition(self._evaluator, c.name) for c in contexts]
defs = [classes.Definition(self._inference_state, c.name) for c in values]
# The additional set here allows the definitions to become unique in an
# API sense. In the internals we want to separate more things than in
# the API.
@@ -276,10 +279,10 @@ class Script(object):
def _goto_assignments(self, follow_imports, follow_builtin_imports,
only_stubs=False, prefer_stubs=False):
def filter_follow_imports(names, check):
def filter_follow_imports(names):
for name in names:
if check(name):
new_names = list(filter_follow_imports(name.goto(), check))
if name.is_import():
new_names = list(filter_follow_imports(name.goto()))
found_builtin = False
if follow_builtin_imports:
for new_name in new_names:
@@ -299,18 +302,18 @@ class Script(object):
# Without a name we really just want to jump to the result e.g.
# executed by `foo()`, if we the cursor is after `)`.
return self.goto_definitions(only_stubs=only_stubs, prefer_stubs=prefer_stubs)
context = self._evaluator.create_context(self._get_module(), tree_name)
names = list(self._evaluator.goto(context, tree_name))
context = self._get_module_context().create_context(tree_name)
names = list(self._inference_state.goto(context, tree_name))
if follow_imports:
names = filter_follow_imports(names, lambda name: name.is_import())
names = filter_follow_imports(names)
names = convert_names(
names,
only_stubs=only_stubs,
prefer_stubs=prefer_stubs,
)
defs = [classes.Definition(self._evaluator, d) for d in set(names)]
defs = [classes.Definition(self._inference_state, d) for d in set(names)]
return helpers.sorted_definitions(defs)
def usages(self, additional_module_paths=(), **kwargs):
@@ -340,9 +343,9 @@ class Script(object):
# Must be syntax
return []
names = usages.usages(self._get_module(), tree_name)
names = usages.usages(self._get_module_context(), tree_name)
definitions = [classes.Definition(self._evaluator, n) for n in names]
definitions = [classes.Definition(self._inference_state, n) for n in names]
if not include_builtins:
definitions = [d for d in definitions if not d.in_builtin_module()]
return helpers.sorted_definitions(definitions)
@@ -368,12 +371,9 @@ class Script(object):
if call_details is None:
return []
context = self._evaluator.create_context(
self._get_module(),
call_details.bracket_leaf
)
context = self._get_module_context().create_context(call_details.bracket_leaf)
definitions = helpers.cache_call_signatures(
self._evaluator,
self._inference_state,
context,
call_details.bracket_leaf,
self._code_lines,
@@ -381,21 +381,21 @@ class Script(object):
)
debug.speed('func_call followed')
# TODO here we use stubs instead of the actual contexts. We should use
# the signatures from stubs, but the actual contexts, probably?!
return [classes.CallSignature(self._evaluator, signature, call_details)
# TODO here we use stubs instead of the actual values. We should use
# the signatures from stubs, but the actual values, probably?!
return [classes.CallSignature(self._inference_state, signature, call_details)
for signature in definitions.get_signatures()]
def _analysis(self):
self._evaluator.is_analysis = True
self._evaluator.analysis_modules = [self._module_node]
module = self._get_module()
self._inference_state.is_analysis = True
self._inference_state.analysis_modules = [self._module_node]
module = self._get_module_context()
try:
for node in get_executable_nodes(self._module_node):
context = module.create_context(node)
if node.type in ('funcdef', 'classdef'):
# Resolve the decorators.
tree_name_to_contexts(self._evaluator, context, node.children[1])
tree_name_to_values(self._inference_state, context, node.children[1])
elif isinstance(node, tree.Import):
import_names = set(node.get_defined_names())
if node.is_nested():
@@ -403,22 +403,22 @@ class Script(object):
for n in import_names:
imports.infer_import(context, n)
elif node.type == 'expr_stmt':
types = context.eval_node(node)
types = context.infer_node(node)
for testlist in node.children[:-1:2]:
# Iterate tuples.
unpack_tuple_to_dict(context, types, testlist)
else:
if node.type == 'name':
defs = self._evaluator.goto_definitions(context, node)
defs = self._inference_state.goto_definitions(context, node)
else:
defs = evaluate_call_of_leaf(context, node)
defs = infer_call_of_leaf(context, node)
try_iter_content(defs)
self._evaluator.reset_recursion_limitations()
self._inference_state.reset_recursion_limitations()
ana = [a for a in self._evaluator.analysis if self.path == a.path]
ana = [a for a in self._inference_state.analysis if self.path == a.path]
return sorted(set(ana), key=lambda x: x.line)
finally:
self._evaluator.is_analysis = False
self._inference_state.is_analysis = False
class Interpreter(Script):
@@ -467,16 +467,20 @@ class Interpreter(Script):
super(Interpreter, self).__init__(source, environment=environment,
_project=Project(os.getcwd()), **kwds)
self.namespaces = namespaces
self._evaluator.allow_descriptor_getattr = self._allow_descriptor_getattr_default
self._inference_state.allow_descriptor_getattr = self._allow_descriptor_getattr_default
def _get_module(self):
return interpreter.MixedModuleContext(
self._evaluator,
self._module_node,
self.namespaces,
@cache.memoize_method
def _get_module_context(self):
tree_module_value = ModuleValue(
self._inference_state, self._module_node,
file_io=KnownContentFileIO(self.path, self._code),
string_names=('__main__',),
code_lines=self._code_lines,
)
return interpreter.MixedModuleContext(
tree_module_value,
self.namespaces,
)
def names(source=None, path=None, encoding='utf-8', all_scopes=False,
@@ -511,10 +515,10 @@ def names(source=None, path=None, encoding='utf-8', all_scopes=False,
# Set line/column to a random position, because they don't matter.
script = Script(source, line=1, column=0, path=path, encoding=encoding, environment=environment)
module_context = script._get_module()
module_context = script._get_module_context()
defs = [
classes.Definition(
script._evaluator,
script._inference_state,
create_name(name)
) for name in get_module_names(script._module_node, all_scopes)
]
jedi/api/classes.py
+72 -74
View File
@@ -9,15 +9,13 @@ import warnings
from jedi import settings
from jedi import debug
from jedi.evaluate.utils import unite
from jedi.inference.utils import unite
from jedi.cache import memoize_method
from jedi.evaluate import imports
from jedi.evaluate import compiled
from jedi.evaluate.imports import ImportName
from jedi.evaluate.context import FunctionExecutionContext
from jedi.evaluate.gradual.typeshed import StubModuleContext
from jedi.evaluate.gradual.conversion import convert_names, convert_contexts
from jedi.evaluate.base_context import ContextSet
from jedi.inference import imports
from jedi.inference.imports import ImportName
from jedi.inference.gradual.typeshed import StubModuleValue
from jedi.inference.gradual.conversion import convert_names, convert_values
from jedi.inference.base_value import ValueSet
from jedi.api.keywords import KeywordName
@@ -25,20 +23,20 @@ def _sort_names_by_start_pos(names):
return sorted(names, key=lambda s: s.start_pos or (0, 0))
def defined_names(evaluator, context):
def defined_names(inference_state, context):
"""
List sub-definitions (e.g., methods in class).
:type scope: Scope
:rtype: list of Definition
"""
filter = next(context.get_filters(search_global=True))
filter = next(context.get_filters())
names = [name for name in filter.values()]
return [Definition(evaluator, n) for n in _sort_names_by_start_pos(names)]
return [Definition(inference_state, n) for n in _sort_names_by_start_pos(names)]
def _contexts_to_definitions(contexts):
return [Definition(c.evaluator, c.name) for c in contexts]
def _values_to_definitions(values):
return [Definition(c.inference_state, c.name) for c in values]
class BaseDefinition(object):
@@ -62,8 +60,8 @@ class BaseDefinition(object):
'argparse._ActionsContainer': 'argparse.ArgumentParser',
}.items())
def __init__(self, evaluator, name):
self._evaluator = evaluator
def __init__(self, inference_state, name):
self._inference_state = inference_state
self._name = name
"""
An instance of :class:`parso.python.tree.Name` subclass.
@@ -71,7 +69,7 @@ class BaseDefinition(object):
self.is_keyword = isinstance(self._name, KeywordName)
@memoize_method
def _get_module(self):
def _get_module_context(self):
# This can take a while to complete, because in the worst case of
# imports (consider `import a` completions), we need to load all
# modules starting with a first.
@@ -80,11 +78,11 @@ class BaseDefinition(object):
@property
def module_path(self):
"""Shows the file path of a module. e.g. ``/usr/lib/python2.7/os.py``"""
module = self._get_module()
module = self._get_module_context()
if module.is_stub() or not module.is_compiled():
# Compiled modules should not return a module path even if they
# have one.
return self._get_module().py__file__()
return self._get_module_context().py__file__()
return None
@@ -97,7 +95,7 @@ class BaseDefinition(object):
:rtype: str or None
"""
return self._name.string_name
return self._name.get_public_name()
@property
def type(self):
@@ -167,8 +165,8 @@ class BaseDefinition(object):
resolve = True
if isinstance(self._name, imports.SubModuleName) or resolve:
for context in self._name.infer():
return context.api_type
for value in self._name.infer():
return value.api_type
return self._name.api_type
@property
@@ -183,14 +181,14 @@ class BaseDefinition(object):
>>> print(d.module_name) # doctest: +ELLIPSIS
json
"""
return self._get_module().name.string_name
return self._get_module_context().py__name__()
def in_builtin_module(self):
"""Whether this is a builtin module."""
if isinstance(self._get_module(), StubModuleContext):
return any(isinstance(context, compiled.CompiledObject)
for context in self._get_module().non_stub_context_set)
return isinstance(self._get_module(), compiled.CompiledObject)
value = self._get_module_context().get_value()
if isinstance(value, StubModuleValue):
return any(v.is_compiled() for v in value.non_stub_value_set)
return value.is_compiled()
@property
def line(self):
@@ -244,7 +242,7 @@ class BaseDefinition(object):
@property
def description(self):
"""A textual description of the object."""
return self._name.string_name
return self._name.get_public_name()
@property
def full_name(self):
@@ -270,7 +268,7 @@ class BaseDefinition(object):
be ``<module 'posixpath' ...>```. However most users find the latter
more practical.
"""
if not self._name.is_context_name:
if not self._name.is_value_name:
return None
names = self._name.get_qualified_names(include_module_names=True)
@@ -286,7 +284,7 @@ class BaseDefinition(object):
return '.'.join(names)
def is_stub(self):
if not self._name.is_context_name:
if not self._name.is_value_name:
return False
return self._name.get_root_context().is_stub()
@@ -298,7 +296,7 @@ class BaseDefinition(object):
def _goto_assignments(self, only_stubs=False, prefer_stubs=False):
assert not (only_stubs and prefer_stubs)
if not self._name.is_context_name:
if not self._name.is_value_name:
return []
names = convert_names(
@@ -306,7 +304,7 @@ class BaseDefinition(object):
only_stubs=only_stubs,
prefer_stubs=prefer_stubs,
)
return [self if n == self._name else Definition(self._evaluator, n)
return [self if n == self._name else Definition(self._inference_state, n)
for n in names]
def infer(self, **kwargs): # Python 2...
@@ -316,20 +314,20 @@ class BaseDefinition(object):
def _infer(self, only_stubs=False, prefer_stubs=False):
assert not (only_stubs and prefer_stubs)
if not self._name.is_context_name:
if not self._name.is_value_name:
return []
# First we need to make sure that we have stub names (if possible) that
# we can follow. If we don't do that, we can end up with the inferred
# results of Python objects instead of stubs.
names = convert_names([self._name], prefer_stubs=True)
contexts = convert_contexts(
ContextSet.from_sets(n.infer() for n in names),
values = convert_values(
ValueSet.from_sets(n.infer() for n in names),
only_stubs=only_stubs,
prefer_stubs=prefer_stubs,
)
resulting_names = [c.name for c in contexts]
return [self if n == self._name else Definition(self._evaluator, n)
resulting_names = [c.name for c in values]
return [self if n == self._name else Definition(self._inference_state, n)
for n in resulting_names]
@property
@@ -343,10 +341,10 @@ class BaseDefinition(object):
"""
# Only return the first one. There might be multiple one, especially
# with overloading.
for context in self._name.infer():
for signature in context.get_signatures():
for value in self._name.infer():
for signature in value.get_signatures():
return [
Definition(self._evaluator, n)
Definition(self._inference_state, n)
for n in signature.get_param_names(resolve_stars=True)
]
@@ -357,16 +355,16 @@ class BaseDefinition(object):
raise AttributeError('There are no params defined on this.')
def parent(self):
if not self._name.is_context_name:
if not self._name.is_value_name:
return None
context = self._name.parent_context
if context is None:
return None
if isinstance(context, FunctionExecutionContext):
context = context.function_context
return Definition(self._evaluator, context.name)
while context.name is None:
# Happens for comprehension contexts
context = context.parent_context
return Definition(self._inference_state, context.name)
def __repr__(self):
return "<%s %sname=%r, description=%r>" % (
@@ -386,7 +384,7 @@ class BaseDefinition(object):
:return str: Returns the line(s) of code or an empty string if it's a
builtin.
"""
if not self._name.is_context_name or self.in_builtin_module():
if not self._name.is_value_name or self.in_builtin_module():
return ''
lines = self._name.get_root_context().code_lines
@@ -396,10 +394,10 @@ class BaseDefinition(object):
return ''.join(lines[start_index:index + after + 1])
def get_signatures(self):
return [Signature(self._evaluator, s) for s in self._name.infer().get_signatures()]
return [Signature(self._inference_state, s) for s in self._name.infer().get_signatures()]
def execute(self):
return _contexts_to_definitions(self._name.infer().execute_evaluated())
return _values_to_definitions(self._name.infer().execute_with_values())
class Completion(BaseDefinition):
@@ -407,8 +405,8 @@ class Completion(BaseDefinition):
`Completion` objects are returned from :meth:`api.Script.completions`. They
provide additional information about a completion.
"""
def __init__(self, evaluator, name, stack, like_name_length):
super(Completion, self).__init__(evaluator, name)
def __init__(self, inference_state, name, stack, like_name_length):
super(Completion, self).__init__(inference_state, name)
self._like_name_length = like_name_length
self._stack = stack
@@ -429,7 +427,7 @@ class Completion(BaseDefinition):
# TODO this doesn't work for nested calls.
append += '='
name = self._name.string_name
name = self._name.get_public_name()
if like_name:
name = name[self._like_name_length:]
return name + append
@@ -485,7 +483,7 @@ class Completion(BaseDefinition):
return Definition.description.__get__(self)
def __repr__(self):
return '<%s: %s>' % (type(self).__name__, self._name.string_name)
return '<%s: %s>' % (type(self).__name__, self._name.get_public_name())
@memoize_method
def follow_definition(self):
@@ -512,8 +510,8 @@ class Definition(BaseDefinition):
*Definition* objects are returned from :meth:`api.Script.goto_assignments`
or :meth:`api.Script.goto_definitions`.
"""
def __init__(self, evaluator, definition):
super(Definition, self).__init__(evaluator, definition)
def __init__(self, inference_state, definition):
super(Definition, self).__init__(inference_state, definition)
@property
def description(self):
@@ -553,7 +551,7 @@ class Definition(BaseDefinition):
if typ == 'function':
# For the description we want a short and a pythonic way.
typ = 'def'
return typ + ' ' + self._name.string_name
return typ + ' ' + self._name.get_public_name()
definition = tree_name.get_definition() or tree_name
# Remove the prefix, because that's not what we want for get_code
@@ -588,7 +586,7 @@ class Definition(BaseDefinition):
"""
defs = self._name.infer()
return sorted(
unite(defined_names(self._evaluator, d) for d in defs),
unite(defined_names(self._inference_state, d.as_context()) for d in defs),
key=lambda s: s._name.start_pos or (0, 0)
)
@@ -606,13 +604,13 @@ class Definition(BaseDefinition):
return self._name.start_pos == other._name.start_pos \
and self.module_path == other.module_path \
and self.name == other.name \
and self._evaluator == other._evaluator
and self._inference_state == other._inference_state
def __ne__(self, other):
return not self.__eq__(other)
def __hash__(self):
return hash((self._name.start_pos, self.module_path, self.name, self._evaluator))
return hash((self._name.start_pos, self.module_path, self.name, self._inference_state))
class Signature(Definition):
@@ -621,8 +619,8 @@ class Signature(Definition):
It knows what functions you are currently in. e.g. `isinstance(` would
return the `isinstance` function. without `(` it would return nothing.
"""
def __init__(self, evaluator, signature):
super(Signature, self).__init__(evaluator, signature.name)
def __init__(self, inference_state, signature):
super(Signature, self).__init__(inference_state, signature.name)
self._signature = signature
@property
@@ -630,7 +628,7 @@ class Signature(Definition):
"""
:return list of ParamDefinition:
"""
return [ParamDefinition(self._evaluator, n)
return [ParamDefinition(self._inference_state, n)
for n in self._signature.get_param_names(resolve_stars=True)]
def to_string(self):
@@ -644,8 +642,8 @@ class CallSignature(Signature):
return the `isinstance` function with its params. Without `(` it would
return nothing.
"""
def __init__(self, evaluator, signature, call_details):
super(CallSignature, self).__init__(evaluator, signature)
def __init__(self, inference_state, signature, call_details):
super(CallSignature, self).__init__(inference_state, signature)
self._call_details = call_details
self._signature = signature
@@ -680,7 +678,7 @@ class ParamDefinition(Definition):
"""
:return list of Definition:
"""
return _contexts_to_definitions(self._name.infer_default())
return _values_to_definitions(self._name.infer_default())
def infer_annotation(self, **kwargs):
"""
@@ -689,7 +687,7 @@ class ParamDefinition(Definition):
:param execute_annotation: If False, the values are not executed and
you get classes instead of instances.
"""
return _contexts_to_definitions(self._name.infer_annotation(**kwargs))
return _values_to_definitions(self._name.infer_annotation(ignore_stars=True, **kwargs))
def to_string(self):
return self._name.to_string()
@@ -709,10 +707,10 @@ class ParamDefinition(Definition):
return self._name.get_kind()
def _format_signatures(context):
def _format_signatures(value):
return '\n'.join(
signature.to_string()
for signature in context.get_signatures()
for signature in value.get_signatures()
)
@@ -725,7 +723,7 @@ class _Help(object):
self._name = definition
@memoize_method
def _get_contexts(self, fast):
def _get_values(self, fast):
if isinstance(self._name, ImportName) and fast:
return {}
@@ -742,20 +740,20 @@ class _Help(object):
"""
full_doc = ''
# Using the first docstring that we see.
for context in self._get_contexts(fast=fast):
for value in self._get_values(fast=fast):
if full_doc:
# In case we have multiple contexts, just return all of them
# In case we have multiple values, just return all of them
# separated by a few dashes.
full_doc += '\n' + '-' * 30 + '\n'
doc = context.py__doc__()
doc = value.py__doc__()
signature_text = ''
if self._name.is_context_name:
if self._name.is_value_name:
if not raw:
signature_text = _format_signatures(context)
if not doc and context.is_stub():
for c in convert_contexts(ContextSet({context}), ignore_compiled=False):
signature_text = _format_signatures(value)
if not doc and value.is_stub():
for c in convert_values(ValueSet({value}), ignore_compiled=False):
doc = c.py__doc__()
if doc:
break
jedi/api/completion.py
+45 -70
View File
@@ -11,11 +11,11 @@ from jedi.api import classes
from jedi.api import helpers
from jedi.api import keywords
from jedi.api.file_name import file_name_completions
from jedi.evaluate import imports
from jedi.evaluate.helpers import evaluate_call_of_leaf, parse_dotted_names
from jedi.evaluate.filters import get_global_filters
from jedi.evaluate.gradual.conversion import convert_contexts
from jedi.parser_utils import get_statement_of_position, cut_value_at_position
from jedi.inference import imports
from jedi.inference.helpers import infer_call_of_leaf, parse_dotted_names
from jedi.inference.context import get_global_filters
from jedi.inference.gradual.conversion import convert_values
from jedi.parser_utils import cut_value_at_position
def get_call_signature_param_names(call_signatures):
@@ -28,7 +28,7 @@ def get_call_signature_param_names(call_signatures):
yield p._name
def filter_names(evaluator, completion_names, stack, like_name):
def filter_names(inference_state, completion_names, stack, like_name):
comp_dct = {}
if settings.case_insensitive_completion:
like_name = like_name.lower()
@@ -39,7 +39,7 @@ def filter_names(evaluator, completion_names, stack, like_name):
if string.startswith(like_name):
new = classes.Completion(
evaluator,
inference_state,
name,
stack,
len(like_name)
@@ -52,28 +52,12 @@ def filter_names(evaluator, completion_names, stack, like_name):
yield new
def get_user_scope(module_context, position):
def get_user_context(module_context, position):
"""
Returns the scope in which the user resides. This includes flows.
"""
user_stmt = get_statement_of_position(module_context.tree_node, position)
if user_stmt is None:
def scan(scope):
for s in scope.children:
if s.start_pos <= position <= s.end_pos:
if isinstance(s, (tree.Scope, tree.Flow)) \
or s.type in ('async_stmt', 'async_funcdef'):
return scan(s) or s
elif s.type in ('suite', 'decorated'):
return scan(s)
return None
scanned_node = scan(module_context.tree_node)
if scanned_node:
return module_context.create_context(scanned_node, node_is_context=True)
return module_context
else:
return module_context.create_context(user_stmt)
leaf = module_context.tree_node.get_leaf_for_position(position, include_prefixes=True)
return module_context.create_context(leaf)
def get_flow_scope_node(module_node, position):
@@ -85,10 +69,11 @@ def get_flow_scope_node(module_node, position):
class Completion:
def __init__(self, evaluator, module, code_lines, position, call_signatures_callback):
self._evaluator = evaluator
self._module_context = module
self._module_node = module.tree_node
def __init__(self, inference_state, module_context, code_lines, position,
call_signatures_callback):
self._inference_state = inference_state
self._module_context = module_context
self._module_node = module_context.tree_node
self._code_lines = code_lines
# The first step of completions is to get the name
@@ -104,25 +89,25 @@ class Completion:
string, start_leaf = _extract_string_while_in_string(leaf, self._position)
if string is not None:
completions = list(file_name_completions(
self._evaluator, self._module_context, start_leaf, string,
self._inference_state, self._module_context, start_leaf, string,
self._like_name, self._call_signatures_callback,
self._code_lines, self._original_position
))
if completions:
return completions
completion_names = self._get_context_completions(leaf)
completion_names = self._get_value_completions(leaf)
completions = filter_names(self._evaluator, completion_names,
completions = filter_names(self._inference_state, completion_names,
self.stack, self._like_name)
return sorted(completions, key=lambda x: (x.name.startswith('__'),
x.name.startswith('_'),
x.name.lower()))
def _get_context_completions(self, leaf):
def _get_value_completions(self, leaf):
"""
Analyzes the context that a completion is made in and decides what to
Analyzes the value that a completion is made in and decides what to
return.
Technically this works by generating a parser stack and analysing the
@@ -135,7 +120,7 @@ class Completion:
- In params (also lambda): no completion before =
"""
grammar = self._evaluator.grammar
grammar = self._inference_state.grammar
self.stack = stack = None
try:
@@ -149,7 +134,7 @@ class Completion:
# completions since this probably just confuses the user.
return []
# If we don't have a context, just use global completion.
# If we don't have a value, just use global completion.
return self._global_completions()
allowed_transitions = \
@@ -208,7 +193,7 @@ class Completion:
if nodes and nodes[-1] in ('as', 'def', 'class'):
# No completions for ``with x as foo`` and ``import x as foo``.
# Also true for defining names as a class or function.
return list(self._get_class_context_completions(is_function=True))
return list(self._get_class_value_completions(is_function=True))
elif "import_stmt" in nonterminals:
level, names = parse_dotted_names(nodes, "import_from" in nonterminals)
@@ -223,7 +208,7 @@ class Completion:
completion_names += self._trailer_completions(dot.get_previous_leaf())
else:
completion_names += self._global_completions()
completion_names += self._get_class_context_completions(is_function=False)
completion_names += self._get_class_value_completions(is_function=False)
if 'trailer' in nonterminals:
call_signatures = self._call_signatures_callback()
@@ -234,17 +219,16 @@ class Completion:
def _get_keyword_completion_names(self, allowed_transitions):
for k in allowed_transitions:
if isinstance(k, str) and k.isalpha():
yield keywords.KeywordName(self._evaluator, k)
yield keywords.KeywordName(self._inference_state, k)
def _global_completions(self):
context = get_user_scope(self._module_context, self._position)
context = get_user_context(self._module_context, self._position)
debug.dbg('global completion scope: %s', context)
flow_scope_node = get_flow_scope_node(self._module_node, self._position)
filters = get_global_filters(
self._evaluator,
context,
self._position,
origin_scope=flow_scope_node
flow_scope_node
)
completion_names = []
for filter in filters:
@@ -252,52 +236,43 @@ class Completion:
return completion_names
def _trailer_completions(self, previous_leaf):
user_context = get_user_scope(self._module_context, self._position)
evaluation_context = self._evaluator.create_context(
self._module_context, previous_leaf
)
contexts = evaluate_call_of_leaf(evaluation_context, previous_leaf)
user_value = get_user_context(self._module_context, self._position)
inferred_context = self._module_context.create_context(previous_leaf)
values = infer_call_of_leaf(inferred_context, previous_leaf)
completion_names = []
debug.dbg('trailer completion contexts: %s', contexts, color='MAGENTA')
for context in contexts:
for filter in context.get_filters(
search_global=False,
origin_scope=user_context.tree_node):
debug.dbg('trailer completion values: %s', values, color='MAGENTA')
for value in values:
for filter in value.get_filters(origin_scope=user_value.tree_node):
completion_names += filter.values()
python_contexts = convert_contexts(contexts)
for c in python_contexts:
if c not in contexts:
for filter in c.get_filters(
search_global=False,
origin_scope=user_context.tree_node):
python_values = convert_values(values)
for c in python_values:
if c not in values:
for filter in c.get_filters(origin_scope=user_value.tree_node):
completion_names += filter.values()
return completion_names
def _get_importer_names(self, names, level=0, only_modules=True):
names = [n.value for n in names]
i = imports.Importer(self._evaluator, names, self._module_context, level)
return i.completion_names(self._evaluator, only_modules=only_modules)
i = imports.Importer(self._inference_state, names, self._module_context, level)
return i.completion_names(self._inference_state, only_modules=only_modules)
def _get_class_context_completions(self, is_function=True):
def _get_class_value_completions(self, is_function=True):
"""
Autocomplete inherited methods when overriding in child class.
"""
leaf = self._module_node.get_leaf_for_position(self._position, include_prefixes=True)
cls = tree.search_ancestor(leaf, 'classdef')
if isinstance(cls, (tree.Class, tree.Function)):
# Complete the methods that are defined in the super classes.
random_context = self._module_context.create_context(
cls,
node_is_context=True
)
else:
if cls is None:
return
# Complete the methods that are defined in the super classes.
class_value = self._module_context.create_value(cls)
if cls.start_pos[1] >= leaf.start_pos[1]:
return
filters = random_context.get_filters(search_global=False, is_instance=True)
filters = class_value.get_filters(is_instance=True)
# The first dict is the dictionary of class itself.
next(filters)
for filter in filters:
jedi/api/environment.py
+7 -7
View File
@@ -10,8 +10,8 @@ from collections import namedtuple
from jedi._compatibility import highest_pickle_protocol, which
from jedi.cache import memoize_method, time_cache
from jedi.evaluate.compiled.subprocess import CompiledSubprocess, \
EvaluatorSameProcess, EvaluatorSubprocess
from jedi.inference.compiled.subprocess import CompiledSubprocess, \
InferenceStateSameProcess, InferenceStateSubprocess
import parso
@@ -109,8 +109,8 @@ class Environment(_BaseEnvironment):
version = '.'.join(str(i) for i in self.version_info)
return '<%s: %s in %s>' % (self.__class__.__name__, version, self.path)
def get_evaluator_subprocess(self, evaluator):
return EvaluatorSubprocess(evaluator, self._get_subprocess())
def get_inference_state_subprocess(self, inference_state):
return InferenceStateSubprocess(inference_state, self._get_subprocess())
@memoize_method
def get_sys_path(self):
@@ -140,8 +140,8 @@ class SameEnvironment(_SameEnvironmentMixin, Environment):
class InterpreterEnvironment(_SameEnvironmentMixin, _BaseEnvironment):
def get_evaluator_subprocess(self, evaluator):
return EvaluatorSameProcess(evaluator)
def get_inference_state_subprocess(self, inference_state):
return InferenceStateSameProcess(inference_state)
def get_sys_path(self):
return sys.path
@@ -286,7 +286,7 @@ def find_virtualenvs(paths=None, **kwargs):
for path in os.listdir(directory):
path = os.path.join(directory, path)
if path in _used_paths:
# A path shouldn't be evaluated twice.
# A path shouldn't be inferred twice.
continue
_used_paths.add(path)
jedi/api/file_name.py
+13 -37
View File
@@ -1,37 +1,13 @@
import os
import sys
from jedi._compatibility import FileNotFoundError, force_unicode, scandir
from jedi.evaluate.names import AbstractArbitraryName
from jedi.inference.names import AbstractArbitraryName
from jedi.api import classes
from jedi.evaluate.helpers import get_str_or_none
from jedi.inference.helpers import get_str_or_none
from jedi.parser_utils import get_string_quote
if sys.version_info < (3,6) or True:
"""
A super-minimal shim around listdir that behave like
scandir for the information we need.
"""
class DirEntry:
def __init__(self, name, basepath):
self.name = name
self.basepath = basepath
def is_dir(self):
path_for_name = os.path.join(self.basepath, self.name)
return os.path.isdir(path_for_name)
def scandir(dir):
return [DirEntry(name, dir) for name in os.listdir(dir)]
else:
from os import scandir
def file_name_completions(evaluator, module_context, start_leaf, string,
def file_name_completions(inference_state, module_context, start_leaf, string,
like_name, call_signatures_callback, code_lines, position):
# First we want to find out what can actually be changed as a name.
like_name_length = len(os.path.basename(string) + like_name)
@@ -54,7 +30,7 @@ def file_name_completions(evaluator, module_context, start_leaf, string,
is_in_os_path_join = False
else:
string = to_be_added + string
base_path = os.path.join(evaluator.project._path, string)
base_path = os.path.join(inference_state.project._path, string)
try:
listed = scandir(base_path)
except FileNotFoundError:
@@ -77,8 +53,8 @@ def file_name_completions(evaluator, module_context, start_leaf, string,
name += os.path.sep
yield classes.Completion(
evaluator,
FileName(evaluator, name[len(must_start_with) - like_name_length:]),
inference_state,
FileName(inference_state, name[len(must_start_with) - like_name_length:]),
stack=None,
like_name_length=like_name_length
)
@@ -109,10 +85,10 @@ def _add_strings(context, nodes, add_slash=False):
string = ''
first = True
for child_node in nodes:
contexts = context.eval_node(child_node)
if len(contexts) != 1:
values = context.infer_node(child_node)
if len(values) != 1:
return None
c, = contexts
c, = values
s = get_str_or_none(c)
if s is None:
return None
@@ -125,7 +101,7 @@ def _add_strings(context, nodes, add_slash=False):
class FileName(AbstractArbitraryName):
api_type = u'path'
is_context_name = False
is_value_name = False
def _add_os_path_join(module_context, start_leaf, bracket_start):
@@ -140,10 +116,10 @@ def _add_os_path_join(module_context, start_leaf, bracket_start):
if start_leaf.type == 'error_leaf':
# Unfinished string literal, like `join('`
context_node = start_leaf.parent
index = context_node.children.index(start_leaf)
value_node = start_leaf.parent
index = value_node.children.index(start_leaf)
if index > 0:
error_node = context_node.children[index - 1]
error_node = value_node.children[index - 1]
if error_node.type == 'error_node' and len(error_node.children) >= 2:
index = -2
if error_node.children[-1].type == 'arglist':
jedi/api/helpers.py
+15 -15
View File
@@ -9,10 +9,10 @@ from parso.python.parser import Parser
from parso.python import tree
from jedi._compatibility import u, Parameter
from jedi.evaluate.base_context import NO_CONTEXTS
from jedi.evaluate.syntax_tree import eval_atom
from jedi.evaluate.helpers import evaluate_call_of_leaf
from jedi.evaluate.compiled import get_string_context_set
from jedi.inference.base_value import NO_VALUES
from jedi.inference.syntax_tree import infer_atom
from jedi.inference.helpers import infer_call_of_leaf
from jedi.inference.compiled import get_string_value_set
from jedi.cache import call_signature_time_cache
@@ -87,7 +87,7 @@ def _get_code_for_stack(code_lines, leaf, position):
if is_after_newline:
if user_stmt.start_pos[1] > position[1]:
# This means that it's actually a dedent and that means that we
# start without context (part of a suite).
# start without value (part of a suite).
return u('')
# This is basically getting the relevant lines.
@@ -136,25 +136,25 @@ def get_stack_at_position(grammar, code_lines, leaf, pos):
)
def evaluate_goto_definition(evaluator, context, leaf):
def infer_goto_definition(inference_state, context, leaf):
if leaf.type == 'name':
# In case of a name we can just use goto_definition which does all the
# magic itself.
return evaluator.goto_definitions(context, leaf)
return inference_state.goto_definitions(context, leaf)
parent = leaf.parent
definitions = NO_CONTEXTS
definitions = NO_VALUES
if parent.type == 'atom':
# e.g. `(a + b)`
definitions = context.eval_node(leaf.parent)
definitions = context.infer_node(leaf.parent)
elif parent.type == 'trailer':
# e.g. `a()`
definitions = evaluate_call_of_leaf(context, leaf)
definitions = infer_call_of_leaf(context, leaf)
elif isinstance(leaf, tree.Literal):
# e.g. `"foo"` or `1.0`
return eval_atom(context, leaf)
return infer_atom(context, leaf)
elif leaf.type in ('fstring_string', 'fstring_start', 'fstring_end'):
return get_string_context_set(evaluator)
return get_string_value_set(inference_state)
return definitions
@@ -376,7 +376,7 @@ def get_call_signature_details(module, position):
@call_signature_time_cache("call_signatures_validity")
def cache_call_signatures(evaluator, context, bracket_leaf, code_lines, user_pos):
def cache_call_signatures(inference_state, context, bracket_leaf, code_lines, user_pos):
"""This function calculates the cache key."""
line_index = user_pos[0] - 1
@@ -390,8 +390,8 @@ def cache_call_signatures(evaluator, context, bracket_leaf, code_lines, user_pos
yield None # Don't cache!
else:
yield (module_path, before_bracket, bracket_leaf.start_pos)
yield evaluate_goto_definition(
evaluator,
yield infer_goto_definition(
inference_state,
context,
bracket_leaf.get_previous_leaf(),
)
jedi/api/interpreter.py
+12 -21
View File
@@ -2,16 +2,15 @@
TODO Some parts of this module are still not well documented.
"""
from jedi.evaluate.context import ModuleContext
from jedi.evaluate import compiled
from jedi.evaluate.compiled import mixed
from jedi.evaluate.compiled.access import create_access_path
from jedi.evaluate.base_context import ContextWrapper
from jedi.inference import compiled
from jedi.inference.compiled import mixed
from jedi.inference.compiled.access import create_access_path
from jedi.inference.context import ModuleContext
def _create(evaluator, obj):
def _create(inference_state, obj):
return compiled.create_from_access_path(
evaluator, create_access_path(evaluator, obj)
inference_state, create_access_path(inference_state, obj)
)
@@ -20,28 +19,20 @@ class NamespaceObject(object):
self.__dict__ = dct
class MixedModuleContext(ContextWrapper):
type = 'mixed_module'
def __init__(self, evaluator, tree_module, namespaces, file_io, code_lines):
module_context = ModuleContext(
evaluator, tree_module,
file_io=file_io,
string_names=('__main__',),
code_lines=code_lines
)
super(MixedModuleContext, self).__init__(module_context)
class MixedModuleContext(ModuleContext):
def __init__(self, tree_module_value, namespaces):
super(MixedModuleContext, self).__init__(tree_module_value)
self._namespace_objects = [NamespaceObject(n) for n in namespaces]
def get_filters(self, *args, **kwargs):
for filter in self._wrapped_context.get_filters(*args, **kwargs):
for filter in self._value.as_context().get_filters(*args, **kwargs):
yield filter
for namespace_obj in self._namespace_objects:
compiled_object = _create(self.evaluator, namespace_obj)
compiled_object = _create(self.inference_state, namespace_obj)
mixed_object = mixed.MixedObject(
compiled_object=compiled_object,
tree_context=self._wrapped_context
tree_value=self._value
)
for filter in mixed_object.get_filters(*args, **kwargs):
yield filter
jedi/api/keywords.py
+9 -9
View File
@@ -1,7 +1,7 @@
import pydoc
from jedi.evaluate.utils import ignored
from jedi.evaluate.names import AbstractArbitraryName
from jedi.inference.utils import ignored
from jedi.inference.names import AbstractArbitraryName
try:
from pydoc_data import topics as pydoc_topics
@@ -15,24 +15,24 @@ except ImportError:
pydoc_topics = None
def get_operator(evaluator, string, pos):
return Keyword(evaluator, string, pos)
def get_operator(inference_state, string, pos):
return Keyword(inference_state, string, pos)
class KeywordName(AbstractArbitraryName):
api_type = u'keyword'
def infer(self):
return [Keyword(self.evaluator, self.string_name, (0, 0))]
return [Keyword(self.inference_state, self.string_name, (0, 0))]
class Keyword(object):
api_type = u'keyword'
def __init__(self, evaluator, name, pos):
self.name = KeywordName(evaluator, name)
def __init__(self, inference_state, name, pos):
self.name = KeywordName(inference_state, name)
self.start_pos = pos
self.parent = evaluator.builtins_module
self.parent = inference_state.builtins_module
@property
def names(self):
@@ -44,7 +44,7 @@ class Keyword(object):
def get_signatures(self):
# TODO this makes no sense, I think Keyword should somehow merge with
# Context to make it easier for the api/classes.py to deal with all
# Value to make it easier for the api/classes.py to deal with all
# of it.
return []
jedi/api/project.py
+10 -10
View File
@@ -6,8 +6,8 @@ from jedi.api.environment import SameEnvironment, \
get_cached_default_environment
from jedi.api.exceptions import WrongVersion
from jedi._compatibility import force_unicode
from jedi.evaluate.sys_path import discover_buildout_paths
from jedi.evaluate.cache import evaluator_as_method_param_cache
from jedi.inference.sys_path import discover_buildout_paths
from jedi.inference.cache import inference_state_as_method_param_cache
from jedi.common.utils import traverse_parents
_CONFIG_FOLDER = '.jedi'
@@ -77,8 +77,8 @@ class Project(object):
py2_comp(path, **kwargs)
@evaluator_as_method_param_cache()
def _get_base_sys_path(self, evaluator, environment=None):
@inference_state_as_method_param_cache()
def _get_base_sys_path(self, inference_state, environment=None):
if self._sys_path is not None:
return self._sys_path
@@ -93,8 +93,8 @@ class Project(object):
pass
return sys_path
@evaluator_as_method_param_cache()
def _get_sys_path(self, evaluator, environment=None, add_parent_paths=True):
@inference_state_as_method_param_cache()
def _get_sys_path(self, inference_state, environment=None, add_parent_paths=True):
"""
Keep this method private for all users of jedi. However internally this
one is used like a public method.
@@ -102,15 +102,15 @@ class Project(object):
suffixed = []
prefixed = []
sys_path = list(self._get_base_sys_path(evaluator, environment))
sys_path = list(self._get_base_sys_path(inference_state, environment))
if self._smart_sys_path:
prefixed.append(self._path)
if evaluator.script_path is not None:
suffixed += discover_buildout_paths(evaluator, evaluator.script_path)
if inference_state.script_path is not None:
suffixed += discover_buildout_paths(inference_state, inference_state.script_path)
if add_parent_paths:
traversed = list(traverse_parents(evaluator.script_path))
traversed = list(traverse_parents(inference_state.script_path))
# AFAIK some libraries have imports like `foo.foo.bar`, which
# leads to the conclusion to by default prefer longer paths
jedi/common/__init__.py
+1 -1
View File
@@ -1 +1 @@
from jedi.common.context import BaseContextSet, BaseContext
from jedi.common.value import BaseValueSet, BaseValue
jedi/common/utils.py
+1 -1
View File
@@ -16,7 +16,7 @@ def traverse_parents(path, include_current=False):
@contextmanager
def monkeypatch(obj, attribute_name, new_value):
"""
Like pytest's monkeypatch, but as a context manager.
Like pytest's monkeypatch, but as a value manager.
"""
old_value = getattr(obj, attribute_name)
try:
jedi/common/context.py → jedi/common/value.py
+13 -13
View File
@@ -1,21 +1,21 @@
class BaseContext(object):
def __init__(self, evaluator, parent_context=None):
self.evaluator = evaluator
class BaseValue(object):
def __init__(self, inference_state, parent_context=None):
self.inference_state = inference_state
self.parent_context = parent_context
def get_root_context(self):
context = self
value = self
while True:
if context.parent_context is None:
return context
context = context.parent_context
if value.parent_context is None:
return value
value = value.parent_context
class BaseContextSet(object):
class BaseValueSet(object):
def __init__(self, iterable):
self._set = frozenset(iterable)
for context in iterable:
assert not isinstance(context, BaseContextSet)
for value in iterable:
assert not isinstance(value, BaseValueSet)
@classmethod
def _from_frozen_set(cls, frozenset_):
@@ -30,7 +30,7 @@ class BaseContextSet(object):
"""
aggregated = set()
for set_ in sets:
if isinstance(set_, BaseContextSet):
if isinstance(set_, BaseValueSet):
aggregated |= set_._set
else:
aggregated |= frozenset(set_)
@@ -61,8 +61,8 @@ class BaseContextSet(object):
def __getattr__(self, name):
def mapper(*args, **kwargs):
return self.from_sets(
getattr(context, name)(*args, **kwargs)
for context in self._set
getattr(value, name)(*args, **kwargs)
for value in self._set
)
return mapper
jedi/evaluate/context/__init__.py
-6
View File
@@ -1,6 +0,0 @@
from jedi.evaluate.context.module import ModuleContext
from jedi.evaluate.context.klass import ClassContext
from jedi.evaluate.context.function import FunctionContext, \
MethodContext, FunctionExecutionContext
from jedi.evaluate.context.instance import AnonymousInstance, BoundMethod, \
CompiledInstance, AbstractInstanceContext, TreeInstance
jedi/evaluate/context/decorator.py
-15
View File
@@ -1,15 +0,0 @@
'''
Decorators are not really contexts, however we need some wrappers to improve
docstrings and other things around decorators.
'''
from jedi.evaluate.base_context import ContextWrapper
class Decoratee(ContextWrapper):
def __init__(self, wrapped_context, original_context):
self._wrapped_context = wrapped_context
self._original_context = original_context
def py__doc__(self):
return self._original_context.py__doc__()
jedi/evaluate/finder.py
-290
View File
@@ -1,290 +0,0 @@
"""
Searching for names with given scope and name. This is very central in Jedi and
Python. The name resolution is quite complicated with descripter,
``__getattribute__``, ``__getattr__``, ``global``, etc.
If you want to understand name resolution, please read the first few chapters
in http://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/.
Flow checks
+++++++++++
Flow checks are not really mature. There's only a check for ``isinstance``. It
would check whether a flow has the form of ``if isinstance(a, type_or_tuple)``.
Unfortunately every other thing is being ignored (e.g. a == '' would be easy to
check for -> a is a string). There's big potential in these checks.
"""
from parso.python import tree
from parso.tree import search_ancestor
from jedi import debug
from jedi import settings
from jedi.evaluate import compiled
from jedi.evaluate import analysis
from jedi.evaluate import flow_analysis
from jedi.evaluate.arguments import TreeArguments
from jedi.evaluate import helpers
from jedi.evaluate.context import iterable
from jedi.evaluate.filters import get_global_filters
from jedi.evaluate.names import TreeNameDefinition
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS
from jedi.parser_utils import is_scope, get_parent_scope
from jedi.evaluate.gradual.conversion import convert_contexts
class NameFinder(object):
def __init__(self, evaluator, context, name_context, name_or_str,
position=None, analysis_errors=True):
self._evaluator = evaluator
# Make sure that it's not just a syntax tree node.
self._context = context
self._name_context = name_context
self._name = name_or_str
if isinstance(name_or_str, tree.Name):
self._string_name = name_or_str.value
else:
self._string_name = name_or_str
self._position = position
self._found_predefined_types = None
self._analysis_errors = analysis_errors
def find(self, filters, attribute_lookup):
"""
:params bool attribute_lookup: Tell to logic if we're accessing the
attribute or the contents of e.g. a function.
"""
names = self.filter_name(filters)
if self._found_predefined_types is not None and names:
check = flow_analysis.reachability_check(
context=self._context,
context_scope=self._context.tree_node,
node=self._name,
)
if check is flow_analysis.UNREACHABLE:
return NO_CONTEXTS
return self._found_predefined_types
types = self._names_to_types(names, attribute_lookup)
if not names and self._analysis_errors and not types \
and not (isinstance(self._name, tree.Name) and
isinstance(self._name.parent.parent, tree.Param)):
if isinstance(self._name, tree.Name):
if attribute_lookup:
analysis.add_attribute_error(
self._name_context, self._context, self._name)
else:
message = ("NameError: name '%s' is not defined."
% self._string_name)
analysis.add(self._name_context, 'name-error', self._name, message)
return types
def _get_origin_scope(self):
if isinstance(self._name, tree.Name):
scope = self._name
while scope.parent is not None:
# TODO why if classes?
if not isinstance(scope, tree.Scope):
break
scope = scope.parent
return scope
else:
return None
def get_filters(self, search_global=False):
origin_scope = self._get_origin_scope()
if search_global:
position = self._position
# For functions and classes the defaults don't belong to the
# function and get evaluated in the context before the function. So
# make sure to exclude the function/class name.
if origin_scope is not None:
ancestor = search_ancestor(origin_scope, 'funcdef', 'classdef', 'lambdef')
lambdef = None
if ancestor == 'lambdef':
# For lambdas it's even more complicated since parts will
# be evaluated later.
lambdef = ancestor
ancestor = search_ancestor(origin_scope, 'funcdef', 'classdef')
if ancestor is not None:
colon = ancestor.children[-2]
if position is not None and position < colon.start_pos:
if lambdef is None or position < lambdef.children[-2].start_pos:
position = ancestor.start_pos
return get_global_filters(self._evaluator, self._context, position, origin_scope)
else:
return self._get_context_filters(origin_scope)
def _get_context_filters(self, origin_scope):
for f in self._context.get_filters(False, self._position, origin_scope=origin_scope):
yield f
# This covers the case where a stub files are incomplete.
if self._context.is_stub():
for c in convert_contexts(ContextSet({self._context})):
for f in c.get_filters():
yield f
def filter_name(self, filters):
"""
Searches names that are defined in a scope (the different
``filters``), until a name fits.
"""
names = []
# This paragraph is currently needed for proper branch evaluation
# (static analysis).
if self._context.predefined_names and isinstance(self._name, tree.Name):
node = self._name
while node is not None and not is_scope(node):
node = node.parent
if node.type in ("if_stmt", "for_stmt", "comp_for", 'sync_comp_for'):
try:
name_dict = self._context.predefined_names[node]
types = name_dict[self._string_name]
except KeyError:
continue
else:
self._found_predefined_types = types
break
for filter in filters:
names = filter.get(self._string_name)
if names:
if len(names) == 1:
n, = names
if isinstance(n, TreeNameDefinition):
# Something somewhere went terribly wrong. This
# typically happens when using goto on an import in an
# __init__ file. I think we need a better solution, but
# it's kind of hard, because for Jedi it's not clear
# that that name has not been defined, yet.
if n.tree_name == self._name:
def_ = self._name.get_definition()
if def_ is not None and def_.type == 'import_from':
continue
break
debug.dbg('finder.filter_name %s in (%s): %s@%s',
self._string_name, self._context, names, self._position)
return list(names)
def _check_getattr(self, inst):
"""Checks for both __getattr__ and __getattribute__ methods"""
# str is important, because it shouldn't be `Name`!
name = compiled.create_simple_object(self._evaluator, self._string_name)
# This is a little bit special. `__getattribute__` is in Python
# executed before `__getattr__`. But: I know no use case, where
# this could be practical and where Jedi would return wrong types.
# If you ever find something, let me know!
# We are inversing this, because a hand-crafted `__getattribute__`
# could still call another hand-crafted `__getattr__`, but not the
# other way around.
names = (inst.get_function_slot_names(u'__getattr__') or
inst.get_function_slot_names(u'__getattribute__'))
return inst.execute_function_slots(names, name)
def _names_to_types(self, names, attribute_lookup):
contexts = ContextSet.from_sets(name.infer() for name in names)
debug.dbg('finder._names_to_types: %s -> %s', names, contexts)
if not names and self._context.is_instance() and not self._context.is_compiled():
# handling __getattr__ / __getattribute__
return self._check_getattr(self._context)
# Add isinstance and other if/assert knowledge.
if not contexts and isinstance(self._name, tree.Name) and \
not self._name_context.is_instance() and not self._context.is_compiled():
flow_scope = self._name
base_nodes = [self._name_context.tree_node]
if any(b.type in ('comp_for', 'sync_comp_for') for b in base_nodes):
return contexts
while True:
flow_scope = get_parent_scope(flow_scope, include_flows=True)
n = _check_flow_information(self._name_context, flow_scope,
self._name, self._position)
if n is not None:
return n
if flow_scope in base_nodes:
break
return contexts
def _check_flow_information(context, flow, search_name, pos):
""" Try to find out the type of a variable just with the information that
is given by the flows: e.g. It is also responsible for assert checks.::
if isinstance(k, str):
k. # <- completion here
ensures that `k` is a string.
"""
if not settings.dynamic_flow_information:
return None
result = None
if is_scope(flow):
# Check for asserts.
module_node = flow.get_root_node()
try:
names = module_node.get_used_names()[search_name.value]
except KeyError:
return None
names = reversed([
n for n in names
if flow.start_pos <= n.start_pos < (pos or flow.end_pos)
])
for name in names:
ass = search_ancestor(name, 'assert_stmt')
if ass is not None:
result = _check_isinstance_type(context, ass.assertion, search_name)
if result is not None:
return result
if flow.type in ('if_stmt', 'while_stmt'):
potential_ifs = [c for c in flow.children[1::4] if c != ':']
for if_test in reversed(potential_ifs):
if search_name.start_pos > if_test.end_pos:
return _check_isinstance_type(context, if_test, search_name)
return result
def _check_isinstance_type(context, element, search_name):
try:
assert element.type in ('power', 'atom_expr')
# this might be removed if we analyze and, etc
assert len(element.children) == 2
first, trailer = element.children
assert first.type == 'name' and first.value == 'isinstance'
assert trailer.type == 'trailer' and trailer.children[0] == '('
assert len(trailer.children) == 3
# arglist stuff
arglist = trailer.children[1]
args = TreeArguments(context.evaluator, context, arglist, trailer)
param_list = list(args.unpack())
# Disallow keyword arguments
assert len(param_list) == 2
(key1, lazy_context_object), (key2, lazy_context_cls) = param_list
assert key1 is None and key2 is None
call = helpers.call_of_leaf(search_name)
is_instance_call = helpers.call_of_leaf(lazy_context_object.data)
# Do a simple get_code comparison. They should just have the same code,
# and everything will be all right.
normalize = context.evaluator.grammar._normalize
assert normalize(is_instance_call) == normalize(call)
except AssertionError:
return None
context_set = NO_CONTEXTS
for cls_or_tup in lazy_context_cls.infer():
if isinstance(cls_or_tup, iterable.Sequence) and cls_or_tup.array_type == 'tuple':
for lazy_context in cls_or_tup.py__iter__():
context_set |= lazy_context.infer().execute_evaluated()
else:
context_set |= cls_or_tup.execute_evaluated()
return context_set
jedi/evaluate/gradual/stub_context.py
-105
View File
@@ -1,105 +0,0 @@
from jedi.evaluate.base_context import ContextWrapper
from jedi.evaluate.context.module import ModuleContext
from jedi.evaluate.filters import ParserTreeFilter, \
TreeNameDefinition
from jedi.evaluate.gradual.typing import TypingModuleFilterWrapper
class StubModuleContext(ModuleContext):
def __init__(self, non_stub_context_set, *args, **kwargs):
super(StubModuleContext, self).__init__(*args, **kwargs)
self.non_stub_context_set = non_stub_context_set
def is_stub(self):
return True
def sub_modules_dict(self):
"""
We have to overwrite this, because it's possible to have stubs that
don't have code for all the child modules. At the time of writing this
there are for example no stubs for `json.tool`.
"""
names = {}
for context in self.non_stub_context_set:
try:
method = context.sub_modules_dict
except AttributeError:
pass
else:
names.update(method())
names.update(super(StubModuleContext, self).sub_modules_dict())
return names
def _get_first_non_stub_filters(self):
for context in self.non_stub_context_set:
yield next(context.get_filters(search_global=False))
def _get_stub_filters(self, search_global, **filter_kwargs):
return [StubFilter(
self.evaluator,
context=self,
search_global=search_global,
**filter_kwargs
)] + list(self.iter_star_filters(search_global=search_global))
def get_filters(self, search_global=False, until_position=None,
origin_scope=None, **kwargs):
filters = super(StubModuleContext, self).get_filters(
search_global, until_position, origin_scope, **kwargs
)
next(filters) # Ignore the first filter and replace it with our own
stub_filters = self._get_stub_filters(
search_global=search_global,
until_position=until_position,
origin_scope=origin_scope,
)
for f in stub_filters:
yield f
for f in filters:
yield f
class TypingModuleWrapper(StubModuleContext):
def get_filters(self, *args, **kwargs):
filters = super(TypingModuleWrapper, self).get_filters(*args, **kwargs)
yield TypingModuleFilterWrapper(next(filters))
for f in filters:
yield f
# From here on down we make looking up the sys.version_info fast.
class _StubName(TreeNameDefinition):
def infer(self):
inferred = super(_StubName, self).infer()
if self.string_name == 'version_info' and self.get_root_context().py__name__() == 'sys':
return [VersionInfo(c) for c in inferred]
return inferred
class StubFilter(ParserTreeFilter):
name_class = _StubName
def __init__(self, *args, **kwargs):
self._search_global = kwargs.pop('search_global') # Python 2 :/
super(StubFilter, self).__init__(*args, **kwargs)
def _is_name_reachable(self, name):
if not super(StubFilter, self)._is_name_reachable(name):
return False
if not self._search_global:
# Imports in stub files are only public if they have an "as"
# export.
definition = name.get_definition()
if definition.type in ('import_from', 'import_name'):
if name.parent.type not in ('import_as_name', 'dotted_as_name'):
return False
n = name.value
if n.startswith('_') and not (n.startswith('__') and n.endswith('__')):
return False
return True
class VersionInfo(ContextWrapper):
pass
jedi/evaluate/lazy_context.py
-59
View File
@@ -1,59 +0,0 @@
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS
from jedi.common.utils import monkeypatch
class AbstractLazyContext(object):
def __init__(self, data):
self.data = data
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self.data)
def infer(self):
raise NotImplementedError
class LazyKnownContext(AbstractLazyContext):
"""data is a context."""
def infer(self):
return ContextSet([self.data])
class LazyKnownContexts(AbstractLazyContext):
"""data is a ContextSet."""
def infer(self):
return self.data
class LazyUnknownContext(AbstractLazyContext):
def __init__(self):
super(LazyUnknownContext, self).__init__(None)
def infer(self):
return NO_CONTEXTS
class LazyTreeContext(AbstractLazyContext):
def __init__(self, context, node):
super(LazyTreeContext, self).__init__(node)
self.context = context
# We need to save the predefined names. It's an unfortunate side effect
# that needs to be tracked otherwise results will be wrong.
self._predefined_names = dict(context.predefined_names)
def infer(self):
with monkeypatch(self.context, 'predefined_names', self._predefined_names):
return self.context.eval_node(self.data)
def get_merged_lazy_context(lazy_contexts):
if len(lazy_contexts) > 1:
return MergedLazyContexts(lazy_contexts)
else:
return lazy_contexts[0]
class MergedLazyContexts(AbstractLazyContext):
"""data is a list of lazy contexts."""
def infer(self):
return ContextSet.from_sets(l.infer() for l in self.data)
jedi/evaluate/parser_cache.py
-6
View File
@@ -1,6 +0,0 @@
from jedi.evaluate.cache import evaluator_function_cache
@evaluator_function_cache()
def get_yield_exprs(evaluator, funcdef):
return list(funcdef.iter_yield_exprs())
jedi/evaluate/__init__.py → jedi/inference/__init__.py
+84 -148
View File
@@ -1,5 +1,5 @@
"""
Evaluation of Python code in |jedi| is based on three assumptions:
Type inference of Python code in |jedi| is based on three assumptions:
* The code uses as least side effects as possible. Jedi understands certain
list/tuple/set modifications, but there's no guarantee that Jedi detects
@@ -12,32 +12,32 @@ Evaluation of Python code in |jedi| is based on three assumptions:
* The programmer is not a total dick, e.g. like `this
<https://github.com/davidhalter/jedi/issues/24>`_ :-)
The actual algorithm is based on a principle called lazy evaluation. That
The actual algorithm is based on a principle I call lazy type inference. That
said, the typical entry point for static analysis is calling
``eval_expr_stmt``. There's separate logic for autocompletion in the API, the
evaluator is all about evaluating an expression.
``infer_expr_stmt``. There's separate logic for autocompletion in the API, the
inference_state is all about inferring an expression.
TODO this paragraph is not what jedi does anymore, it's similar, but not the
same.
Now you need to understand what follows after ``eval_expr_stmt``. Let's
Now you need to understand what follows after ``infer_expr_stmt``. Let's
make an example::
import datetime
datetime.date.toda# <-- cursor here
First of all, this module doesn't care about completion. It really just cares
about ``datetime.date``. At the end of the procedure ``eval_expr_stmt`` will
about ``datetime.date``. At the end of the procedure ``infer_expr_stmt`` will
return the ``date`` class.
To *visualize* this (simplified):
- ``Evaluator.eval_expr_stmt`` doesn't do much, because there's no assignment.
- ``Context.eval_node`` cares for resolving the dotted path
- ``Evaluator.find_types`` searches for global definitions of datetime, which
- ``InferenceState.infer_expr_stmt`` doesn't do much, because there's no assignment.
- ``Value.infer_node`` cares for resolving the dotted path
- ``InferenceState.find_types`` searches for global definitions of datetime, which
it finds in the definition of an import, by scanning the syntax tree.
- Using the import logic, the datetime module is found.
- Now ``find_types`` is called again by ``eval_node`` to find ``date``
- Now ``find_types`` is called again by ``infer_node`` to find ``date``
inside the datetime module.
Now what would happen if we wanted ``datetime.date.foo.bar``? Two more
@@ -49,7 +49,7 @@ What if the import would contain another ``ExprStmt`` like this::
from foo import bar
Date = bar.baz
Well... You get it. Just another ``eval_expr_stmt`` recursion. It's really
Well... You get it. Just another ``infer_expr_stmt`` recursion. It's really
easy. Python can obviously get way more complicated then this. To understand
tuple assignments, list comprehensions and everything else, a lot more code had
to be written.
@@ -58,8 +58,8 @@ Jedi has been tested very well, so you can just start modifying code. It's best
to write your own test first for your "new" feature. Don't be scared of
breaking stuff. As long as the tests pass, you're most likely to be fine.
I need to mention now that lazy evaluation is really good because it
only *evaluates* what needs to be *evaluated*. All the statements and modules
I need to mention now that lazy type inference is really good because it
only *inferes* what needs to be *inferred*. All the statements and modules
that are not used are just being ignored.
"""
from parso.python import tree
@@ -69,38 +69,37 @@ from jedi.file_io import FileIO
from jedi import debug
from jedi import parser_utils
from jedi.evaluate.utils import unite
from jedi.evaluate import imports
from jedi.evaluate import recursion
from jedi.evaluate.cache import evaluator_function_cache
from jedi.evaluate import helpers
from jedi.evaluate.names import TreeNameDefinition, ParamName
from jedi.evaluate.base_context import ContextualizedName, ContextualizedNode, \
ContextSet, NO_CONTEXTS, iterate_contexts
from jedi.evaluate.context import ClassContext, FunctionContext, \
AnonymousInstance, BoundMethod
from jedi.evaluate.context.iterable import CompForContext
from jedi.evaluate.syntax_tree import eval_trailer, eval_expr_stmt, \
eval_node, check_tuple_assignments
from jedi.inference.utils import unite
from jedi.inference import imports
from jedi.inference import recursion
from jedi.inference.cache import inference_state_function_cache
from jedi.inference import helpers
from jedi.inference.names import TreeNameDefinition, ParamName
from jedi.inference.base_value import ContextualizedName, ContextualizedNode, \
ValueSet, NO_VALUES, iterate_values
from jedi.inference.value import ClassValue, FunctionValue
from jedi.inference.context import CompForContext
from jedi.inference.syntax_tree import infer_trailer, infer_expr_stmt, \
infer_node, check_tuple_assignments
from jedi.plugins import plugin_manager
class Evaluator(object):
class InferenceState(object):
def __init__(self, project, environment=None, script_path=None):
if environment is None:
environment = project.get_environment()
self.environment = environment
self.script_path = script_path
self.compiled_subprocess = environment.get_evaluator_subprocess(self)
self.compiled_subprocess = environment.get_inference_state_subprocess(self)
self.grammar = environment.get_grammar()
self.latest_grammar = parso.load_grammar(version='3.7')
self.memoize_cache = {} # for memoize decorators
self.module_cache = imports.ModuleCache() # does the job of `sys.modules`.
self.stub_module_cache = {} # Dict[Tuple[str, ...], Optional[ModuleContext]]
self.compiled_cache = {} # see `evaluate.compiled.create()`
self.stub_module_cache = {} # Dict[Tuple[str, ...], Optional[ModuleValue]]
self.compiled_cache = {} # see `inference.compiled.create()`
self.inferred_element_counts = {}
self.mixed_cache = {} # see `evaluate.compiled.mixed._create()`
self.mixed_cache = {} # see `inference.compiled.mixed._create()`
self.analysis = []
self.dynamic_params_depth = 0
self.is_analysis = False
@@ -111,24 +110,24 @@ class Evaluator(object):
self.reset_recursion_limitations()
self.allow_different_encoding = True
def import_module(self, import_names, parent_module_context=None,
def import_module(self, import_names, parent_module_value=None,
sys_path=None, prefer_stubs=True):
if sys_path is None:
sys_path = self.get_sys_path()
return imports.import_module(self, import_names, parent_module_context,
return imports.import_module(self, import_names, parent_module_value,
sys_path, prefer_stubs=prefer_stubs)
@staticmethod
@plugin_manager.decorate()
def execute(context, arguments):
debug.dbg('execute: %s %s', context, arguments)
def execute(value, arguments):
debug.dbg('execute: %s %s', value, arguments)
with debug.increase_indent_cm():
context_set = context.py__call__(arguments=arguments)
debug.dbg('execute result: %s in %s', context_set, context)
return context_set
value_set = value.py__call__(arguments=arguments)
debug.dbg('execute result: %s in %s', value_set, value)
return value_set
@property
@evaluator_function_cache()
@inference_state_function_cache()
def builtins_module(self):
module_name = u'builtins'
if self.environment.version_info.major == 2:
@@ -137,7 +136,7 @@ class Evaluator(object):
return builtins_module
@property
@evaluator_function_cache()
@inference_state_function_cache()
def typing_module(self):
typing_module, = self.import_module((u'typing',))
return typing_module
@@ -150,9 +149,9 @@ class Evaluator(object):
"""Convenience function"""
return self.project._get_sys_path(self, environment=self.environment, **kwargs)
def eval_element(self, context, element):
def infer_element(self, context, element):
if isinstance(context, CompForContext):
return eval_node(context, element)
return infer_node(context, element)
if_stmt = element
while if_stmt is not None:
@@ -171,7 +170,7 @@ class Evaluator(object):
if_stmt_test = if_stmt.children[1]
name_dicts = [{}]
# If we already did a check, we don't want to do it again -> If
# context.predefined_names is filled, we stop.
# value.predefined_names is filled, we stop.
# We don't want to check the if stmt itself, it's just about
# the content.
if element.start_pos > if_stmt_test.end_pos:
@@ -188,9 +187,9 @@ class Evaluator(object):
# never fall below 1.
if len(definitions) > 1:
if len(name_dicts) * len(definitions) > 16:
debug.dbg('Too many options for if branch evaluation %s.', if_stmt)
debug.dbg('Too many options for if branch inference %s.', if_stmt)
# There's only a certain amount of branches
# Jedi can evaluate, otherwise it will take to
# Jedi can infer, otherwise it will take to
# long.
name_dicts = [{}]
break
@@ -201,41 +200,41 @@ class Evaluator(object):
new_name_dicts = list(original_name_dicts)
for i, name_dict in enumerate(new_name_dicts):
new_name_dicts[i] = name_dict.copy()
new_name_dicts[i][if_name.value] = ContextSet([definition])
new_name_dicts[i][if_name.value] = ValueSet([definition])
name_dicts += new_name_dicts
else:
for name_dict in name_dicts:
name_dict[if_name.value] = definitions
if len(name_dicts) > 1:
result = NO_CONTEXTS
result = NO_VALUES
for name_dict in name_dicts:
with helpers.predefine_names(context, if_stmt, name_dict):
result |= eval_node(context, element)
with context.predefine_names(if_stmt, name_dict):
result |= infer_node(context, element)
return result
else:
return self._eval_element_if_evaluated(context, element)
return self._infer_element_if_inferred(context, element)
else:
if predefined_if_name_dict:
return eval_node(context, element)
return infer_node(context, element)
else:
return self._eval_element_if_evaluated(context, element)
return self._infer_element_if_inferred(context, element)
def _eval_element_if_evaluated(self, context, element):
def _infer_element_if_inferred(self, context, element):
"""
TODO This function is temporary: Merge with eval_element.
TODO This function is temporary: Merge with infer_element.
"""
parent = element
while parent is not None:
parent = parent.parent
predefined_if_name_dict = context.predefined_names.get(parent)
if predefined_if_name_dict is not None:
return eval_node(context, element)
return self._eval_element_cached(context, element)
return infer_node(context, element)
return self._infer_element_cached(context, element)
@evaluator_function_cache(default=NO_CONTEXTS)
def _eval_element_cached(self, context, element):
return eval_node(context, element)
@inference_state_function_cache(default=NO_VALUES)
def _infer_element_cached(self, context, element):
return infer_node(context, element)
def goto_definitions(self, context, name):
def_ = name.get_definition(import_name_always=True)
@@ -244,21 +243,21 @@ class Evaluator(object):
is_classdef = type_ == 'classdef'
if is_classdef or type_ == 'funcdef':
if is_classdef:
c = ClassContext(self, context, name.parent)
c = ClassValue(self, context, name.parent)
else:
c = FunctionContext.from_context(context, name.parent)
return ContextSet([c])
c = FunctionValue.from_context(context, name.parent)
return ValueSet([c])
if type_ == 'expr_stmt':
is_simple_name = name.parent.type not in ('power', 'trailer')
if is_simple_name:
return eval_expr_stmt(context, def_, name)
return infer_expr_stmt(context, def_, name)
if type_ == 'for_stmt':
container_types = context.eval_node(def_.children[3])
container_types = context.infer_node(def_.children[3])
cn = ContextualizedNode(context, def_.children[3])
for_types = iterate_contexts(container_types, cn)
for_types = iterate_values(container_types, cn)
c_node = ContextualizedName(context, name)
return check_tuple_assignments(self, c_node, for_types)
return check_tuple_assignments(c_node, for_types)
if type_ in ('import_from', 'import_name'):
return imports.infer_import(context, name)
else:
@@ -266,7 +265,7 @@ class Evaluator(object):
if result is not None:
return result
return helpers.evaluate_call_of_leaf(context, name)
return helpers.infer_call_of_leaf(context, name)
def _follow_error_node_imports_if_possible(self, context, name):
error_node = tree.search_ancestor(name, 'error_node')
@@ -308,14 +307,14 @@ class Evaluator(object):
elif type_ == 'param':
return [ParamName(context, name)]
elif type_ in ('import_from', 'import_name'):
module_names = imports.infer_import(context, name, is_goto=True)
module_names = imports.goto_import(context, name)
return module_names
else:
return [TreeNameDefinition(context, name)]
else:
contexts = self._follow_error_node_imports_if_possible(context, name)
if contexts is not None:
return [context.name for context in contexts]
values = self._follow_error_node_imports_if_possible(context, name)
if values is not None:
return [value.name for value in values]
par = name.parent
node_type = par.type
@@ -326,18 +325,18 @@ class Evaluator(object):
trailer = trailer.parent
if trailer.type != 'classdef':
if trailer.type == 'decorator':
context_set = context.eval_node(trailer.children[1])
value_set = context.infer_node(trailer.children[1])
else:
i = trailer.parent.children.index(trailer)
to_evaluate = trailer.parent.children[:i]
if to_evaluate[0] == 'await':
to_evaluate.pop(0)
context_set = context.eval_node(to_evaluate[0])
for trailer in to_evaluate[1:]:
context_set = eval_trailer(context, context_set, trailer)
to_infer = trailer.parent.children[:i]
if to_infer[0] == 'await':
to_infer.pop(0)
value_set = context.infer_node(to_infer[0])
for trailer in to_infer[1:]:
value_set = infer_trailer(context, value_set, trailer)
param_names = []
for context in context_set:
for signature in context.get_signatures():
for value in value_set:
for signature in value.get_signatures():
for param_name in signature.get_param_names():
if param_name.string_name == name.value:
param_names.append(param_name)
@@ -347,85 +346,22 @@ class Evaluator(object):
if index > 0:
new_dotted = helpers.deep_ast_copy(par)
new_dotted.children[index - 1:] = []
values = context.eval_node(new_dotted)
values = context.infer_node(new_dotted)
return unite(
value.py__getattribute__(name, name_context=context, is_goto=True)
value.goto(name, name_context=value.as_context())
for value in values
)
if node_type == 'trailer' and par.children[0] == '.':
values = helpers.evaluate_call_of_leaf(context, name, cut_own_trailer=True)
return values.py__getattribute__(name, name_context=context, is_goto=True)
values = helpers.infer_call_of_leaf(context, name, cut_own_trailer=True)
return values.goto(name, name_context=context)
else:
stmt = tree.search_ancestor(
name, 'expr_stmt', 'lambdef'
) or name
if stmt.type == 'lambdef':
stmt = name
return context.py__getattribute__(
name,
position=stmt.start_pos,
search_global=True, is_goto=True
)
def create_context(self, base_context, node, node_is_context=False, node_is_object=False):
def parent_scope(node):
while True:
node = node.parent
if parser_utils.is_scope(node):
return node
elif node.type in ('argument', 'testlist_comp'):
if node.children[1].type in ('comp_for', 'sync_comp_for'):
return node.children[1]
elif node.type == 'dictorsetmaker':
for n in node.children[1:4]:
# In dictionaries it can be pretty much anything.
if n.type in ('comp_for', 'sync_comp_for'):
return n
def from_scope_node(scope_node, is_nested=True, node_is_object=False):
if scope_node == base_node:
return base_context
is_funcdef = scope_node.type in ('funcdef', 'lambdef')
parent_scope = parser_utils.get_parent_scope(scope_node)
parent_context = from_scope_node(parent_scope)
if is_funcdef:
func = FunctionContext.from_context(parent_context, scope_node)
if parent_context.is_class():
instance = AnonymousInstance(
self, parent_context.parent_context, parent_context)
func = BoundMethod(
instance=instance,
function=func
)
if is_nested and not node_is_object:
return func.get_function_execution()
return func
elif scope_node.type == 'classdef':
return ClassContext(self, parent_context, scope_node)
elif scope_node.type in ('comp_for', 'sync_comp_for'):
if node.start_pos >= scope_node.children[-1].start_pos:
return parent_context
return CompForContext.from_comp_for(parent_context, scope_node)
raise Exception("There's a scope that was not managed.")
base_node = base_context.tree_node
if node_is_context and parser_utils.is_scope(node):
scope_node = node
else:
scope_node = parent_scope(node)
if scope_node.type in ('funcdef', 'classdef'):
colon = scope_node.children[scope_node.children.index(':')]
if node.start_pos < colon.start_pos:
parent = node.parent
if not (parent.type == 'param' and parent.name == node):
scope_node = parent_scope(scope_node)
return from_scope_node(scope_node, is_nested=True, node_is_object=node_is_object)
return context.goto(name, position=stmt.start_pos)
def parse_and_get_code(self, code=None, path=None, encoding='utf-8',
use_latest_grammar=False, file_io=None, **kwargs):
jedi/evaluate/analysis.py → jedi/inference/analysis.py
+23 -22
View File
@@ -5,7 +5,7 @@ from parso.python import tree
from jedi._compatibility import force_unicode
from jedi import debug
from jedi.evaluate.helpers import is_string
from jedi.inference.helpers import is_string
CODES = {
@@ -87,7 +87,7 @@ def add(node_context, error_name, node, message=None, typ=Error, payload=None):
module_path = module_context.py__file__()
issue_instance = typ(error_name, module_path, node.start_pos, message)
debug.warning(str(issue_instance), format=False)
node_context.evaluator.analysis.append(issue_instance)
node_context.inference_state.analysis.append(issue_instance)
return issue_instance
@@ -112,15 +112,15 @@ def _check_for_setattr(instance):
for n in stmt_names)
def add_attribute_error(name_context, lookup_context, name):
message = ('AttributeError: %s has no attribute %s.' % (lookup_context, name))
from jedi.evaluate.context.instance import CompiledInstanceName
def add_attribute_error(name_context, lookup_value, name):
message = ('AttributeError: %s has no attribute %s.' % (lookup_value, name))
from jedi.inference.value.instance import CompiledInstanceName
# Check for __getattr__/__getattribute__ existance and issue a warning
# instead of an error, if that happens.
typ = Error
if lookup_context.is_instance() and not lookup_context.is_compiled():
slot_names = lookup_context.get_function_slot_names(u'__getattr__') + \
lookup_context.get_function_slot_names(u'__getattribute__')
if lookup_value.is_instance() and not lookup_value.is_compiled():
slot_names = lookup_value.get_function_slot_names(u'__getattr__') + \
lookup_value.get_function_slot_names(u'__getattribute__')
for n in slot_names:
# TODO do we even get here?
if isinstance(name, CompiledInstanceName) and \
@@ -128,10 +128,10 @@ def add_attribute_error(name_context, lookup_context, name):
typ = Warning
break
if _check_for_setattr(lookup_context):
if _check_for_setattr(lookup_value):
typ = Warning
payload = lookup_context, name
payload = lookup_value, name
add(name_context, 'attribute-error', name, message, typ, payload)
@@ -149,7 +149,7 @@ def _check_for_exception_catch(node_context, jedi_name, exception, payload=None)
for python_cls in exception.mro():
if cls.py__name__() == python_cls.__name__ \
and cls.parent_context == cls.evaluator.builtins_module:
and cls.parent_context.is_builtins_module():
return True
return False
@@ -167,14 +167,14 @@ def _check_for_exception_catch(node_context, jedi_name, exception, payload=None)
if node is None:
return True # An exception block that catches everything.
else:
except_classes = node_context.eval_node(node)
except_classes = node_context.infer_node(node)
for cls in except_classes:
from jedi.evaluate.context import iterable
from jedi.inference.value import iterable
if isinstance(cls, iterable.Sequence) and \
cls.array_type == 'tuple':
# multiple exceptions
for lazy_context in cls.py__iter__():
for typ in lazy_context.infer():
for lazy_value in cls.py__iter__():
for typ in lazy_value.infer():
if check_match(typ, exception):
return True
else:
@@ -191,20 +191,21 @@ def _check_for_exception_catch(node_context, jedi_name, exception, payload=None)
assert trailer.type == 'trailer'
arglist = trailer.children[1]
assert arglist.type == 'arglist'
from jedi.evaluate.arguments import TreeArguments
args = list(TreeArguments(node_context.evaluator, node_context, arglist).unpack())
from jedi.inference.arguments import TreeArguments
args = TreeArguments(node_context.inference_state, node_context, arglist)
unpacked_args = list(args.unpack())
# Arguments should be very simple
assert len(args) == 2
assert len(unpacked_args) == 2
# Check name
key, lazy_context = args[1]
names = list(lazy_context.infer())
key, lazy_value = unpacked_args[1]
names = list(lazy_value.infer())
assert len(names) == 1 and is_string(names[0])
assert force_unicode(names[0].get_safe_value()) == payload[1].value
# Check objects
key, lazy_context = args[0]
objects = lazy_context.infer()
key, lazy_value = unpacked_args[0]
objects = lazy_value.infer()
return payload[0] in objects
except AssertionError:
return False
jedi/evaluate/arguments.py → jedi/inference/arguments.py
+49 -55
View File
@@ -4,15 +4,15 @@ from parso.python import tree
from jedi._compatibility import zip_longest
from jedi import debug
from jedi.evaluate.utils import PushBackIterator
from jedi.evaluate import analysis
from jedi.evaluate.lazy_context import LazyKnownContext, LazyKnownContexts, \
LazyTreeContext, get_merged_lazy_context
from jedi.evaluate.names import ParamName, TreeNameDefinition
from jedi.evaluate.base_context import NO_CONTEXTS, ContextSet, ContextualizedNode
from jedi.evaluate.context import iterable
from jedi.evaluate.cache import evaluator_as_method_param_cache
from jedi.evaluate.param import get_executed_params_and_issues, ExecutedParam
from jedi.inference.utils import PushBackIterator
from jedi.inference import analysis
from jedi.inference.lazy_value import LazyKnownValue, LazyKnownValues, \
LazyTreeValue, get_merged_lazy_value
from jedi.inference.names import ParamName, TreeNameDefinition
from jedi.inference.base_value import NO_VALUES, ValueSet, ContextualizedNode
from jedi.inference.value import iterable
from jedi.inference.cache import inference_state_as_method_param_cache
from jedi.inference.param import get_executed_param_names_and_issues
def try_iter_content(types, depth=0):
@@ -28,8 +28,8 @@ def try_iter_content(types, depth=0):
except AttributeError:
pass
else:
for lazy_context in f():
try_iter_content(lazy_context.infer(), depth + 1)
for lazy_value in f():
try_iter_content(lazy_value.infer(), depth + 1)
class ParamIssue(Exception):
@@ -59,12 +59,12 @@ def repack_with_argument_clinic(string, keep_arguments_param=False, keep_callbac
kwargs.pop('callback', None)
try:
args += tuple(_iterate_argument_clinic(
context.evaluator,
context.inference_state,
arguments,
clinic_args
))
except ParamIssue:
return NO_CONTEXTS
return NO_VALUES
else:
return func(context, *args, **kwargs)
@@ -72,20 +72,20 @@ def repack_with_argument_clinic(string, keep_arguments_param=False, keep_callbac
return decorator
def _iterate_argument_clinic(evaluator, arguments, parameters):
def _iterate_argument_clinic(inference_state, arguments, parameters):
"""Uses a list with argument clinic information (see PEP 436)."""
iterator = PushBackIterator(arguments.unpack())
for i, (name, optional, allow_kwargs, stars) in enumerate(parameters):
if stars == 1:
lazy_contexts = []
lazy_values = []
for key, argument in iterator:
if key is not None:
iterator.push_back((key, argument))
break
lazy_contexts.append(argument)
yield ContextSet([iterable.FakeSequence(evaluator, u'tuple', lazy_contexts)])
lazy_contexts
lazy_values.append(argument)
yield ValueSet([iterable.FakeSequence(inference_state, u'tuple', lazy_values)])
lazy_values
continue
elif stars == 2:
raise NotImplementedError()
@@ -98,15 +98,15 @@ def _iterate_argument_clinic(evaluator, arguments, parameters):
name, len(parameters), i)
raise ParamIssue
context_set = NO_CONTEXTS if argument is None else argument.infer()
value_set = NO_VALUES if argument is None else argument.infer()
if not context_set and not optional:
if not value_set and not optional:
# For the stdlib we always want values. If we don't get them,
# that's ok, maybe something is too hard to resolve, however,
# we will not proceed with the evaluation of that function.
# we will not proceed with the type inference of that function.
debug.warning('argument_clinic "%s" not resolvable.', name)
raise ParamIssue
yield context_set
yield value_set
def _parse_argument_clinic(string):
@@ -132,20 +132,20 @@ def _parse_argument_clinic(string):
class _AbstractArgumentsMixin(object):
def eval_all(self, funcdef=None):
def infer_all(self, funcdef=None):
"""
Evaluates all arguments as a support for static analysis
Inferes all arguments as a support for static analysis
(normally Jedi).
"""
for key, lazy_context in self.unpack():
types = lazy_context.infer()
for key, lazy_value in self.unpack():
types = lazy_value.infer()
try_iter_content(types)
def unpack(self, funcdef=None):
raise NotImplementedError
def get_executed_params_and_issues(self, execution_context):
return get_executed_params_and_issues(execution_context, self)
def get_executed_param_names_and_issues(self, execution_context):
return get_executed_param_names_and_issues(execution_context, self)
def get_calling_nodes(self):
return []
@@ -158,10 +158,10 @@ class AbstractArguments(_AbstractArgumentsMixin):
class AnonymousArguments(AbstractArguments):
def get_executed_params_and_issues(self, execution_context):
from jedi.evaluate.dynamic import search_params
return search_params(
execution_context.evaluator,
def get_executed_param_names_and_issues(self, execution_context):
from jedi.inference.dynamic import search_param_names
return search_param_names(
execution_context.inference_state,
execution_context,
execution_context.tree_node
), []
@@ -198,21 +198,17 @@ def unpack_arglist(arglist):
class TreeArguments(AbstractArguments):
def __init__(self, evaluator, context, argument_node, trailer=None):
def __init__(self, inference_state, context, argument_node, trailer=None):
"""
The argument_node is either a parser node or a list of evaluated
objects. Those evaluated objects may be lists of evaluated objects
themselves (one list for the first argument, one for the second, etc).
:param argument_node: May be an argument_node or a list of nodes.
"""
self.argument_node = argument_node
self.context = context
self._evaluator = evaluator
self._inference_state = inference_state
self.trailer = trailer # Can be None, e.g. in a class definition.
@classmethod
@evaluator_as_method_param_cache()
@inference_state_as_method_param_cache()
def create_cached(cls, *args, **kwargs):
return cls(*args, **kwargs)
@@ -220,17 +216,17 @@ class TreeArguments(AbstractArguments):
named_args = []
for star_count, el in unpack_arglist(self.argument_node):
if star_count == 1:
arrays = self.context.eval_node(el)
arrays = self.context.infer_node(el)
iterators = [_iterate_star_args(self.context, a, el, funcdef)
for a in arrays]
for values in list(zip_longest(*iterators)):
# TODO zip_longest yields None, that means this would raise
# an exception?
yield None, get_merged_lazy_context(
yield None, get_merged_lazy_value(
[v for v in values if v is not None]
)
elif star_count == 2:
arrays = self.context.eval_node(el)
arrays = self.context.infer_node(el)
for dct in arrays:
for key, values in _star_star_dict(self.context, dct, el, funcdef):
yield key, values
@@ -238,21 +234,21 @@ class TreeArguments(AbstractArguments):
if el.type == 'argument':
c = el.children
if len(c) == 3: # Keyword argument.
named_args.append((c[0].value, LazyTreeContext(self.context, c[2]),))
named_args.append((c[0].value, LazyTreeValue(self.context, c[2]),))
else: # Generator comprehension.
# Include the brackets with the parent.
sync_comp_for = el.children[1]
if sync_comp_for.type == 'comp_for':
sync_comp_for = sync_comp_for.children[1]
comp = iterable.GeneratorComprehension(
self._evaluator,
self._inference_state,
defining_context=self.context,
sync_comp_for_node=sync_comp_for,
entry_node=el.children[0],
)
yield None, LazyKnownContext(comp)
yield None, LazyKnownValue(comp)
else:
yield None, LazyTreeContext(self.context, el)
yield None, LazyTreeValue(self.context, el)
# Reordering arguments is necessary, because star args sometimes appear
# after named argument, but in the actual order it's prepended.
@@ -279,7 +275,7 @@ class TreeArguments(AbstractArguments):
return '<%s: %s>' % (self.__class__.__name__, self.argument_node)
def get_calling_nodes(self):
from jedi.evaluate.dynamic import DynamicExecutedParams
from jedi.inference.dynamic import DynamicExecutedParamName
old_arguments_list = []
arguments = self
@@ -294,12 +290,10 @@ class TreeArguments(AbstractArguments):
break
if not isinstance(names[0], ParamName):
break
param = names[0].get_param()
if isinstance(param, DynamicExecutedParams):
param = names[0].get_executed_param_name()
if isinstance(param, DynamicExecutedParamName):
# For dynamic searches we don't even want to see errors.
return []
if not isinstance(param, ExecutedParam):
break
if param.var_args is None:
break
arguments = param.var_args
@@ -318,7 +312,7 @@ class ValuesArguments(AbstractArguments):
def unpack(self, funcdef=None):
for values in self._values_list:
yield None, LazyKnownContexts(values)
yield None, LazyKnownValues(values)
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self._values_list)
@@ -362,12 +356,12 @@ def _iterate_star_args(context, array, input_node, funcdef=None):
except AttributeError:
pass
else:
for lazy_context in iter_():
yield lazy_context
for lazy_value in iter_():
yield lazy_value
def _star_star_dict(context, array, input_node, funcdef):
from jedi.evaluate.context.instance import CompiledInstance
from jedi.inference.value.instance import CompiledInstance
if isinstance(array, CompiledInstance) and array.name.string_name == 'dict':
# For now ignore this case. In the future add proper iterators and just
# make one call without crazy isinstance checks.
jedi/evaluate/base_context.py → jedi/inference/base_value.py
+163 -142
View File
@@ -1,122 +1,144 @@
"""
Contexts are the "values" that Python would return. However Contexts are at the
same time also the "contexts" that a user is currently sitting in.
Values are the "values" that Python would return. However Values are at the
same time also the "values" that a user is currently sitting in.
A ContextSet is typically used to specify the return of a function or any other
A ValueSet is typically used to specify the return of a function or any other
static analysis operation. In jedi there are always multiple returns and not
just one.
"""
from functools import reduce
from operator import add
from parso.python.tree import ExprStmt, SyncCompFor
from parso.python.tree import ExprStmt, SyncCompFor, Name
from jedi import debug
from jedi._compatibility import zip_longest, unicode
from jedi.parser_utils import clean_scope_docstring
from jedi.common import BaseContextSet, BaseContext
from jedi.evaluate.helpers import SimpleGetItemNotFound
from jedi.evaluate.utils import safe_property
from jedi.evaluate.cache import evaluator_as_method_param_cache
from jedi.common import BaseValueSet, BaseValue
from jedi.inference.helpers import SimpleGetItemNotFound
from jedi.inference.utils import safe_property
from jedi.inference.cache import inference_state_as_method_param_cache
from jedi.cache import memoize_method
_sentinel = object()
sentinel = object()
class HelperContextMixin(object):
class HelperValueMixin(object):
def get_root_context(self):
context = self
value = self
if value.parent_context is None:
return value.as_context()
while True:
if context.parent_context is None:
return context
context = context.parent_context
if value.parent_context is None:
return value
value = value.parent_context
@classmethod
@evaluator_as_method_param_cache()
@inference_state_as_method_param_cache()
def create_cached(cls, *args, **kwargs):
return cls(*args, **kwargs)
def execute(self, arguments):
return self.evaluator.execute(self, arguments=arguments)
return self.inference_state.execute(self, arguments=arguments)
def execute_evaluated(self, *value_list):
from jedi.evaluate.arguments import ValuesArguments
arguments = ValuesArguments([ContextSet([value]) for value in value_list])
return self.evaluator.execute(self, arguments)
def execute_with_values(self, *value_list):
from jedi.inference.arguments import ValuesArguments
arguments = ValuesArguments([ValueSet([value]) for value in value_list])
return self.inference_state.execute(self, arguments)
def execute_annotation(self):
return self.execute_evaluated()
return self.execute_with_values()
def gather_annotation_classes(self):
return ContextSet([self])
return ValueSet([self])
def merge_types_of_iterate(self, contextualized_node=None, is_async=False):
return ContextSet.from_sets(
lazy_context.infer()
for lazy_context in self.iterate(contextualized_node, is_async)
return ValueSet.from_sets(
lazy_value.infer()
for lazy_value in self.iterate(contextualized_node, is_async)
)
def _get_value_filters(self, name_or_str):
origin_scope = name_or_str if isinstance(name_or_str, Name) else None
for f in self.get_filters(origin_scope=origin_scope):
yield f
# This covers the case where a stub files are incomplete.
if self.is_stub():
from jedi.inference.gradual.conversion import convert_values
for c in convert_values(ValueSet({self})):
for f in c.get_filters():
yield f
def goto(self, name_or_str, name_context=None, analysis_errors=True):
if name_context is None:
name_context = self
from jedi.inference import finder
filters = self._get_value_filters(name_or_str)
names = finder.filter_name(filters, name_or_str)
debug.dbg('context.goto %s in (%s): %s', name_or_str, self, names)
return names
def py__getattribute__(self, name_or_str, name_context=None, position=None,
search_global=False, is_goto=False,
analysis_errors=True):
"""
:param position: Position of the last statement -> tuple of line, column
"""
if name_context is None:
name_context = self
from jedi.evaluate import finder
f = finder.NameFinder(self.evaluator, self, name_context, name_or_str,
position, analysis_errors=analysis_errors)
filters = f.get_filters(search_global)
if is_goto:
return f.filter_name(filters)
return f.find(filters, attribute_lookup=not search_global)
names = self.goto(name_or_str, name_context, analysis_errors)
values = ValueSet.from_sets(name.infer() for name in names)
if not values:
n = name_or_str.value if isinstance(name_or_str, Name) else name_or_str
values = self.py__getattribute__alternatives(n)
if not names and not values and analysis_errors:
if isinstance(name_or_str, Name):
from jedi.inference import analysis
analysis.add_attribute_error(
name_context, self, name_or_str)
debug.dbg('context.names_to_types: %s -> %s', names, values)
return values
def py__await__(self):
await_context_set = self.py__getattribute__(u"__await__")
if not await_context_set:
debug.warning('Tried to run __await__ on context %s', self)
return await_context_set.execute_evaluated()
def eval_node(self, node):
return self.evaluator.eval_element(self, node)
def create_context(self, node, node_is_context=False, node_is_object=False):
return self.evaluator.create_context(self, node, node_is_context, node_is_object)
await_value_set = self.py__getattribute__(u"__await__")
if not await_value_set:
debug.warning('Tried to run __await__ on value %s', self)
return await_value_set.execute_with_values()
def iterate(self, contextualized_node=None, is_async=False):
debug.dbg('iterate %s', self)
if is_async:
from jedi.evaluate.lazy_context import LazyKnownContexts
# TODO if no __aiter__ contexts are there, error should be:
from jedi.inference.lazy_value import LazyKnownValues
# TODO if no __aiter__ values are there, error should be:
# TypeError: 'async for' requires an object with __aiter__ method, got int
return iter([
LazyKnownContexts(
self.py__getattribute__('__aiter__').execute_evaluated()
.py__getattribute__('__anext__').execute_evaluated()
.py__getattribute__('__await__').execute_evaluated()
LazyKnownValues(
self.py__getattribute__('__aiter__').execute_with_values()
.py__getattribute__('__anext__').execute_with_values()
.py__getattribute__('__await__').execute_with_values()
.py__stop_iteration_returns()
) # noqa
])
return self.py__iter__(contextualized_node)
def is_sub_class_of(self, class_context):
def is_sub_class_of(self, class_value):
for cls in self.py__mro__():
if cls.is_same_class(class_context):
if cls.is_same_class(class_value):
return True
return False
def is_same_class(self, class2):
# Class matching should prefer comparisons that are not this function.
if type(class2).is_same_class != HelperContextMixin.is_same_class:
if type(class2).is_same_class != HelperValueMixin.is_same_class:
return class2.is_same_class(self)
return self == class2
@memoize_method
def as_context(self, *args, **kwargs):
return self._as_context(*args, **kwargs)
class Context(HelperContextMixin, BaseContext):
"""
Should be defined, otherwise the API returns empty types.
"""
predefined_names = {}
class Value(HelperValueMixin, BaseValue):
"""
To be defined by subclasses.
"""
@@ -128,20 +150,20 @@ class Context(HelperContextMixin, BaseContext):
# overwritten.
return self.__class__.__name__.lower()
def py__getitem__(self, index_context_set, contextualized_node):
from jedi.evaluate import analysis
# TODO this context is probably not right.
def py__getitem__(self, index_value_set, contextualized_node):
from jedi.inference import analysis
# TODO this value is probably not right.
analysis.add(
contextualized_node.context,
'type-error-not-subscriptable',
contextualized_node.node,
message="TypeError: '%s' object is not subscriptable" % self
)
return NO_CONTEXTS
return NO_VALUES
def py__iter__(self, contextualized_node=None):
if contextualized_node is not None:
from jedi.evaluate import analysis
from jedi.inference import analysis
analysis.add(
contextualized_node.context,
'type-error-not-iterable',
@@ -173,6 +195,9 @@ class Context(HelperContextMixin, BaseContext):
def is_bound_method(self):
return False
def is_builtins_module(self):
return False
def py__bool__(self):
"""
Since Wrapper is a super class for classes, functions and modules,
@@ -189,88 +214,94 @@ class Context(HelperContextMixin, BaseContext):
return clean_scope_docstring(self.tree_node)
return None
def get_safe_value(self, default=_sentinel):
if default is _sentinel:
raise ValueError("There exists no safe value for context %s" % self)
def get_safe_value(self, default=sentinel):
if default is sentinel:
raise ValueError("There exists no safe value for value %s" % self)
return default
def py__call__(self, arguments):
debug.warning("no execution possible %s", self)
return NO_CONTEXTS
return NO_VALUES
def py__stop_iteration_returns(self):
debug.warning("Not possible to return the stop iterations of %s", self)
return NO_CONTEXTS
return NO_VALUES
def py__getattribute__alternatives(self, name_or_str):
"""
For now a way to add values in cases like __getattr__.
"""
return NO_VALUES
def get_qualified_names(self):
# Returns Optional[Tuple[str, ...]]
return None
def is_stub(self):
# The root context knows if it's a stub or not.
# The root value knows if it's a stub or not.
return self.parent_context.is_stub()
def _as_context(self):
raise NotImplementedError('Not all values need to be converted to contexts')
def iterate_contexts(contexts, contextualized_node=None, is_async=False):
def iterate_values(values, contextualized_node=None, is_async=False):
"""
Calls `iterate`, on all contexts but ignores the ordering and just returns
all contexts that the iterate functions yield.
Calls `iterate`, on all values but ignores the ordering and just returns
all values that the iterate functions yield.
"""
return ContextSet.from_sets(
lazy_context.infer()
for lazy_context in contexts.iterate(contextualized_node, is_async=is_async)
return ValueSet.from_sets(
lazy_value.infer()
for lazy_value in values.iterate(contextualized_node, is_async=is_async)
)
class _ContextWrapperBase(HelperContextMixin):
predefined_names = {}
class _ValueWrapperBase(HelperValueMixin):
@safe_property
def name(self):
from jedi.evaluate.names import ContextName
wrapped_name = self._wrapped_context.name
from jedi.inference.names import ValueName
wrapped_name = self._wrapped_value.name
if wrapped_name.tree_name is not None:
return ContextName(self, wrapped_name.tree_name)
return ValueName(self, wrapped_name.tree_name)
else:
from jedi.evaluate.compiled import CompiledContextName
return CompiledContextName(self, wrapped_name.string_name)
from jedi.inference.compiled import CompiledValueName
return CompiledValueName(self, wrapped_name.string_name)
@classmethod
@evaluator_as_method_param_cache()
def create_cached(cls, evaluator, *args, **kwargs):
@inference_state_as_method_param_cache()
def create_cached(cls, inference_state, *args, **kwargs):
return cls(*args, **kwargs)
def __getattr__(self, name):
assert name != '_wrapped_context', 'Problem with _get_wrapped_context'
return getattr(self._wrapped_context, name)
assert name != '_wrapped_value', 'Problem with _get_wrapped_value'
return getattr(self._wrapped_value, name)
class LazyContextWrapper(_ContextWrapperBase):
class LazyValueWrapper(_ValueWrapperBase):
@safe_property
@memoize_method
def _wrapped_context(self):
with debug.increase_indent_cm('Resolve lazy context wrapper'):
return self._get_wrapped_context()
def _wrapped_value(self):
with debug.increase_indent_cm('Resolve lazy value wrapper'):
return self._get_wrapped_value()
def __repr__(self):
return '<%s>' % (self.__class__.__name__)
def _get_wrapped_context(self):
def _get_wrapped_value(self):
raise NotImplementedError
class ContextWrapper(_ContextWrapperBase):
def __init__(self, wrapped_context):
self._wrapped_context = wrapped_context
class ValueWrapper(_ValueWrapperBase):
def __init__(self, wrapped_value):
self._wrapped_value = wrapped_value
def __repr__(self):
return '%s(%s)' % (self.__class__.__name__, self._wrapped_context)
return '%s(%s)' % (self.__class__.__name__, self._wrapped_value)
class TreeContext(Context):
def __init__(self, evaluator, parent_context, tree_node):
super(TreeContext, self).__init__(evaluator, parent_context)
self.predefined_names = {}
class TreeValue(Value):
def __init__(self, inference_state, parent_context, tree_node):
super(TreeValue, self).__init__(inference_state, parent_context)
self.tree_node = tree_node
def __repr__(self):
@@ -286,7 +317,7 @@ class ContextualizedNode(object):
return self.context.get_root_context()
def infer(self):
return self.context.eval_node(self.node)
return self.context.infer_node(self.node)
def __repr__(self):
return '<%s: %s in %s>' % (self.__class__.__name__, self.node, self.context)
@@ -340,27 +371,16 @@ class ContextualizedName(ContextualizedNode):
return indexes
def _getitem(context, index_contexts, contextualized_node):
from jedi.evaluate.context.iterable import Slice
def _getitem(value, index_values, contextualized_node):
# The actual getitem call.
simple_getitem = getattr(context, 'py__simple_getitem__', None)
simple_getitem = getattr(value, 'py__simple_getitem__', None)
result = NO_CONTEXTS
unused_contexts = set()
for index_context in index_contexts:
result = NO_VALUES
unused_values = set()
for index_value in index_values:
if simple_getitem is not None:
index = index_context
if isinstance(index_context, Slice):
index = index.obj
try:
method = index.get_safe_value
except AttributeError:
pass
else:
index = method(default=None)
index = index_value.get_safe_value(default=None)
if type(index) in (float, int, str, unicode, slice, bytes):
try:
result |= simple_getitem(index)
@@ -368,69 +388,70 @@ def _getitem(context, index_contexts, contextualized_node):
except SimpleGetItemNotFound:
pass
unused_contexts.add(index_context)
unused_values.add(index_value)
# The index was somehow not good enough or simply a wrong type.
# Therefore we now iterate through all the contexts and just take
# Therefore we now iterate through all the values and just take
# all results.
if unused_contexts or not index_contexts:
result |= context.py__getitem__(
ContextSet(unused_contexts),
if unused_values or not index_values:
result |= value.py__getitem__(
ValueSet(unused_values),
contextualized_node
)
debug.dbg('py__getitem__ result: %s', result)
return result
class ContextSet(BaseContextSet):
class ValueSet(BaseValueSet):
def py__class__(self):
return ContextSet(c.py__class__() for c in self._set)
return ValueSet(c.py__class__() for c in self._set)
def iterate(self, contextualized_node=None, is_async=False):
from jedi.evaluate.lazy_context import get_merged_lazy_context
from jedi.inference.lazy_value import get_merged_lazy_value
type_iters = [c.iterate(contextualized_node, is_async=is_async) for c in self._set]
for lazy_contexts in zip_longest(*type_iters):
yield get_merged_lazy_context(
[l for l in lazy_contexts if l is not None]
for lazy_values in zip_longest(*type_iters):
yield get_merged_lazy_value(
[l for l in lazy_values if l is not None]
)
def execute(self, arguments):
return ContextSet.from_sets(c.evaluator.execute(c, arguments) for c in self._set)
return ValueSet.from_sets(c.inference_state.execute(c, arguments) for c in self._set)
def execute_evaluated(self, *args, **kwargs):
return ContextSet.from_sets(c.execute_evaluated(*args, **kwargs) for c in self._set)
def execute_with_values(self, *args, **kwargs):
return ValueSet.from_sets(c.execute_with_values(*args, **kwargs) for c in self._set)
def goto(self, *args, **kwargs):
return reduce(add, [c.goto(*args, **kwargs) for c in self._set], [])
def py__getattribute__(self, *args, **kwargs):
if kwargs.get('is_goto'):
return reduce(add, [c.py__getattribute__(*args, **kwargs) for c in self._set], [])
return ContextSet.from_sets(c.py__getattribute__(*args, **kwargs) for c in self._set)
return ValueSet.from_sets(c.py__getattribute__(*args, **kwargs) for c in self._set)
def get_item(self, *args, **kwargs):
return ContextSet.from_sets(_getitem(c, *args, **kwargs) for c in self._set)
return ValueSet.from_sets(_getitem(c, *args, **kwargs) for c in self._set)
def try_merge(self, function_name):
context_set = self.__class__([])
value_set = self.__class__([])
for c in self._set:
try:
method = getattr(c, function_name)
except AttributeError:
pass
else:
context_set |= method()
return context_set
value_set |= method()
return value_set
def gather_annotation_classes(self):
return ContextSet.from_sets([c.gather_annotation_classes() for c in self._set])
return ValueSet.from_sets([c.gather_annotation_classes() for c in self._set])
def get_signatures(self):
return [sig for c in self._set for sig in c.get_signatures()]
NO_CONTEXTS = ContextSet([])
NO_VALUES = ValueSet([])
def iterator_to_context_set(func):
def iterator_to_value_set(func):
def wrapper(*args, **kwargs):
return ContextSet(func(*args, **kwargs))
return ValueSet(func(*args, **kwargs))
return wrapper
jedi/evaluate/cache.py → jedi/inference/cache.py
+12 -12
View File
@@ -10,7 +10,7 @@ _NO_DEFAULT = object()
_RECURSION_SENTINEL = object()
def _memoize_default(default=_NO_DEFAULT, evaluator_is_first_arg=False, second_arg_is_evaluator=False):
def _memoize_default(default=_NO_DEFAULT, inference_state_is_first_arg=False, second_arg_is_inference_state=False):
""" This is a typical memoization decorator, BUT there is one difference:
To prevent recursion it sets defaults.
@@ -21,12 +21,12 @@ def _memoize_default(default=_NO_DEFAULT, evaluator_is_first_arg=False, second_a
def func(function):
def wrapper(obj, *args, **kwargs):
# TODO These checks are kind of ugly and slow.
if evaluator_is_first_arg:
if inference_state_is_first_arg:
cache = obj.memoize_cache
elif second_arg_is_evaluator:
elif second_arg_is_inference_state:
cache = args[0].memoize_cache # needed for meta classes
else:
cache = obj.evaluator.memoize_cache
cache = obj.inference_state.memoize_cache
try:
memo = cache[function]
@@ -47,23 +47,23 @@ def _memoize_default(default=_NO_DEFAULT, evaluator_is_first_arg=False, second_a
return func
def evaluator_function_cache(default=_NO_DEFAULT):
def inference_state_function_cache(default=_NO_DEFAULT):
def decorator(func):
return _memoize_default(default=default, evaluator_is_first_arg=True)(func)
return _memoize_default(default=default, inference_state_is_first_arg=True)(func)
return decorator
def evaluator_method_cache(default=_NO_DEFAULT):
def inference_state_method_cache(default=_NO_DEFAULT):
def decorator(func):
return _memoize_default(default=default)(func)
return decorator
def evaluator_as_method_param_cache():
def inference_state_as_method_param_cache():
def decorator(call):
return _memoize_default(second_arg_is_evaluator=True)(call)
return _memoize_default(second_arg_is_inference_state=True)(call)
return decorator
@@ -74,19 +74,19 @@ class CachedMetaClass(type):
class initializations. Either you do it this way or with decorators, but
with decorators you lose class access (isinstance, etc).
"""
@evaluator_as_method_param_cache()
@inference_state_as_method_param_cache()
def __call__(self, *args, **kwargs):
return super(CachedMetaClass, self).__call__(*args, **kwargs)
def evaluator_method_generator_cache():
def inference_state_method_generator_cache():
"""
This is a special memoizer. It memoizes generators and also checks for
recursion errors and returns no further iterator elemends in that case.
"""
def func(function):
def wrapper(obj, *args, **kwargs):
cache = obj.evaluator.memoize_cache
cache = obj.inference_state.memoize_cache
try:
memo = cache[function]
except KeyError:
jedi/evaluate/compiled/__init__.py → jedi/inference/compiled/__init__.py
+20 -20
View File
@@ -1,24 +1,24 @@
from jedi._compatibility import unicode
from jedi.evaluate.compiled.context import CompiledObject, CompiledName, \
CompiledObjectFilter, CompiledContextName, create_from_access_path
from jedi.evaluate.base_context import ContextWrapper, LazyContextWrapper
from jedi.inference.compiled.value import CompiledObject, CompiledName, \
CompiledObjectFilter, CompiledValueName, create_from_access_path
from jedi.inference.base_value import ValueWrapper, LazyValueWrapper
def builtin_from_name(evaluator, string):
typing_builtins_module = evaluator.builtins_module
def builtin_from_name(inference_state, string):
typing_builtins_module = inference_state.builtins_module
if string in ('None', 'True', 'False'):
builtins, = typing_builtins_module.non_stub_context_set
builtins, = typing_builtins_module.non_stub_value_set
filter_ = next(builtins.get_filters())
else:
filter_ = next(typing_builtins_module.get_filters())
name, = filter_.get(string)
context, = name.infer()
return context
value, = name.infer()
return value
class CompiledValue(LazyContextWrapper):
class CompiledValue(LazyValueWrapper):
def __init__(self, compiled_obj):
self.evaluator = compiled_obj.evaluator
self.inference_state = compiled_obj.inference_state
self._compiled_obj = compiled_obj
def __getattribute__(self, name):
@@ -27,38 +27,38 @@ class CompiledValue(LazyContextWrapper):
return getattr(self._compiled_obj, name)
return super(CompiledValue, self).__getattribute__(name)
def _get_wrapped_context(self):
def _get_wrapped_value(self):
instance, = builtin_from_name(
self.evaluator, self._compiled_obj.name.string_name).execute_evaluated()
self.inference_state, self._compiled_obj.name.string_name).execute_with_values()
return instance
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self._compiled_obj)
def create_simple_object(evaluator, obj):
def create_simple_object(inference_state, obj):
"""
Only allows creations of objects that are easily picklable across Python
versions.
"""
assert type(obj) in (int, float, str, bytes, unicode, slice, complex, bool), obj
compiled_obj = create_from_access_path(
evaluator,
evaluator.compiled_subprocess.create_simple_object(obj)
inference_state,
inference_state.compiled_subprocess.create_simple_object(obj)
)
return CompiledValue(compiled_obj)
def get_string_context_set(evaluator):
return builtin_from_name(evaluator, u'str').execute_evaluated()
def get_string_value_set(inference_state):
return builtin_from_name(inference_state, u'str').execute_with_values()
def load_module(evaluator, dotted_name, **kwargs):
def load_module(inference_state, dotted_name, **kwargs):
# Temporary, some tensorflow builtins cannot be loaded, so it's tried again
# and again and it's really slow.
if dotted_name.startswith('tensorflow.'):
return None
access_path = evaluator.compiled_subprocess.load_module(dotted_name=dotted_name, **kwargs)
access_path = inference_state.compiled_subprocess.load_module(dotted_name=dotted_name, **kwargs)
if access_path is None:
return None
return create_from_access_path(evaluator, access_path)
return create_from_access_path(inference_state, access_path)
jedi/evaluate/compiled/access.py → jedi/inference/compiled/access.py
+23 -17
View File
@@ -7,7 +7,7 @@ from collections import namedtuple
from jedi._compatibility import unicode, is_py3, builtins, \
py_version, force_unicode
from jedi.evaluate.compiled.getattr_static import getattr_static
from jedi.inference.compiled.getattr_static import getattr_static
ALLOWED_GETITEM_TYPES = (str, list, tuple, unicode, bytes, bytearray, dict)
@@ -109,8 +109,8 @@ def compiled_objects_cache(attribute_name):
Caching the id has the advantage that an object doesn't need to be
hashable.
"""
def wrapper(evaluator, obj, parent_context=None):
cache = getattr(evaluator, attribute_name)
def wrapper(inference_state, obj, parent_context=None):
cache = getattr(inference_state, attribute_name)
# Do a very cheap form of caching here.
key = id(obj)
try:
@@ -119,9 +119,9 @@ def compiled_objects_cache(attribute_name):
except KeyError:
# TODO wuaaaarrghhhhhhhh
if attribute_name == 'mixed_cache':
result = func(evaluator, obj, parent_context)
result = func(inference_state, obj, parent_context)
else:
result = func(evaluator, obj)
result = func(inference_state, obj)
# Need to cache all of them, otherwise the id could be overwritten.
cache[key] = result, obj, parent_context
return result
@@ -130,11 +130,11 @@ def compiled_objects_cache(attribute_name):
return decorator
def create_access(evaluator, obj):
return evaluator.compiled_subprocess.get_or_create_access_handle(obj)
def create_access(inference_state, obj):
return inference_state.compiled_subprocess.get_or_create_access_handle(obj)
def load_module(evaluator, dotted_name, sys_path):
def load_module(inference_state, dotted_name, sys_path):
temp, sys.path = sys.path, sys_path
try:
__import__(dotted_name)
@@ -154,7 +154,7 @@ def load_module(evaluator, dotted_name, sys_path):
# Just access the cache after import, because of #59 as well as the very
# complicated import structure of Python.
module = sys.modules[dotted_name]
return create_access_path(evaluator, module)
return create_access_path(inference_state, module)
class AccessPath(object):
@@ -171,8 +171,8 @@ class AccessPath(object):
self.accesses = value
def create_access_path(evaluator, obj):
access = create_access(evaluator, obj)
def create_access_path(inference_state, obj):
access = create_access(inference_state, obj)
return AccessPath(access.get_access_path_tuples())
@@ -193,18 +193,18 @@ def get_api_type(obj):
class DirectObjectAccess(object):
def __init__(self, evaluator, obj):
self._evaluator = evaluator
def __init__(self, inference_state, obj):
self._inference_state = inference_state
self._obj = obj
def __repr__(self):
return '%s(%s)' % (self.__class__.__name__, self.get_repr())
def _create_access(self, obj):
return create_access(self._evaluator, obj)
return create_access(self._inference_state, obj)
def _create_access_path(self, obj):
return create_access_path(self._evaluator, obj)
return create_access_path(self._inference_state, obj)
def py__bool__(self):
return bool(self._obj)
@@ -273,7 +273,13 @@ class DirectObjectAccess(object):
return [self._create_access_path(base) for base in self._obj.__bases__]
def py__path__(self):
return self._obj.__path__
paths = getattr(self._obj, '__path__', None)
# Avoid some weird hacks that would just fail, because they cannot be
# used by pickle.
if not isinstance(paths, list) \
or not all(isinstance(p, (bytes, unicode)) for p in paths):
return None
return paths
@_force_unicode_decorator
def get_repr(self):
@@ -376,7 +382,7 @@ class DirectObjectAccess(object):
return get_api_type(self._obj)
def get_access_path_tuples(self):
accesses = [create_access(self._evaluator, o) for o in self._get_objects_path()]
accesses = [create_access(self._inference_state, o) for o in self._get_objects_path()]
return [(access.py__name__(), access) for access in accesses]
def _get_objects_path(self):
jedi/evaluate/compiled/getattr_static.py → jedi/inference/compiled/getattr_static.py
View File
jedi/evaluate/compiled/mixed.py → jedi/inference/compiled/mixed.py
+73 -63
View File
@@ -9,23 +9,24 @@ import sys
from jedi.parser_utils import get_cached_code_lines
from jedi import settings
from jedi.evaluate import compiled
from jedi.inference import compiled
from jedi.cache import underscore_memoization
from jedi.file_io import FileIO
from jedi.evaluate.base_context import ContextSet, ContextWrapper
from jedi.evaluate.helpers import SimpleGetItemNotFound
from jedi.evaluate.context import ModuleContext
from jedi.evaluate.cache import evaluator_function_cache
from jedi.evaluate.compiled.getattr_static import getattr_static
from jedi.evaluate.compiled.access import compiled_objects_cache, \
from jedi.inference.base_value import ValueSet, ValueWrapper
from jedi.inference.helpers import SimpleGetItemNotFound
from jedi.inference.value import ModuleValue
from jedi.inference.cache import inference_state_function_cache
from jedi.inference.compiled.getattr_static import getattr_static
from jedi.inference.compiled.access import compiled_objects_cache, \
ALLOWED_GETITEM_TYPES, get_api_type
from jedi.evaluate.compiled.context import create_cached_compiled_object
from jedi.evaluate.gradual.conversion import to_stub
from jedi.inference.compiled.value import create_cached_compiled_object
from jedi.inference.gradual.conversion import to_stub
from jedi.inference.context import CompiledContext, TreeContextMixin
_sentinel = object()
class MixedObject(ContextWrapper):
class MixedObject(ValueWrapper):
"""
A ``MixedObject`` is used in two ways:
@@ -42,13 +43,13 @@ class MixedObject(ContextWrapper):
fewer special cases, because we in Python you don't have the same freedoms
to modify the runtime.
"""
def __init__(self, compiled_object, tree_context):
super(MixedObject, self).__init__(tree_context)
def __init__(self, compiled_object, tree_value):
super(MixedObject, self).__init__(tree_value)
self.compiled_object = compiled_object
self.access_handle = compiled_object.access_handle
def get_filters(self, *args, **kwargs):
yield MixedObjectFilter(self.evaluator, self)
yield MixedObjectFilter(self.inference_state, self)
def get_signatures(self):
# Prefer `inspect.signature` over somehow analyzing Python code. It
@@ -56,7 +57,7 @@ class MixedObject(ContextWrapper):
return self.compiled_object.get_signatures()
def py__call__(self, arguments):
return (to_stub(self._wrapped_context) or self._wrapped_context).py__call__(arguments)
return (to_stub(self._wrapped_value) or self._wrapped_value).py__call__(arguments)
def get_safe_value(self, default=_sentinel):
if default is _sentinel:
@@ -70,6 +71,9 @@ class MixedObject(ContextWrapper):
return self.compiled_object.py__simple_getitem__(index)
raise SimpleGetItemNotFound
def _as_context(self):
return MixedContext(self)
def __repr__(self):
return '<%s: %s>' % (
type(self).__name__,
@@ -77,40 +81,51 @@ class MixedObject(ContextWrapper):
)
class MixedContext(CompiledContext, TreeContextMixin):
@property
def compiled_object(self):
return self._value.compiled_object
class MixedName(compiled.CompiledName):
"""
The ``CompiledName._compiled_object`` is our MixedObject.
"""
@property
def start_pos(self):
contexts = list(self.infer())
if not contexts:
values = list(self.infer())
if not values:
# This means a start_pos that doesn't exist (compiled objects).
return 0, 0
return contexts[0].name.start_pos
@start_pos.setter
def start_pos(self, value):
# Ignore the __init__'s start_pos setter call.
pass
return values[0].name.start_pos
@underscore_memoization
def infer(self):
def access_to_value(parent_value, access):
if parent_value is None:
parent_context = None
else:
parent_context = parent_value.as_context()
if parent_context is None or isinstance(parent_context, MixedContext):
return _create(self._inference_state, access, parent_context=parent_context)
else:
return ValueSet({
create_cached_compiled_object(
parent_context.inference_state, access, parent_context
)
})
# TODO use logic from compiled.CompiledObjectFilter
access_paths = self.parent_context.access_handle.getattr_paths(
self.string_name,
default=None
)
assert len(access_paths)
contexts = [None]
values = [None]
for access in access_paths:
contexts = ContextSet.from_sets(
_create(self._evaluator, access, parent_context=c)
if c is None or isinstance(c, MixedObject)
else ContextSet({create_cached_compiled_object(c.evaluator, access, c)})
for c in contexts
)
return contexts
values = ValueSet.from_sets(access_to_value(v, access) for v in values)
return values
@property
def api_type(self):
@@ -121,9 +136,9 @@ class MixedObjectFilter(compiled.CompiledObjectFilter):
name_class = MixedName
@evaluator_function_cache()
def _load_module(evaluator, path):
module_node = evaluator.parse(
@inference_state_function_cache()
def _load_module(inference_state, path):
module_node = inference_state.parse(
path=path,
cache=True,
diff_cache=settings.fast_parser,
@@ -131,7 +146,7 @@ def _load_module(evaluator, path):
).get_root_node()
# python_module = inspect.getmodule(python_object)
# TODO we should actually make something like this possible.
#evaluator.modules[python_module.__name__] = module_node
#inference_state.modules[python_module.__name__] = module_node
return module_node
@@ -155,7 +170,7 @@ def _get_object_to_check(python_object):
raise TypeError # Prevents computation of `repr` within inspect.
def _find_syntax_node_name(evaluator, python_object):
def _find_syntax_node_name(inference_state, python_object):
original_object = python_object
try:
python_object = _get_object_to_check(python_object)
@@ -168,13 +183,13 @@ def _find_syntax_node_name(evaluator, python_object):
return None
file_io = FileIO(path)
module_node = _load_module(evaluator, path)
module_node = _load_module(inference_state, path)
if inspect.ismodule(python_object):
# We don't need to check names for modules, because there's not really
# a way to write a module in a module in Python (and also __name__ can
# be something like ``email.utils``).
code_lines = get_cached_code_lines(evaluator.grammar, path)
code_lines = get_cached_code_lines(inference_state.grammar, path)
return module_node, module_node, file_io, code_lines
try:
@@ -214,7 +229,7 @@ def _find_syntax_node_name(evaluator, python_object):
if line_names:
names = line_names
code_lines = get_cached_code_lines(evaluator.grammar, path)
code_lines = get_cached_code_lines(inference_state.grammar, path)
# It's really hard to actually get the right definition, here as a last
# resort we just return the last one. This chance might lead to odd
# completions at some points but will lead to mostly correct type
@@ -230,25 +245,26 @@ def _find_syntax_node_name(evaluator, python_object):
@compiled_objects_cache('mixed_cache')
def _create(evaluator, access_handle, parent_context, *args):
def _create(inference_state, access_handle, parent_context, *args):
compiled_object = create_cached_compiled_object(
evaluator,
inference_state,
access_handle,
parent_context=parent_context and parent_context.compiled_object
parent_context=None if parent_context is None
else parent_context.compiled_object.as_context() # noqa
)
# TODO accessing this is bad, but it probably doesn't matter that much,
# because we're working with interpreteters only here.
python_object = access_handle.access._obj
result = _find_syntax_node_name(evaluator, python_object)
result = _find_syntax_node_name(inference_state, python_object)
if result is None:
# TODO Care about generics from stuff like `[1]` and don't return like this.
if type(python_object) in (dict, list, tuple):
return ContextSet({compiled_object})
return ValueSet({compiled_object})
tree_contexts = to_stub(compiled_object)
if not tree_contexts:
return ContextSet({compiled_object})
tree_values = to_stub(compiled_object)
if not tree_values:
return ValueSet({compiled_object})
else:
module_node, tree_node, file_io, code_lines = result
@@ -256,36 +272,30 @@ def _create(evaluator, access_handle, parent_context, *args):
# TODO this __name__ is probably wrong.
name = compiled_object.get_root_context().py__name__()
string_names = tuple(name.split('.'))
module_context = ModuleContext(
evaluator, module_node,
module_context = ModuleValue(
inference_state, module_node,
file_io=file_io,
string_names=string_names,
code_lines=code_lines,
is_package=hasattr(compiled_object, 'py__path__'),
)
is_package=compiled_object.is_package,
).as_context()
if name is not None:
evaluator.module_cache.add(string_names, ContextSet([module_context]))
inference_state.module_cache.add(string_names, ValueSet([module_context]))
else:
if parent_context.tree_node.get_root_node() != module_node:
# This happens e.g. when __module__ is wrong, or when using
# TypeVar('foo'), where Jedi uses 'foo' as the name and
# Python's TypeVar('foo').__module__ will be typing.
return ContextSet({compiled_object})
return ValueSet({compiled_object})
module_context = parent_context.get_root_context()
tree_contexts = ContextSet({
module_context.create_context(
tree_node,
node_is_context=True,
node_is_object=True
)
})
tree_values = ValueSet({module_context.create_value(tree_node)})
if tree_node.type == 'classdef':
if not access_handle.is_class():
# Is an instance, not a class.
tree_contexts = tree_contexts.execute_evaluated()
tree_values = tree_values.execute_with_values()
return ContextSet(
MixedObject(compiled_object, tree_context=tree_context)
for tree_context in tree_contexts
return ValueSet(
MixedObject(compiled_object, tree_value=tree_value)
for tree_value in tree_values
)
jedi/evaluate/compiled/subprocess/__init__.py → jedi/inference/compiled/subprocess/__init__.py
+44 -44
View File
@@ -24,8 +24,8 @@ from jedi._compatibility import queue, is_py3, force_unicode, \
pickle_dump, pickle_load, GeneralizedPopen, weakref
from jedi import debug
from jedi.cache import memoize_method
from jedi.evaluate.compiled.subprocess import functions
from jedi.evaluate.compiled.access import DirectObjectAccess, AccessPath, \
from jedi.inference.compiled.subprocess import functions
from jedi.inference.compiled.access import DirectObjectAccess, AccessPath, \
SignatureParam
from jedi.api.exceptions import InternalError
@@ -70,10 +70,10 @@ def _cleanup_process(process, thread):
pass
class _EvaluatorProcess(object):
def __init__(self, evaluator):
self._evaluator_weakref = weakref.ref(evaluator)
self._evaluator_id = id(evaluator)
class _InferenceStateProcess(object):
def __init__(self, inference_state):
self._inference_state_weakref = weakref.ref(inference_state)
self._inference_state_id = id(inference_state)
self._handles = {}
def get_or_create_access_handle(self, obj):
@@ -81,7 +81,7 @@ class _EvaluatorProcess(object):
try:
return self.get_access_handle(id_)
except KeyError:
access = DirectObjectAccess(self._evaluator_weakref(), obj)
access = DirectObjectAccess(self._inference_state_weakref(), obj)
handle = AccessHandle(self, access, id_)
self.set_access_handle(handle)
return handle
@@ -93,19 +93,19 @@ class _EvaluatorProcess(object):
self._handles[handle.id] = handle
class EvaluatorSameProcess(_EvaluatorProcess):
class InferenceStateSameProcess(_InferenceStateProcess):
"""
Basically just an easy access to functions.py. It has the same API
as EvaluatorSubprocess and does the same thing without using a subprocess.
as InferenceStateSubprocess and does the same thing without using a subprocess.
This is necessary for the Interpreter process.
"""
def __getattr__(self, name):
return partial(_get_function(name), self._evaluator_weakref())
return partial(_get_function(name), self._inference_state_weakref())
class EvaluatorSubprocess(_EvaluatorProcess):
def __init__(self, evaluator, compiled_subprocess):
super(EvaluatorSubprocess, self).__init__(evaluator)
class InferenceStateSubprocess(_InferenceStateProcess):
def __init__(self, inference_state, compiled_subprocess):
super(InferenceStateSubprocess, self).__init__(inference_state)
self._used = False
self._compiled_subprocess = compiled_subprocess
@@ -116,7 +116,7 @@ class EvaluatorSubprocess(_EvaluatorProcess):
self._used = True
result = self._compiled_subprocess.run(
self._evaluator_weakref(),
self._inference_state_weakref(),
func,
args=args,
kwargs=kwargs,
@@ -148,7 +148,7 @@ class EvaluatorSubprocess(_EvaluatorProcess):
def __del__(self):
if self._used and not self._compiled_subprocess.is_crashed:
self._compiled_subprocess.delete_evaluator(self._evaluator_id)
self._compiled_subprocess.delete_inference_state(self._inference_state_id)
class CompiledSubprocess(object):
@@ -158,7 +158,7 @@ class CompiledSubprocess(object):
def __init__(self, executable):
self._executable = executable
self._evaluator_deletion_queue = queue.deque()
self._inference_state_deletion_queue = queue.deque()
self._cleanup_callable = lambda: None
def __repr__(self):
@@ -205,18 +205,18 @@ class CompiledSubprocess(object):
t)
return process
def run(self, evaluator, function, args=(), kwargs={}):
# Delete old evaluators.
def run(self, inference_state, function, args=(), kwargs={}):
# Delete old inference_states.
while True:
try:
evaluator_id = self._evaluator_deletion_queue.pop()
inference_state_id = self._inference_state_deletion_queue.pop()
except IndexError:
break
else:
self._send(evaluator_id, None)
self._send(inference_state_id, None)
assert callable(function)
return self._send(id(evaluator), function, args, kwargs)
return self._send(id(inference_state), function, args, kwargs)
def get_sys_path(self):
return self._send(None, functions.get_sys_path, (), {})
@@ -225,7 +225,7 @@ class CompiledSubprocess(object):
self.is_crashed = True
self._cleanup_callable()
def _send(self, evaluator_id, function, args=(), kwargs={}):
def _send(self, inference_state_id, function, args=(), kwargs={}):
if self.is_crashed:
raise InternalError("The subprocess %s has crashed." % self._executable)
@@ -233,7 +233,7 @@ class CompiledSubprocess(object):
# Python 2 compatibility
kwargs = {force_unicode(key): value for key, value in kwargs.items()}
data = evaluator_id, function, args, kwargs
data = inference_state_id, function, args, kwargs
try:
pickle_dump(data, self._get_process().stdin, self._pickle_protocol)
except (socket.error, IOError) as e:
@@ -272,59 +272,59 @@ class CompiledSubprocess(object):
raise result
return result
def delete_evaluator(self, evaluator_id):
def delete_inference_state(self, inference_state_id):
"""
Currently we are not deleting evalutors instantly. They only get
Currently we are not deleting inference_state instantly. They only get
deleted once the subprocess is used again. It would probably a better
solution to move all of this into a thread. However, the memory usage
of a single evaluator shouldn't be that high.
of a single inference_state shouldn't be that high.
"""
# With an argument - the evaluator gets deleted.
self._evaluator_deletion_queue.append(evaluator_id)
# With an argument - the inference_state gets deleted.
self._inference_state_deletion_queue.append(inference_state_id)
class Listener(object):
def __init__(self, pickle_protocol):
self._evaluators = {}
self._inference_states = {}
# TODO refactor so we don't need to process anymore just handle
# controlling.
self._process = _EvaluatorProcess(Listener)
self._process = _InferenceStateProcess(Listener)
self._pickle_protocol = pickle_protocol
def _get_evaluator(self, function, evaluator_id):
from jedi.evaluate import Evaluator
def _get_inference_state(self, function, inference_state_id):
from jedi.inference import InferenceState
try:
evaluator = self._evaluators[evaluator_id]
inference_state = self._inference_states[inference_state_id]
except KeyError:
from jedi.api.environment import InterpreterEnvironment
evaluator = Evaluator(
inference_state = InferenceState(
# The project is not actually needed. Nothing should need to
# access it.
project=None,
environment=InterpreterEnvironment()
)
self._evaluators[evaluator_id] = evaluator
return evaluator
self._inference_states[inference_state_id] = inference_state
return inference_state
def _run(self, evaluator_id, function, args, kwargs):
if evaluator_id is None:
def _run(self, inference_state_id, function, args, kwargs):
if inference_state_id is None:
return function(*args, **kwargs)
elif function is None:
del self._evaluators[evaluator_id]
del self._inference_states[inference_state_id]
else:
evaluator = self._get_evaluator(function, evaluator_id)
inference_state = self._get_inference_state(function, inference_state_id)
# Exchange all handles
args = list(args)
for i, arg in enumerate(args):
if isinstance(arg, AccessHandle):
args[i] = evaluator.compiled_subprocess.get_access_handle(arg.id)
args[i] = inference_state.compiled_subprocess.get_access_handle(arg.id)
for key, value in kwargs.items():
if isinstance(value, AccessHandle):
kwargs[key] = evaluator.compiled_subprocess.get_access_handle(value.id)
kwargs[key] = inference_state.compiled_subprocess.get_access_handle(value.id)
return function(evaluator, *args, **kwargs)
return function(inference_state, *args, **kwargs)
def listen(self):
stdout = sys.stdout
@@ -399,7 +399,7 @@ class AccessHandle(object):
@memoize_method
def _cached_results(self, name, *args, **kwargs):
#if type(self._subprocess) == EvaluatorSubprocess:
#if type(self._subprocess) == InferenceStateSubprocess:
#print(name, args, kwargs,
#self._subprocess.get_compiled_method_return(self.id, name, *args, **kwargs)
#)
jedi/evaluate/compiled/subprocess/__main__.py → jedi/inference/compiled/subprocess/__main__.py
+2 -2
View File
@@ -31,7 +31,7 @@ if sys.version_info > (3, 4):
# Try to import jedi/parso.
sys.meta_path.insert(0, _ExactImporter(_get_paths()))
from jedi.evaluate.compiled import subprocess # NOQA
from jedi.inference.compiled import subprocess # NOQA
sys.meta_path.pop(0)
else:
import imp
@@ -43,7 +43,7 @@ else:
load('parso')
load('jedi')
from jedi.evaluate.compiled import subprocess # NOQA
from jedi.inference.compiled import subprocess # NOQA
from jedi._compatibility import highest_pickle_protocol # noqa: E402
jedi/evaluate/compiled/subprocess/functions.py → jedi/inference/compiled/subprocess/functions.py
+13 -13
View File
@@ -4,7 +4,7 @@ import os
from jedi._compatibility import find_module, cast_path, force_unicode, \
iter_modules, all_suffixes
from jedi.evaluate.compiled import access
from jedi.inference.compiled import access
from jedi import parser_utils
@@ -12,20 +12,20 @@ def get_sys_path():
return list(map(cast_path, sys.path))
def load_module(evaluator, **kwargs):
return access.load_module(evaluator, **kwargs)
def load_module(inference_state, **kwargs):
return access.load_module(inference_state, **kwargs)
def get_compiled_method_return(evaluator, id, attribute, *args, **kwargs):
handle = evaluator.compiled_subprocess.get_access_handle(id)
def get_compiled_method_return(inference_state, id, attribute, *args, **kwargs):
handle = inference_state.compiled_subprocess.get_access_handle(id)
return getattr(handle.access, attribute)(*args, **kwargs)
def create_simple_object(evaluator, obj):
return access.create_access_path(evaluator, obj)
def create_simple_object(inference_state, obj):
return access.create_access_path(inference_state, obj)
def get_module_info(evaluator, sys_path=None, full_name=None, **kwargs):
def get_module_info(inference_state, sys_path=None, full_name=None, **kwargs):
"""
Returns Tuple[Union[NamespaceInfo, FileIO, None], Optional[bool]]
"""
@@ -40,25 +40,25 @@ def get_module_info(evaluator, sys_path=None, full_name=None, **kwargs):
sys.path = temp
def list_module_names(evaluator, search_path):
def list_module_names(inference_state, search_path):
return [
force_unicode(name)
for module_loader, name, is_pkg in iter_modules(search_path)
]
def get_builtin_module_names(evaluator):
def get_builtin_module_names(inference_state):
return list(map(force_unicode, sys.builtin_module_names))
def _test_raise_error(evaluator, exception_type):
def _test_raise_error(inference_state, exception_type):
"""
Raise an error to simulate certain problems for unit tests.
"""
raise exception_type
def _test_print(evaluator, stderr=None, stdout=None):
def _test_print(inference_state, stderr=None, stdout=None):
"""
Force some prints in the subprocesses. This exists for unit tests.
"""
@@ -82,5 +82,5 @@ def _get_init_path(directory_path):
return None
def safe_literal_eval(evaluator, value):
def safe_literal_eval(inference_state, value):
return parser_utils.safe_literal_eval(value)
jedi/evaluate/compiled/context.py → jedi/inference/compiled/value.py
+95 -84
View File
@@ -5,18 +5,19 @@ import re
from functools import partial
from jedi import debug
from jedi.evaluate.utils import to_list
from jedi.inference.utils import to_list
from jedi._compatibility import force_unicode, Parameter, cast_path
from jedi.cache import underscore_memoization, memoize_method
from jedi.evaluate.filters import AbstractFilter
from jedi.evaluate.names import AbstractNameDefinition, ContextNameMixin, \
from jedi.inference.filters import AbstractFilter
from jedi.inference.names import AbstractNameDefinition, ValueNameMixin, \
ParamNameInterface
from jedi.evaluate.base_context import Context, ContextSet, NO_CONTEXTS
from jedi.evaluate.lazy_context import LazyKnownContext
from jedi.evaluate.compiled.access import _sentinel
from jedi.evaluate.cache import evaluator_function_cache
from jedi.evaluate.helpers import reraise_getitem_errors
from jedi.evaluate.signature import BuiltinSignature
from jedi.inference.base_value import Value, ValueSet, NO_VALUES
from jedi.inference.lazy_value import LazyKnownValue
from jedi.inference.compiled.access import _sentinel
from jedi.inference.cache import inference_state_function_cache
from jedi.inference.helpers import reraise_getitem_errors
from jedi.inference.signature import BuiltinSignature
from jedi.inference.context import CompiledContext
class CheckAttribute(object):
@@ -40,16 +41,16 @@ class CheckAttribute(object):
return partial(self.func, instance)
class CompiledObject(Context):
def __init__(self, evaluator, access_handle, parent_context=None):
super(CompiledObject, self).__init__(evaluator, parent_context)
class CompiledObject(Value):
def __init__(self, inference_state, access_handle, parent_context=None):
super(CompiledObject, self).__init__(inference_state, parent_context)
self.access_handle = access_handle
def py__call__(self, arguments):
return_annotation = self.access_handle.get_return_annotation()
if return_annotation is not None:
# TODO the return annotation may also be a string.
return create_from_access_path(self.evaluator, return_annotation).execute_annotation()
return create_from_access_path(self.inference_state, return_annotation).execute_annotation()
try:
self.access_handle.getattr_paths(u'__call__')
@@ -57,34 +58,39 @@ class CompiledObject(Context):
return super(CompiledObject, self).py__call__(arguments)
else:
if self.access_handle.is_class():
from jedi.evaluate.context import CompiledInstance
return ContextSet([
CompiledInstance(self.evaluator, self.parent_context, self, arguments)
from jedi.inference.value import CompiledInstance
return ValueSet([
CompiledInstance(self.inference_state, self.parent_context, self, arguments)
])
else:
return ContextSet(self._execute_function(arguments))
return ValueSet(self._execute_function(arguments))
@CheckAttribute()
def py__class__(self):
return create_from_access_path(self.evaluator, self.access_handle.py__class__())
return create_from_access_path(self.inference_state, self.access_handle.py__class__())
@CheckAttribute()
def py__mro__(self):
return (self,) + tuple(
create_from_access_path(self.evaluator, access)
create_from_access_path(self.inference_state, access)
for access in self.access_handle.py__mro__accesses()
)
@CheckAttribute()
def py__bases__(self):
return tuple(
create_from_access_path(self.evaluator, access)
create_from_access_path(self.inference_state, access)
for access in self.access_handle.py__bases__()
)
@CheckAttribute()
def py__path__(self):
return map(cast_path, self.access_handle.py__path__())
paths = self.access_handle.py__path__()
if paths is None:
return None
return map(cast_path, paths)
def is_package(self):
return self.py__path__() is not None
@property
def string_names(self):
@@ -168,35 +174,30 @@ class CompiledObject(Context):
# Ensures that a CompiledObject is returned that is not an instance (like list)
return self
def get_filters(self, search_global=False, is_instance=False,
until_position=None, origin_scope=None):
def get_filters(self, is_instance=False, origin_scope=None):
yield self._ensure_one_filter(is_instance)
@memoize_method
def _ensure_one_filter(self, is_instance):
"""
search_global shouldn't change the fact that there's one dict, this way
there's only one `object`.
"""
return CompiledObjectFilter(self.evaluator, self, is_instance)
return CompiledObjectFilter(self.inference_state, self, is_instance)
@CheckAttribute(u'__getitem__')
def py__simple_getitem__(self, index):
with reraise_getitem_errors(IndexError, KeyError, TypeError):
access = self.access_handle.py__simple_getitem__(index)
if access is None:
return NO_CONTEXTS
return NO_VALUES
return ContextSet([create_from_access_path(self.evaluator, access)])
return ValueSet([create_from_access_path(self.inference_state, access)])
def py__getitem__(self, index_context_set, contextualized_node):
def py__getitem__(self, index_value_set, contextualized_node):
all_access_paths = self.access_handle.py__getitem__all_values()
if all_access_paths is None:
# This means basically that no __getitem__ has been defined on this
# object.
return super(CompiledObject, self).py__getitem__(index_context_set, contextualized_node)
return ContextSet(
create_from_access_path(self.evaluator, access)
return super(CompiledObject, self).py__getitem__(index_value_set, contextualized_node)
return ValueSet(
create_from_access_path(self.inference_state, access)
for access in all_access_paths
)
@@ -215,7 +216,7 @@ class CompiledObject(Context):
return
for access in access_path_list:
yield LazyKnownContext(create_from_access_path(self.evaluator, access))
yield LazyKnownValue(create_from_access_path(self.inference_state, access))
def py__name__(self):
return self.access_handle.py__name__()
@@ -225,11 +226,11 @@ class CompiledObject(Context):
name = self.py__name__()
if name is None:
name = self.access_handle.get_repr()
return CompiledContextName(self, name)
return CompiledValueName(self, name)
def _execute_function(self, params):
from jedi.evaluate import docstrings
from jedi.evaluate.compiled import builtin_from_name
from jedi.inference import docstrings
from jedi.inference.compiled import builtin_from_name
if self.api_type != 'function':
return
@@ -237,12 +238,12 @@ class CompiledObject(Context):
try:
# TODO wtf is this? this is exactly the same as the thing
# below. It uses getattr as well.
self.evaluator.builtins_module.access_handle.getattr_paths(name)
self.inference_state.builtins_module.access_handle.getattr_paths(name)
except AttributeError:
continue
else:
bltn_obj = builtin_from_name(self.evaluator, name)
for result in self.evaluator.execute(bltn_obj, params):
bltn_obj = builtin_from_name(self.inference_state, name)
for result in self.inference_state.execute(bltn_obj, params):
yield result
for type_ in docstrings.infer_return_types(self):
yield type_
@@ -257,20 +258,23 @@ class CompiledObject(Context):
def execute_operation(self, other, operator):
return create_from_access_path(
self.evaluator,
self.inference_state,
self.access_handle.execute_operation(other.access_handle, operator)
)
def negate(self):
return create_from_access_path(self.evaluator, self.access_handle.negate())
return create_from_access_path(self.inference_state, self.access_handle.negate())
def get_metaclasses(self):
return NO_CONTEXTS
return NO_VALUES
def _as_context(self):
return CompiledContext(self)
class CompiledName(AbstractNameDefinition):
def __init__(self, evaluator, parent_context, name):
self._evaluator = evaluator
def __init__(self, inference_state, parent_context, name):
self._inference_state = inference_state
self.parent_context = parent_context
self.string_name = name
@@ -295,8 +299,8 @@ class CompiledName(AbstractNameDefinition):
@underscore_memoization
def infer(self):
return ContextSet([_create_from_name(
self._evaluator, self.parent_context, self.string_name
return ValueSet([_create_from_name(
self._inference_state, self.parent_context, self.string_name
)])
@@ -322,14 +326,14 @@ class SignatureParamName(ParamNameInterface, AbstractNameDefinition):
def infer(self):
p = self._signature_param
evaluator = self.parent_context.evaluator
contexts = NO_CONTEXTS
inference_state = self.parent_context.inference_state
values = NO_VALUES
if p.has_default:
contexts = ContextSet([create_from_access_path(evaluator, p.default)])
values = ValueSet([create_from_access_path(inference_state, p.default)])
if p.has_annotation:
annotation = create_from_access_path(evaluator, p.annotation)
contexts |= annotation.execute_evaluated()
return contexts
annotation = create_from_access_path(inference_state, p.annotation)
values |= annotation.execute_with_values()
return values
class UnresolvableParamName(ParamNameInterface, AbstractNameDefinition):
@@ -348,14 +352,14 @@ class UnresolvableParamName(ParamNameInterface, AbstractNameDefinition):
return string
def infer(self):
return NO_CONTEXTS
return NO_VALUES
class CompiledContextName(ContextNameMixin, AbstractNameDefinition):
def __init__(self, context, name):
class CompiledValueName(ValueNameMixin, AbstractNameDefinition):
def __init__(self, value, name):
self.string_name = name
self._context = context
self.parent_context = context.parent_context
self._value = value
self.parent_context = value.parent_context
class EmptyCompiledName(AbstractNameDefinition):
@@ -364,19 +368,19 @@ class EmptyCompiledName(AbstractNameDefinition):
completions, just give Jedi the option to return this object. It infers to
nothing.
"""
def __init__(self, evaluator, name):
self.parent_context = evaluator.builtins_module
def __init__(self, inference_state, name):
self.parent_context = inference_state.builtins_module
self.string_name = name
def infer(self):
return NO_CONTEXTS
return NO_VALUES
class CompiledObjectFilter(AbstractFilter):
name_class = CompiledName
def __init__(self, evaluator, compiled_object, is_instance=False):
self._evaluator = evaluator
def __init__(self, inference_state, compiled_object, is_instance=False):
self._inference_state = inference_state
self.compiled_object = compiled_object
self.is_instance = is_instance
@@ -399,7 +403,7 @@ class CompiledObjectFilter(AbstractFilter):
# Always use unicode objects in Python 2 from here.
name = force_unicode(name)
if (is_descriptor and not self._evaluator.allow_descriptor_getattr) or not has_attribute:
if (is_descriptor and not self._inference_state.allow_descriptor_getattr) or not has_attribute:
return [self._get_cached_name(name, is_empty=True)]
if self.is_instance and name not in dir_callback():
@@ -409,12 +413,12 @@ class CompiledObjectFilter(AbstractFilter):
@memoize_method
def _get_cached_name(self, name, is_empty=False):
if is_empty:
return EmptyCompiledName(self._evaluator, name)
return EmptyCompiledName(self._inference_state, name)
else:
return self._create_name(name)
def values(self):
from jedi.evaluate.compiled import builtin_from_name
from jedi.inference.compiled import builtin_from_name
names = []
needs_type_completions, dir_infos = self.compiled_object.access_handle.get_dir_infos()
for name in dir_infos:
@@ -426,12 +430,12 @@ class CompiledObjectFilter(AbstractFilter):
# ``dir`` doesn't include the type names.
if not self.is_instance and needs_type_completions:
for filter in builtin_from_name(self._evaluator, u'type').get_filters():
for filter in builtin_from_name(self._inference_state, u'type').get_filters():
names += filter.values()
return names
def _create_name(self, name):
return self.name_class(self._evaluator, self.compiled_object, name)
return self.name_class(self._inference_state, self.compiled_object, name)
def __repr__(self):
return "<%s: %s>" % (self.__class__.__name__, self.compiled_object)
@@ -507,35 +511,42 @@ def _parse_function_doc(doc):
return param_str, ret
def _create_from_name(evaluator, compiled_object, name):
def _create_from_name(inference_state, compiled_object, name):
access_paths = compiled_object.access_handle.getattr_paths(name, default=None)
parent_context = compiled_object
if parent_context.is_class():
parent_context = parent_context.parent_context
context = None
value = None
for access_path in access_paths:
context = create_cached_compiled_object(
evaluator, access_path, parent_context=context
value = create_cached_compiled_object(
inference_state,
access_path,
parent_context=None if value is None else value.as_context(),
)
return context
return value
def _normalize_create_args(func):
"""The cache doesn't care about keyword vs. normal args."""
def wrapper(evaluator, obj, parent_context=None):
return func(evaluator, obj, parent_context)
def wrapper(inference_state, obj, parent_context=None):
return func(inference_state, obj, parent_context)
return wrapper
def create_from_access_path(evaluator, access_path):
parent_context = None
def create_from_access_path(inference_state, access_path):
value = None
for name, access in access_path.accesses:
parent_context = create_cached_compiled_object(evaluator, access, parent_context)
return parent_context
value = create_cached_compiled_object(
inference_state,
access,
parent_context=None if value is None else value.as_context()
)
return value
@_normalize_create_args
@evaluator_function_cache()
def create_cached_compiled_object(evaluator, access_handle, parent_context):
return CompiledObject(evaluator, access_handle, parent_context)
@inference_state_function_cache()
def create_cached_compiled_object(inference_state, access_handle, parent_context):
assert not isinstance(parent_context, CompiledObject)
return CompiledObject(inference_state, access_handle, parent_context)
jedi/inference/context.py
+457
View File
@@ -0,0 +1,457 @@
from abc import abstractmethod
from contextlib import contextmanager
from parso.tree import search_ancestor
from parso.python.tree import Name
from jedi.inference.filters import ParserTreeFilter, MergedFilter, \
GlobalNameFilter
from jedi.inference.base_value import NO_VALUES, ValueSet
from jedi.parser_utils import get_parent_scope
from jedi import debug
from jedi import parser_utils
class AbstractContext(object):
# Must be defined: inference_state and tree_node and parent_context as an attribute/property
def __init__(self, inference_state):
self.inference_state = inference_state
self.predefined_names = {}
@abstractmethod
def get_filters(self, until_position=None, origin_scope=None):
raise NotImplementedError
def goto(self, name_or_str, position):
from jedi.inference import finder
filters = _get_global_filters_for_name(
self, name_or_str if isinstance(name_or_str, Name) else None, position,
)
names = finder.filter_name(filters, name_or_str)
debug.dbg('context.goto %s in (%s): %s', name_or_str, self, names)
return names
def py__getattribute__(self, name_or_str, name_context=None, position=None,
analysis_errors=True):
"""
:param position: Position of the last statement -> tuple of line, column
"""
if name_context is None:
name_context = self
names = self.goto(name_or_str, position)
string_name = name_or_str.value if isinstance(name_or_str, Name) else name_or_str
# This paragraph is currently needed for proper branch type inference
# (static analysis).
found_predefined_types = None
if self.predefined_names and isinstance(name_or_str, Name):
node = name_or_str
while node is not None and not parser_utils.is_scope(node):
node = node.parent
if node.type in ("if_stmt", "for_stmt", "comp_for", 'sync_comp_for'):
try:
name_dict = self.predefined_names[node]
types = name_dict[string_name]
except KeyError:
continue
else:
found_predefined_types = types
break
if found_predefined_types is not None and names:
from jedi.inference import flow_analysis
check = flow_analysis.reachability_check(
context=self,
value_scope=self.tree_node,
node=name_or_str,
)
if check is flow_analysis.UNREACHABLE:
values = NO_VALUES
else:
values = found_predefined_types
else:
values = ValueSet.from_sets(name.infer() for name in names)
if not names and not values and analysis_errors:
if isinstance(name_or_str, Name):
from jedi.inference import analysis
message = ("NameError: name '%s' is not defined." % string_name)
analysis.add(name_context, 'name-error', name_or_str, message)
debug.dbg('context.names_to_types: %s -> %s', names, values)
if values:
return values
return self._check_for_additional_knowledge(name_or_str, name_context, position)
def _check_for_additional_knowledge(self, name_or_str, name_context, position):
name_context = name_context or self
# Add isinstance and other if/assert knowledge.
if isinstance(name_or_str, Name) and not name_context.is_instance():
flow_scope = name_or_str
base_nodes = [name_context.tree_node]
if any(b.type in ('comp_for', 'sync_comp_for') for b in base_nodes):
return NO_VALUES
from jedi.inference.finder import check_flow_information
while True:
flow_scope = get_parent_scope(flow_scope, include_flows=True)
n = check_flow_information(name_context, flow_scope,
name_or_str, position)
if n is not None:
return n
if flow_scope in base_nodes:
break
return NO_VALUES
def get_root_context(self):
parent_context = self.parent_context
if parent_context is None:
return self
return parent_context.get_root_context()
def is_module(self):
return False
def is_builtins_module(self):
return False
def is_class(self):
return False
def is_stub(self):
return False
def is_instance(self):
return False
def is_compiled(self):
return False
@abstractmethod
def py__name__(self):
raise NotImplementedError
@property
def name(self):
return None
def get_qualified_names(self):
return ()
def py__doc__(self):
return ''
@contextmanager
def predefine_names(self, flow_scope, dct):
predefined = self.predefined_names
predefined[flow_scope] = dct
try:
yield
finally:
del predefined[flow_scope]
def __repr__(self):
return '%s(%s)' % (self.__class__.__name__, self._value)
class ValueContext(AbstractContext):
"""
Should be defined, otherwise the API returns empty types.
"""
def __init__(self, value):
super(ValueContext, self).__init__(value.inference_state)
self._value = value
@property
def tree_node(self):
return self._value.tree_node
@property
def parent_context(self):
return self._value.parent_context
def is_module(self):
return self._value.is_module()
def is_builtins_module(self):
return self._value == self.inference_state.builtins_module
def is_class(self):
return self._value.is_class()
def is_stub(self):
return self._value.is_stub()
def is_instance(self):
return self._value.is_instance()
def is_compiled(self):
return self._value.is_compiled()
def py__name__(self):
return self._value.py__name__()
@property
def name(self):
return self._value.name
def get_qualified_names(self):
return self._value.get_qualified_names()
def py__doc__(self):
return self._value.py__doc__()
def __repr__(self):
return '%s(%s)' % (self.__class__.__name__, self._value)
class TreeContextMixin(object):
def infer_node(self, node):
return self.inference_state.infer_element(self, node)
def create_value(self, node):
from jedi.inference import value
if node == self.tree_node:
assert self.is_module()
return self.get_value()
parent_context = self.create_context(node)
if node.type in ('funcdef', 'lambdef'):
func = value.FunctionValue.from_context(parent_context, node)
if parent_context.is_class():
class_value = parent_context.parent_context.create_value(parent_context.tree_node)
instance = value.AnonymousInstance(
self.inference_state, parent_context.parent_context, class_value)
func = value.BoundMethod(
instance=instance,
function=func
)
return func
elif node.type == 'classdef':
return value.ClassValue(self.inference_state, parent_context, node)
else:
raise NotImplementedError("Probably shouldn't happen: %s" % node)
def create_context(self, node):
def from_scope_node(scope_node, is_nested=True):
if scope_node == self.tree_node:
return self
if scope_node.type in ('funcdef', 'lambdef', 'classdef'):
return self.create_value(scope_node).as_context()
elif scope_node.type in ('comp_for', 'sync_comp_for'):
parent_scope = parser_utils.get_parent_scope(scope_node)
parent_context = from_scope_node(parent_scope)
if node.start_pos >= scope_node.children[-1].start_pos:
return parent_context
return CompForContext(parent_context, scope_node)
raise Exception("There's a scope that was not managed: %s" % scope_node)
def parent_scope(node):
while True:
node = node.parent
if parser_utils.is_scope(node):
return node
elif node.type in ('argument', 'testlist_comp'):
if node.children[1].type in ('comp_for', 'sync_comp_for'):
return node.children[1]
elif node.type == 'dictorsetmaker':
for n in node.children[1:4]:
# In dictionaries it can be pretty much anything.
if n.type in ('comp_for', 'sync_comp_for'):
return n
scope_node = parent_scope(node)
if scope_node.type in ('funcdef', 'classdef'):
colon = scope_node.children[scope_node.children.index(':')]
if node.start_pos < colon.start_pos:
parent = node.parent
if not (parent.type == 'param' and parent.name == node):
scope_node = parent_scope(scope_node)
return from_scope_node(scope_node, is_nested=True)
class FunctionContext(TreeContextMixin, ValueContext):
def get_filters(self, until_position=None, origin_scope=None):
yield ParserTreeFilter(
self.inference_state,
parent_context=self,
until_position=until_position,
origin_scope=origin_scope
)
class ModuleContext(TreeContextMixin, ValueContext):
def py__file__(self):
return self._value.py__file__()
@property
def py__package__(self):
return self._value.py__package__
@property
def is_package(self):
return self._value.is_package
def get_filters(self, until_position=None, origin_scope=None):
filters = self._value.get_filters(origin_scope)
# Skip the first filter and replace it.
next(filters)
yield MergedFilter(
ParserTreeFilter(
parent_context=self,
until_position=until_position,
origin_scope=origin_scope
),
GlobalNameFilter(self, self.tree_node),
)
for f in filters: # Python 2...
yield f
@property
def string_names(self):
return self._value.string_names
@property
def code_lines(self):
return self._value.code_lines
def get_value(self):
"""
This is the only function that converts a context back to a value.
This is necessary for stub -> python conversion and vice versa. However
this method shouldn't be moved to AbstractContext.
"""
return self._value
class NamespaceContext(TreeContextMixin, ValueContext):
def get_filters(self, until_position=None, origin_scope=None):
return self._value.get_filters()
def py__file__(self):
return self._value.py__file__()
class ClassContext(TreeContextMixin, ValueContext):
def get_filters(self, until_position=None, origin_scope=None):
yield self.get_global_filter(until_position, origin_scope)
def get_global_filter(self, until_position=None, origin_scope=None):
return ParserTreeFilter(
parent_context=self,
until_position=until_position,
origin_scope=origin_scope
)
class CompForContext(TreeContextMixin, AbstractContext):
def __init__(self, parent_context, comp_for):
super(CompForContext, self).__init__(parent_context.inference_state)
self.tree_node = comp_for
self.parent_context = parent_context
def get_filters(self, until_position=None, origin_scope=None):
yield ParserTreeFilter(self)
class CompiledContext(ValueContext):
def get_filters(self, until_position=None, origin_scope=None):
return self._value.get_filters()
def get_value(self):
return self._value
def py__file__(self):
return self._value.py__file__()
def _get_global_filters_for_name(context, name_or_none, position):
# For functions and classes the defaults don't belong to the
# function and get inferred in the value before the function. So
# make sure to exclude the function/class name.
if name_or_none is not None:
ancestor = search_ancestor(name_or_none, 'funcdef', 'classdef', 'lambdef')
lambdef = None
if ancestor == 'lambdef':
# For lambdas it's even more complicated since parts will
# be inferred later.
lambdef = ancestor
ancestor = search_ancestor(name_or_none, 'funcdef', 'classdef')
if ancestor is not None:
colon = ancestor.children[-2]
if position is not None and position < colon.start_pos:
if lambdef is None or position < lambdef.children[-2].start_pos:
position = ancestor.start_pos
return get_global_filters(context, position, name_or_none)
def get_global_filters(context, until_position, origin_scope):
"""
Returns all filters in order of priority for name resolution.
For global name lookups. The filters will handle name resolution
themselves, but here we gather possible filters downwards.
>>> from jedi._compatibility import u, no_unicode_pprint
>>> from jedi import Script
>>> script = Script(u('''
... x = ['a', 'b', 'c']
... def func():
... y = None
... '''))
>>> module_node = script._module_node
>>> scope = next(module_node.iter_funcdefs())
>>> scope
<Function: func@3-5>
>>> context = script._get_module_context().create_context(scope)
>>> filters = list(get_global_filters(context, (4, 0), None))
First we get the names from the function scope.
>>> no_unicode_pprint(filters[0]) # doctest: +ELLIPSIS
MergedFilter(<ParserTreeFilter: ...>, <GlobalNameFilter: ...>)
>>> sorted(str(n) for n in filters[0].values()) # doctest: +NORMALIZE_WHITESPACE
['<TreeNameDefinition: string_name=func start_pos=(3, 4)>',
'<TreeNameDefinition: string_name=x start_pos=(2, 0)>']
>>> filters[0]._filters[0]._until_position
(4, 0)
>>> filters[0]._filters[1]._until_position
Then it yields the names from one level "lower". In this example, this is
the module scope (including globals).
As a side note, you can see, that the position in the filter is None on the
globals filter, because there the whole module is searched.
>>> list(filters[1].values()) # package modules -> Also empty.
[]
>>> sorted(name.string_name for name in filters[2].values()) # Module attributes
['__doc__', '__name__', '__package__']
Finally, it yields the builtin filter, if `include_builtin` is
true (default).
>>> list(filters[3].values()) # doctest: +ELLIPSIS
[...]
"""
base_context = context
from jedi.inference.value.function import FunctionExecutionContext
while context is not None:
# Names in methods cannot be resolved within the class.
for filter in context.get_filters(
until_position=until_position,
origin_scope=origin_scope):
yield filter
if isinstance(context, FunctionExecutionContext):
# The position should be reset if the current scope is a function.
until_position = None
context = context.parent_context
# Add builtins to the global scope.
yield next(base_context.inference_state.builtins_module.get_filters())
jedi/evaluate/docstrings.py → jedi/inference/docstrings.py
+41 -41
View File
@@ -1,6 +1,6 @@
"""
Docstrings are another source of information for functions and classes.
:mod:`jedi.evaluate.dynamic` tries to find all executions of functions, while
:mod:`jedi.inference.dynamic` tries to find all executions of functions, while
the docstring parsing is much easier. There are three different types of
docstrings that |jedi| understands:
@@ -23,11 +23,11 @@ from parso import parse, ParserSyntaxError
from jedi._compatibility import u
from jedi import debug
from jedi.evaluate.utils import indent_block
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.base_context import iterator_to_context_set, ContextSet, \
NO_CONTEXTS
from jedi.evaluate.lazy_context import LazyKnownContexts
from jedi.inference.utils import indent_block
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.base_value import iterator_to_value_set, ValueSet, \
NO_VALUES
from jedi.inference.lazy_value import LazyKnownValues
DOCSTRING_PARAM_PATTERNS = [
@@ -183,7 +183,7 @@ def _strip_rst_role(type_str):
return type_str
def _evaluate_for_statement_string(module_context, string):
def _infer_for_statement_string(module_context, string):
code = dedent(u("""
def pseudo_docstring_stuff():
'''
@@ -205,7 +205,7 @@ def _evaluate_for_statement_string(module_context, string):
# will be impossible to use `...` (Ellipsis) as a token. Docstring types
# don't need to conform with the current grammar.
debug.dbg('Parse docstring code %s', string, color='BLUE')
grammar = module_context.evaluator.latest_grammar
grammar = module_context.inference_state.latest_grammar
try:
module = grammar.parse(code.format(indent_block(string)), error_recovery=False)
except ParserSyntaxError:
@@ -221,13 +221,13 @@ def _evaluate_for_statement_string(module_context, string):
if stmt.type not in ('name', 'atom', 'atom_expr'):
return []
from jedi.evaluate.context import FunctionContext
function_context = FunctionContext(
module_context.evaluator,
from jedi.inference.value import FunctionValue
function_value = FunctionValue(
module_context.inference_state,
module_context,
funcdef
)
func_execution_context = function_context.get_function_execution()
func_execution_context = function_value.as_context()
# Use the module of the param.
# TODO this module is not the module of the param in case of a function
# call. In that case it's the module of the function call.
@@ -241,62 +241,62 @@ def _execute_types_in_stmt(module_context, stmt):
doesn't include tuple, list and dict literals, because the stuff they
contain is executed. (Used as type information).
"""
definitions = module_context.eval_node(stmt)
return ContextSet.from_sets(
_execute_array_values(module_context.evaluator, d)
definitions = module_context.infer_node(stmt)
return ValueSet.from_sets(
_execute_array_values(module_context.inference_state, d)
for d in definitions
)
def _execute_array_values(evaluator, array):
def _execute_array_values(inference_state, array):
"""
Tuples indicate that there's not just one return value, but the listed
ones. `(str, int)` means that it returns a tuple with both types.
"""
from jedi.evaluate.context.iterable import SequenceLiteralContext, FakeSequence
if isinstance(array, SequenceLiteralContext):
from jedi.inference.value.iterable import SequenceLiteralValue, FakeSequence
if isinstance(array, SequenceLiteralValue):
values = []
for lazy_context in array.py__iter__():
objects = ContextSet.from_sets(
_execute_array_values(evaluator, typ)
for typ in lazy_context.infer()
for lazy_value in array.py__iter__():
objects = ValueSet.from_sets(
_execute_array_values(inference_state, typ)
for typ in lazy_value.infer()
)
values.append(LazyKnownContexts(objects))
return {FakeSequence(evaluator, array.array_type, values)}
values.append(LazyKnownValues(objects))
return {FakeSequence(inference_state, array.array_type, values)}
else:
return array.execute_annotation()
@evaluator_method_cache()
@inference_state_method_cache()
def infer_param(execution_context, param):
from jedi.evaluate.context.instance import InstanceArguments
from jedi.evaluate.context import FunctionExecutionContext
from jedi.inference.value.instance import InstanceArguments
from jedi.inference.value import FunctionExecutionContext
def eval_docstring(docstring):
return ContextSet(
def infer_docstring(docstring):
return ValueSet(
p
for param_str in _search_param_in_docstr(docstring, param.name.value)
for p in _evaluate_for_statement_string(module_context, param_str)
for p in _infer_for_statement_string(module_context, param_str)
)
module_context = execution_context.get_root_context()
func = param.get_parent_function()
if func.type == 'lambdef':
return NO_CONTEXTS
return NO_VALUES
types = eval_docstring(execution_context.py__doc__())
types = infer_docstring(execution_context.py__doc__())
if isinstance(execution_context, FunctionExecutionContext) \
and isinstance(execution_context.var_args, InstanceArguments) \
and execution_context.function_context.py__name__() == '__init__':
class_context = execution_context.var_args.instance.class_context
types |= eval_docstring(class_context.py__doc__())
and execution_context.function_value.py__name__() == '__init__':
class_value = execution_context.var_args.instance.class_value
types |= infer_docstring(class_value.py__doc__())
debug.dbg('Found param types for docstring: %s', types, color='BLUE')
return types
@evaluator_method_cache()
@iterator_to_context_set
def infer_return_types(function_context):
@inference_state_method_cache()
@iterator_to_value_set
def infer_return_types(function_value):
def search_return_in_docstr(code):
for p in DOCSTRING_RETURN_PATTERNS:
match = p.search(code)
@@ -306,6 +306,6 @@ def infer_return_types(function_context):
for type_ in _search_return_in_numpydocstr(code):
yield type_
for type_str in search_return_in_docstr(function_context.py__doc__()):
for type_eval in _evaluate_for_statement_string(function_context.get_root_context(), type_str):
yield type_eval
for type_str in search_return_in_docstr(function_value.py__doc__()):
for value in _infer_for_statement_string(function_value.get_root_context(), type_str):
yield value
jedi/evaluate/dynamic.py → jedi/inference/dynamic.py
+54 -54
View File
@@ -19,42 +19,44 @@ It works as follows:
from jedi import settings
from jedi import debug
from jedi.evaluate.cache import evaluator_function_cache
from jedi.evaluate import imports
from jedi.evaluate.arguments import TreeArguments
from jedi.evaluate.param import create_default_params
from jedi.evaluate.helpers import is_stdlib_path
from jedi.evaluate.utils import to_list
from jedi.inference.cache import inference_state_function_cache
from jedi.inference import imports
from jedi.inference.arguments import TreeArguments
from jedi.inference.param import create_default_params
from jedi.inference.helpers import is_stdlib_path
from jedi.inference.utils import to_list
from jedi.parser_utils import get_parent_scope
from jedi.evaluate.context import ModuleContext, instance
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS
from jedi.evaluate import recursion
from jedi.inference.value import instance
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.inference import recursion
from jedi.inference.names import ParamNameWrapper
MAX_PARAM_SEARCHES = 20
class DynamicExecutedParams(object):
class DynamicExecutedParamName(ParamNameWrapper):
"""
Simulates being a parameter while actually just being multiple params.
"""
def __init__(self, evaluator, executed_params):
self.evaluator = evaluator
self._executed_params = executed_params
def __init__(self, executed_param_names):
super(DynamicExecutedParamName, self).__init__(executed_param_names[0])
self._executed_param_names = executed_param_names
def infer(self):
with recursion.execution_allowed(self.evaluator, self) as allowed:
inf = self.parent_context.inference_state
with recursion.execution_allowed(inf, self) as allowed:
# We need to catch recursions that may occur, because an
# anonymous functions can create an anonymous parameter that is
# more or less self referencing.
if allowed:
return ContextSet.from_sets(p.infer() for p in self._executed_params)
return NO_CONTEXTS
return ValueSet.from_sets(p.infer() for p in self._executed_param_names)
return NO_VALUES
@debug.increase_indent
def search_params(evaluator, execution_context, funcdef):
def search_param_names(inference_state, execution_context, funcdef):
"""
A dynamic search for param values. If you try to complete a type:
@@ -70,7 +72,7 @@ def search_params(evaluator, execution_context, funcdef):
if not settings.dynamic_params:
return create_default_params(execution_context, funcdef)
evaluator.dynamic_params_depth += 1
inference_state.dynamic_params_depth += 1
try:
path = execution_context.get_root_context().py__file__()
if path is not None and is_stdlib_path(path):
@@ -91,31 +93,30 @@ def search_params(evaluator, execution_context, funcdef):
try:
module_context = execution_context.get_root_context()
function_executions = _search_function_executions(
evaluator,
inference_state,
module_context,
funcdef,
string_name=string_name,
)
if function_executions:
zipped_params = zip(*list(
function_execution.get_executed_params_and_issues()[0]
zipped_param_names = zip(*list(
function_execution.get_executed_param_names_and_issues()[0]
for function_execution in function_executions
))
params = [DynamicExecutedParams(evaluator, executed_params)
for executed_params in zipped_params]
# Evaluate the ExecutedParams to types.
params = [DynamicExecutedParamName(executed_param_names)
for executed_param_names in zipped_param_names]
else:
return create_default_params(execution_context, funcdef)
finally:
debug.dbg('Dynamic param result finished', color='MAGENTA')
return params
finally:
evaluator.dynamic_params_depth -= 1
inference_state.dynamic_params_depth -= 1
@evaluator_function_cache(default=None)
@inference_state_function_cache(default=None)
@to_list
def _search_function_executions(evaluator, module_context, funcdef, string_name):
def _search_function_executions(inference_state, module_context, funcdef, string_name):
"""
Returns a list of param names.
"""
@@ -128,22 +129,20 @@ def _search_function_executions(evaluator, module_context, funcdef, string_name)
found_executions = False
i = 0
for for_mod_context in imports.get_modules_containing_name(
evaluator, [module_context], string_name):
if not isinstance(module_context, ModuleContext):
return
for name, trailer in _get_possible_nodes(for_mod_context, string_name):
for for_mod_context in imports.get_module_contexts_containing_name(
inference_state, [module_context], string_name):
for name, trailer in _get_potential_nodes(for_mod_context, string_name):
i += 1
# This is a simple way to stop Jedi's dynamic param recursion
# from going wild: The deeper Jedi's in the recursion, the less
# code should be evaluated.
if i * evaluator.dynamic_params_depth > MAX_PARAM_SEARCHES:
# code should be inferred.
if i * inference_state.dynamic_params_depth > MAX_PARAM_SEARCHES:
return
random_context = evaluator.create_context(for_mod_context, name)
random_context = for_mod_context.create_context(name)
for function_execution in _check_name_for_execution(
evaluator, random_context, compare_node, name, trailer):
inference_state, random_context, compare_node, name, trailer):
found_executions = True
yield function_execution
@@ -165,9 +164,9 @@ def _get_lambda_name(node):
return None
def _get_possible_nodes(module_context, func_string_name):
def _get_potential_nodes(module_value, func_string_name):
try:
names = module_context.tree_node.get_used_names()[func_string_name]
names = module_value.tree_node.get_used_names()[func_string_name]
except KeyError:
return
@@ -178,17 +177,17 @@ def _get_possible_nodes(module_context, func_string_name):
yield name, trailer
def _check_name_for_execution(evaluator, context, compare_node, name, trailer):
from jedi.evaluate.context.function import FunctionExecutionContext
def _check_name_for_execution(inference_state, context, compare_node, name, trailer):
from jedi.inference.value.function import FunctionExecutionContext
def create_func_excs():
def create_func_excs(value):
arglist = trailer.children[1]
if arglist == ')':
arglist = None
args = TreeArguments(evaluator, context, arglist, trailer)
if value_node.type == 'classdef':
args = TreeArguments(inference_state, context, arglist, trailer)
if value.tree_node.type == 'classdef':
created_instance = instance.TreeInstance(
evaluator,
inference_state,
value.parent_context,
value,
args
@@ -196,32 +195,33 @@ def _check_name_for_execution(evaluator, context, compare_node, name, trailer):
for execution in created_instance.create_init_executions():
yield execution
else:
yield value.get_function_execution(args)
yield value.as_context(args)
for value in evaluator.goto_definitions(context, name):
for value in inference_state.goto_definitions(context, name):
value_node = value.tree_node
if compare_node == value_node:
for func_execution in create_func_excs():
for func_execution in create_func_excs(value):
yield func_execution
elif isinstance(value.parent_context, FunctionExecutionContext) and \
compare_node.type == 'funcdef':
# Here we're trying to find decorators by checking the first
# parameter. It's not very generic though. Should find a better
# solution that also applies to nested decorators.
params, _ = value.parent_context.get_executed_params_and_issues()
if len(params) != 1:
param_names, _ = value.parent_context.get_executed_param_names_and_issues()
if len(param_names) != 1:
continue
values = params[0].infer()
values = param_names[0].infer()
nodes = [v.tree_node for v in values]
if nodes == [compare_node]:
# Found a decorator.
module_context = context.get_root_context()
execution_context = next(create_func_excs())
for name, trailer in _get_possible_nodes(module_context, params[0].string_name):
execution_context = next(create_func_excs(value))
potential_nodes = _get_potential_nodes(module_context, param_names[0].string_name)
for name, trailer in potential_nodes:
if value_node.start_pos < name.start_pos < value_node.end_pos:
random_context = evaluator.create_context(execution_context, name)
random_context = execution_context.create_context(name)
iterator = _check_name_for_execution(
evaluator,
inference_state,
random_context,
compare_node,
name,
jedi/evaluate/filters.py → jedi/inference/filters.py
+42 -113
View File
@@ -8,12 +8,12 @@ import weakref
from parso.tree import search_ancestor
from jedi._compatibility import use_metaclass
from jedi.evaluate import flow_analysis
from jedi.evaluate.base_context import ContextSet, Context, ContextWrapper, \
LazyContextWrapper
from jedi.inference import flow_analysis
from jedi.inference.base_value import ValueSet, Value, ValueWrapper, \
LazyValueWrapper
from jedi.parser_utils import get_cached_parent_scope
from jedi.evaluate.utils import to_list
from jedi.evaluate.names import TreeNameDefinition, ParamName, AbstractNameDefinition
from jedi.inference.utils import to_list
from jedi.inference.names import TreeNameDefinition, ParamName, AbstractNameDefinition
_definition_name_cache = weakref.WeakKeyDictionary()
@@ -68,11 +68,11 @@ def _get_definition_names(used_names, name_key):
class AbstractUsedNamesFilter(AbstractFilter):
name_class = TreeNameDefinition
def __init__(self, context, parser_scope):
def __init__(self, parent_context, parser_scope):
self._parser_scope = parser_scope
self._module_node = self._parser_scope.get_root_node()
self._used_names = self._module_node.get_used_names()
self.context = context
self.parent_context = parent_context
def get(self, name, **filter_kwargs):
return self._convert_names(self._filter(
@@ -81,7 +81,7 @@ class AbstractUsedNamesFilter(AbstractFilter):
))
def _convert_names(self, names):
return [self.name_class(self.context, name) for name in names]
return [self.name_class(self.parent_context, name) for name in names]
def values(self, **filter_kwargs):
return self._convert_names(
@@ -94,22 +94,21 @@ class AbstractUsedNamesFilter(AbstractFilter):
)
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self.context)
return '<%s: %s>' % (self.__class__.__name__, self.parent_context)
class ParserTreeFilter(AbstractUsedNamesFilter):
# TODO remove evaluator as an argument, it's not used.
def __init__(self, evaluator, context, node_context=None, until_position=None,
def __init__(self, parent_context, node_context=None, until_position=None,
origin_scope=None):
"""
node_context is an option to specify a second context for use cases
node_context is an option to specify a second value for use cases
like the class mro where the parent class of a new name would be the
context, but for some type inference it's important to have a local
context of the other classes.
value, but for some type inference it's important to have a local
value of the other classes.
"""
if node_context is None:
node_context = context
super(ParserTreeFilter, self).__init__(context, node_context.tree_node)
node_context = parent_context
super(ParserTreeFilter, self).__init__(parent_context, node_context.tree_node)
self._node_context = node_context
self._origin_scope = origin_scope
self._until_position = until_position
@@ -130,7 +129,7 @@ class ParserTreeFilter(AbstractUsedNamesFilter):
for name in sorted(names, key=lambda name: name.start_pos, reverse=True):
check = flow_analysis.reachability_check(
context=self._node_context,
context_scope=self._parser_scope,
value_scope=self._parser_scope,
node=name,
origin_scope=self._origin_scope
)
@@ -144,11 +143,10 @@ class ParserTreeFilter(AbstractUsedNamesFilter):
class FunctionExecutionFilter(ParserTreeFilter):
param_name = ParamName
def __init__(self, evaluator, context, node_context=None,
def __init__(self, parent_context, node_context=None,
until_position=None, origin_scope=None):
super(FunctionExecutionFilter, self).__init__(
evaluator,
context,
parent_context,
node_context,
until_position,
origin_scope
@@ -159,15 +157,12 @@ class FunctionExecutionFilter(ParserTreeFilter):
for name in names:
param = search_ancestor(name, 'param')
if param:
yield self.param_name(self.context, name)
yield self.param_name(self.parent_context, name)
else:
yield TreeNameDefinition(self.context, name)
yield TreeNameDefinition(self.parent_context, name)
class GlobalNameFilter(AbstractUsedNamesFilter):
def __init__(self, context, parser_scope):
super(GlobalNameFilter, self).__init__(context, parser_scope)
def get(self, name):
try:
names = self._used_names[name]
@@ -231,14 +226,14 @@ class MergedFilter(object):
return '%s(%s)' % (self.__class__.__name__, ', '.join(str(f) for f in self._filters))
class _BuiltinMappedMethod(Context):
class _BuiltinMappedMethod(Value):
"""``Generator.__next__`` ``dict.values`` methods and so on."""
api_type = u'function'
def __init__(self, builtin_context, method, builtin_func):
def __init__(self, builtin_value, method, builtin_func):
super(_BuiltinMappedMethod, self).__init__(
builtin_context.evaluator,
parent_context=builtin_context
builtin_value.inference_state,
parent_context=builtin_value
)
self._method = method
self._builtin_func = builtin_func
@@ -259,19 +254,19 @@ class SpecialMethodFilter(DictFilter):
class SpecialMethodName(AbstractNameDefinition):
api_type = u'function'
def __init__(self, parent_context, string_name, value, builtin_context):
def __init__(self, parent_context, string_name, value, builtin_value):
callable_, python_version = value
if python_version is not None and \
python_version != parent_context.evaluator.environment.version_info.major:
python_version != parent_context.inference_state.environment.version_info.major:
raise KeyError
self.parent_context = parent_context
self.string_name = string_name
self._callable = callable_
self._builtin_context = builtin_context
self._builtin_value = builtin_value
def infer(self):
for filter in self._builtin_context.get_filters():
for filter in self._builtin_value.get_filters():
# We can take the first index, because on builtin methods there's
# always only going to be one name. The same is true for the
# inferred values.
@@ -281,23 +276,23 @@ class SpecialMethodFilter(DictFilter):
else:
continue
break
return ContextSet([
return ValueSet([
_BuiltinMappedMethod(self.parent_context, self._callable, builtin_func)
])
def __init__(self, context, dct, builtin_context):
def __init__(self, value, dct, builtin_value):
super(SpecialMethodFilter, self).__init__(dct)
self.context = context
self._builtin_context = builtin_context
self.value = value
self._builtin_value = builtin_value
"""
This context is what will be used to introspect the name, where as the
other context will be used to execute the function.
This value is what will be used to introspect the name, where as the
other value will be used to execute the function.
We distinguish, because we have to.
"""
def _convert(self, name, value):
return self.SpecialMethodName(self.context, name, value, self._builtin_context)
return self.SpecialMethodName(self.value, name, value, self._builtin_value)
class _OverwriteMeta(type):
@@ -320,21 +315,21 @@ class _OverwriteMeta(type):
class _AttributeOverwriteMixin(object):
def get_filters(self, search_global=False, *args, **kwargs):
yield SpecialMethodFilter(self, self.overwritten_methods, self._wrapped_context)
def get_filters(self, *args, **kwargs):
yield SpecialMethodFilter(self, self.overwritten_methods, self._wrapped_value)
for filter in self._wrapped_context.get_filters(search_global):
for filter in self._wrapped_value.get_filters():
yield filter
class LazyAttributeOverwrite(use_metaclass(_OverwriteMeta, _AttributeOverwriteMixin,
LazyContextWrapper)):
def __init__(self, evaluator):
self.evaluator = evaluator
LazyValueWrapper)):
def __init__(self, inference_state):
self.inference_state = inference_state
class AttributeOverwrite(use_metaclass(_OverwriteMeta, _AttributeOverwriteMixin,
ContextWrapper)):
ValueWrapper)):
pass
@@ -344,69 +339,3 @@ def publish_method(method_name, python_version_match=None):
dct[method_name] = func, python_version_match
return func
return decorator
def get_global_filters(evaluator, context, until_position, origin_scope):
"""
Returns all filters in order of priority for name resolution.
For global name lookups. The filters will handle name resolution
themselves, but here we gather possible filters downwards.
>>> from jedi._compatibility import u, no_unicode_pprint
>>> from jedi import Script
>>> script = Script(u('''
... x = ['a', 'b', 'c']
... def func():
... y = None
... '''))
>>> module_node = script._module_node
>>> scope = next(module_node.iter_funcdefs())
>>> scope
<Function: func@3-5>
>>> context = script._get_module().create_context(scope)
>>> filters = list(get_global_filters(context.evaluator, context, (4, 0), None))
First we get the names from the function scope.
>>> no_unicode_pprint(filters[0]) # doctest: +ELLIPSIS
MergedFilter(<ParserTreeFilter: ...>, <GlobalNameFilter: ...>)
>>> sorted(str(n) for n in filters[0].values()) # doctest: +NORMALIZE_WHITESPACE
['<TreeNameDefinition: string_name=func start_pos=(3, 4)>',
'<TreeNameDefinition: string_name=x start_pos=(2, 0)>']
>>> filters[0]._filters[0]._until_position
(4, 0)
>>> filters[0]._filters[1]._until_position
Then it yields the names from one level "lower". In this example, this is
the module scope (including globals).
As a side note, you can see, that the position in the filter is None on the
globals filter, because there the whole module is searched.
>>> list(filters[1].values()) # package modules -> Also empty.
[]
>>> sorted(name.string_name for name in filters[2].values()) # Module attributes
['__doc__', '__name__', '__package__']
Finally, it yields the builtin filter, if `include_builtin` is
true (default).
>>> list(filters[3].values()) # doctest: +ELLIPSIS
[...]
"""
from jedi.evaluate.context.function import FunctionExecutionContext
while context is not None:
# Names in methods cannot be resolved within the class.
for filter in context.get_filters(
search_global=True,
until_position=until_position,
origin_scope=origin_scope):
yield filter
if isinstance(context, FunctionExecutionContext):
# The position should be reset if the current scope is a function.
until_position = None
context = context.parent_context
# Add builtins to the global scope.
yield next(evaluator.builtins_module.get_filters())
jedi/inference/finder.py
+118
View File
@@ -0,0 +1,118 @@
"""
Searching for names with given scope and name. This is very central in Jedi and
Python. The name resolution is quite complicated with descripter,
``__getattribute__``, ``__getattr__``, ``global``, etc.
If you want to understand name resolution, please read the first few chapters
in http://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/.
Flow checks
+++++++++++
Flow checks are not really mature. There's only a check for ``isinstance``. It
would check whether a flow has the form of ``if isinstance(a, type_or_tuple)``.
Unfortunately every other thing is being ignored (e.g. a == '' would be easy to
check for -> a is a string). There's big potential in these checks.
"""
from parso.tree import search_ancestor
from parso.python.tree import Name
from jedi import settings
from jedi.inference.arguments import TreeArguments
from jedi.inference import helpers
from jedi.inference.value import iterable
from jedi.inference.base_value import NO_VALUES
from jedi.parser_utils import is_scope
def filter_name(filters, name_or_str):
"""
Searches names that are defined in a scope (the different
``filters``), until a name fits.
"""
string_name = name_or_str.value if isinstance(name_or_str, Name) else name_or_str
names = []
for filter in filters:
names = filter.get(string_name)
if names:
break
return list(names)
def check_flow_information(value, flow, search_name, pos):
""" Try to find out the type of a variable just with the information that
is given by the flows: e.g. It is also responsible for assert checks.::
if isinstance(k, str):
k. # <- completion here
ensures that `k` is a string.
"""
if not settings.dynamic_flow_information:
return None
result = None
if is_scope(flow):
# Check for asserts.
module_node = flow.get_root_node()
try:
names = module_node.get_used_names()[search_name.value]
except KeyError:
return None
names = reversed([
n for n in names
if flow.start_pos <= n.start_pos < (pos or flow.end_pos)
])
for name in names:
ass = search_ancestor(name, 'assert_stmt')
if ass is not None:
result = _check_isinstance_type(value, ass.assertion, search_name)
if result is not None:
return result
if flow.type in ('if_stmt', 'while_stmt'):
potential_ifs = [c for c in flow.children[1::4] if c != ':']
for if_test in reversed(potential_ifs):
if search_name.start_pos > if_test.end_pos:
return _check_isinstance_type(value, if_test, search_name)
return result
def _check_isinstance_type(value, element, search_name):
try:
assert element.type in ('power', 'atom_expr')
# this might be removed if we analyze and, etc
assert len(element.children) == 2
first, trailer = element.children
assert first.type == 'name' and first.value == 'isinstance'
assert trailer.type == 'trailer' and trailer.children[0] == '('
assert len(trailer.children) == 3
# arglist stuff
arglist = trailer.children[1]
args = TreeArguments(value.inference_state, value, arglist, trailer)
param_list = list(args.unpack())
# Disallow keyword arguments
assert len(param_list) == 2
(key1, lazy_value_object), (key2, lazy_value_cls) = param_list
assert key1 is None and key2 is None
call = helpers.call_of_leaf(search_name)
is_instance_call = helpers.call_of_leaf(lazy_value_object.data)
# Do a simple get_code comparison. They should just have the same code,
# and everything will be all right.
normalize = value.inference_state.grammar._normalize
assert normalize(is_instance_call) == normalize(call)
except AssertionError:
return None
value_set = NO_VALUES
for cls_or_tup in lazy_value_cls.infer():
if isinstance(cls_or_tup, iterable.Sequence) and cls_or_tup.array_type == 'tuple':
for lazy_value in cls_or_tup.py__iter__():
value_set |= lazy_value.infer().execute_with_values()
else:
value_set |= cls_or_tup.execute_with_values()
return value_set
jedi/evaluate/flow_analysis.py → jedi/inference/flow_analysis.py
+8 -8
View File
@@ -1,5 +1,5 @@
from jedi.parser_utils import get_flow_branch_keyword, is_scope, get_parent_scope
from jedi.evaluate.recursion import execution_allowed
from jedi.inference.recursion import execution_allowed
class Status(object):
@@ -41,7 +41,7 @@ def _get_flow_scopes(node):
yield node
def reachability_check(context, context_scope, node, origin_scope=None):
def reachability_check(context, value_scope, node, origin_scope=None):
first_flow_scope = get_parent_scope(node, include_flows=True)
if origin_scope is not None:
origin_flow_scopes = list(_get_flow_scopes(origin_scope))
@@ -75,10 +75,10 @@ def reachability_check(context, context_scope, node, origin_scope=None):
return REACHABLE
origin_scope = origin_scope.parent
return _break_check(context, context_scope, first_flow_scope, node)
return _break_check(context, value_scope, first_flow_scope, node)
def _break_check(context, context_scope, flow_scope, node):
def _break_check(context, value_scope, flow_scope, node):
reachable = REACHABLE
if flow_scope.type == 'if_stmt':
if flow_scope.is_node_after_else(node):
@@ -98,19 +98,19 @@ def _break_check(context, context_scope, flow_scope, node):
if reachable in (UNREACHABLE, UNSURE):
return reachable
if context_scope != flow_scope and context_scope != flow_scope.parent:
if value_scope != flow_scope and value_scope != flow_scope.parent:
flow_scope = get_parent_scope(flow_scope, include_flows=True)
return reachable & _break_check(context, context_scope, flow_scope, node)
return reachable & _break_check(context, value_scope, flow_scope, node)
else:
return reachable
def _check_if(context, node):
with execution_allowed(context.evaluator, node) as allowed:
with execution_allowed(context.inference_state, node) as allowed:
if not allowed:
return UNSURE
types = context.eval_node(node)
types = context.infer_node(node)
values = set(x.py__bool__() for x in types)
if len(values) == 1:
return Status.lookup_table[values.pop()]
jedi/evaluate/gradual/__init__.py → jedi/inference/gradual/__init__.py
View File
jedi/evaluate/gradual/annotation.py → jedi/inference/gradual/annotation.py
+101 -99
View File
@@ -9,58 +9,58 @@ import re
from parso import ParserSyntaxError, parse
from jedi._compatibility import force_unicode
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS
from jedi.evaluate.gradual.typing import TypeVar, LazyGenericClass, \
from jedi._compatibility import force_unicode, Parameter
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.inference.gradual.typing import TypeVar, LazyGenericClass, \
AbstractAnnotatedClass
from jedi.evaluate.gradual.typing import GenericClass
from jedi.evaluate.helpers import is_string
from jedi.evaluate.compiled import builtin_from_name
from jedi.inference.gradual.typing import GenericClass
from jedi.inference.helpers import is_string
from jedi.inference.compiled import builtin_from_name
from jedi import debug
from jedi import parser_utils
def eval_annotation(context, annotation):
def infer_annotation(context, annotation):
"""
Evaluates an annotation node. This means that it evaluates the part of
Inferes an annotation node. This means that it inferes the part of
`int` here:
foo: int = 3
Also checks for forward references (strings)
"""
context_set = context.eval_node(annotation)
if len(context_set) != 1:
debug.warning("Eval'ed typing index %s should lead to 1 object, "
" not %s" % (annotation, context_set))
return context_set
value_set = context.infer_node(annotation)
if len(value_set) != 1:
debug.warning("Inferred typing index %s should lead to 1 object, "
" not %s" % (annotation, value_set))
return value_set
evaled_context = list(context_set)[0]
if is_string(evaled_context):
result = _get_forward_reference_node(context, evaled_context.get_safe_value())
inferred_value = list(value_set)[0]
if is_string(inferred_value):
result = _get_forward_reference_node(context, inferred_value.get_safe_value())
if result is not None:
return context.eval_node(result)
return context_set
return context.infer_node(result)
return value_set
def _evaluate_annotation_string(context, string, index=None):
def _infer_annotation_string(context, string, index=None):
node = _get_forward_reference_node(context, string)
if node is None:
return NO_CONTEXTS
return NO_VALUES
context_set = context.eval_node(node)
value_set = context.infer_node(node)
if index is not None:
context_set = context_set.filter(
lambda context: context.array_type == u'tuple' # noqa
and len(list(context.py__iter__())) >= index
value_set = value_set.filter(
lambda value: value.array_type == u'tuple' # noqa
and len(list(value.py__iter__())) >= index
).py__simple_getitem__(index)
return context_set
return value_set
def _get_forward_reference_node(context, string):
try:
new_node = context.evaluator.grammar.parse(
new_node = context.inference_state.grammar.parse(
force_unicode(string),
start_symbol='eval_input',
error_recovery=False
@@ -106,24 +106,26 @@ def _split_comment_param_declaration(decl_text):
return params
@evaluator_method_cache()
def infer_param(execution_context, param):
contexts = _infer_param(execution_context, param)
evaluator = execution_context.evaluator
@inference_state_method_cache()
def infer_param(execution_context, param, ignore_stars=False):
values = _infer_param(execution_context, param)
if ignore_stars:
return values
inference_state = execution_context.inference_state
if param.star_count == 1:
tuple_ = builtin_from_name(evaluator, 'tuple')
return ContextSet([GenericClass(
tuple_ = builtin_from_name(inference_state, 'tuple')
return ValueSet([GenericClass(
tuple_,
generics=(contexts,),
) for c in contexts])
generics=(values,),
) for c in values])
elif param.star_count == 2:
dct = builtin_from_name(evaluator, 'dict')
return ContextSet([GenericClass(
dct = builtin_from_name(inference_state, 'dict')
return ValueSet([GenericClass(
dct,
generics=(ContextSet([builtin_from_name(evaluator, 'str')]), contexts),
) for c in contexts])
generics=(ValueSet([builtin_from_name(inference_state, 'str')]), values),
) for c in values])
pass
return contexts
return values
def _infer_param(execution_context, param):
@@ -142,11 +144,11 @@ def _infer_param(execution_context, param):
node = param.parent.parent
comment = parser_utils.get_following_comment_same_line(node)
if comment is None:
return NO_CONTEXTS
return NO_VALUES
match = re.match(r"^#\s*type:\s*\(([^#]*)\)\s*->", comment)
if not match:
return NO_CONTEXTS
return NO_VALUES
params_comments = _split_comment_param_declaration(match.group(1))
# Find the specific param being investigated
@@ -158,23 +160,23 @@ def _infer_param(execution_context, param):
"Comments length != Params length %s %s",
params_comments, all_params
)
from jedi.evaluate.context.instance import InstanceArguments
from jedi.inference.value.instance import InstanceArguments
if isinstance(execution_context.var_args, InstanceArguments):
if index == 0:
# Assume it's self, which is already handled
return NO_CONTEXTS
return NO_VALUES
index -= 1
if index >= len(params_comments):
return NO_CONTEXTS
return NO_VALUES
param_comment = params_comments[index]
return _evaluate_annotation_string(
execution_context.function_context.get_default_param_context(),
return _infer_annotation_string(
execution_context.function_value.get_default_param_context(),
param_comment
)
# Annotations are like default params and resolve in the same way.
context = execution_context.function_context.get_default_param_context()
return eval_annotation(context, annotation)
context = execution_context.function_value.get_default_param_context()
return infer_annotation(context, annotation)
def py__annotations__(funcdef):
@@ -190,7 +192,7 @@ def py__annotations__(funcdef):
return dct
@evaluator_method_cache()
@inference_state_method_cache()
def infer_return_types(function_execution_context):
"""
Infers the type of a function's return value,
@@ -203,31 +205,31 @@ def infer_return_types(function_execution_context):
node = function_execution_context.tree_node
comment = parser_utils.get_following_comment_same_line(node)
if comment is None:
return NO_CONTEXTS
return NO_VALUES
match = re.match(r"^#\s*type:\s*\([^#]*\)\s*->\s*([^#]*)", comment)
if not match:
return NO_CONTEXTS
return NO_VALUES
return _evaluate_annotation_string(
function_execution_context.function_context.get_default_param_context(),
return _infer_annotation_string(
function_execution_context.function_value.get_default_param_context(),
match.group(1).strip()
).execute_annotation()
if annotation is None:
return NO_CONTEXTS
return NO_VALUES
context = function_execution_context.function_context.get_default_param_context()
context = function_execution_context.function_value.get_default_param_context()
unknown_type_vars = list(find_unknown_type_vars(context, annotation))
annotation_contexts = eval_annotation(context, annotation)
annotation_values = infer_annotation(context, annotation)
if not unknown_type_vars:
return annotation_contexts.execute_annotation()
return annotation_values.execute_annotation()
type_var_dict = infer_type_vars_for_execution(function_execution_context, all_annotations)
return ContextSet.from_sets(
return ValueSet.from_sets(
ann.define_generics(type_var_dict)
if isinstance(ann, (AbstractAnnotatedClass, TypeVar)) else ContextSet({ann})
for ann in annotation_contexts
if isinstance(ann, (AbstractAnnotatedClass, TypeVar)) else ValueSet({ann})
for ann in annotation_values
).execute_annotation()
@@ -241,48 +243,48 @@ def infer_type_vars_for_execution(execution_context, annotation_dict):
2. Infer type vars with the execution state we have.
3. Return the union of all type vars that have been found.
"""
context = execution_context.function_context.get_default_param_context()
context = execution_context.function_value.get_default_param_context()
annotation_variable_results = {}
executed_params, _ = execution_context.get_executed_params_and_issues()
for executed_param in executed_params:
executed_param_names, _ = execution_context.get_executed_param_names_and_issues()
for executed_param_name in executed_param_names:
try:
annotation_node = annotation_dict[executed_param.string_name]
annotation_node = annotation_dict[executed_param_name.string_name]
except KeyError:
continue
annotation_variables = find_unknown_type_vars(context, annotation_node)
if annotation_variables:
# Infer unknown type var
annotation_context_set = context.eval_node(annotation_node)
star_count = executed_param._param_node.star_count
actual_context_set = executed_param.infer(use_hints=False)
if star_count == 1:
actual_context_set = actual_context_set.merge_types_of_iterate()
elif star_count == 2:
annotation_value_set = context.infer_node(annotation_node)
kind = executed_param_name.get_kind()
actual_value_set = executed_param_name.infer()
if kind is Parameter.VAR_POSITIONAL:
actual_value_set = actual_value_set.merge_types_of_iterate()
elif kind is Parameter.VAR_KEYWORD:
# TODO _dict_values is not public.
actual_context_set = actual_context_set.try_merge('_dict_values')
for ann in annotation_context_set:
actual_value_set = actual_value_set.try_merge('_dict_values')
for ann in annotation_value_set:
_merge_type_var_dicts(
annotation_variable_results,
_infer_type_vars(ann, actual_context_set),
_infer_type_vars(ann, actual_value_set),
)
return annotation_variable_results
def _merge_type_var_dicts(base_dict, new_dict):
for type_var_name, contexts in new_dict.items():
for type_var_name, values in new_dict.items():
try:
base_dict[type_var_name] |= contexts
base_dict[type_var_name] |= values
except KeyError:
base_dict[type_var_name] = contexts
base_dict[type_var_name] = values
def _infer_type_vars(annotation_context, context_set):
def _infer_type_vars(annotation_value, value_set):
"""
This function tries to find information about undefined type vars and
returns a dict from type var name to context set.
returns a dict from type var name to value set.
This is for example important to understand what `iter([1])` returns.
According to typeshed, `iter` returns an `Iterator[_T]`:
@@ -293,45 +295,45 @@ def _infer_type_vars(annotation_context, context_set):
unpacks the `Iterable`.
"""
type_var_dict = {}
if isinstance(annotation_context, TypeVar):
return {annotation_context.py__name__(): context_set.py__class__()}
elif isinstance(annotation_context, LazyGenericClass):
name = annotation_context.py__name__()
if isinstance(annotation_value, TypeVar):
return {annotation_value.py__name__(): value_set.py__class__()}
elif isinstance(annotation_value, LazyGenericClass):
name = annotation_value.py__name__()
if name == 'Iterable':
given = annotation_context.get_generics()
given = annotation_value.get_generics()
if given:
for nested_annotation_context in given[0]:
for nested_annotation_value in given[0]:
_merge_type_var_dicts(
type_var_dict,
_infer_type_vars(
nested_annotation_context,
context_set.merge_types_of_iterate()
nested_annotation_value,
value_set.merge_types_of_iterate()
)
)
elif name == 'Mapping':
given = annotation_context.get_generics()
given = annotation_value.get_generics()
if len(given) == 2:
for context in context_set:
for value in value_set:
try:
method = context.get_mapping_item_contexts
method = value.get_mapping_item_values
except AttributeError:
continue
key_contexts, value_contexts = method()
key_values, value_values = method()
for nested_annotation_context in given[0]:
for nested_annotation_value in given[0]:
_merge_type_var_dicts(
type_var_dict,
_infer_type_vars(
nested_annotation_context,
key_contexts,
nested_annotation_value,
key_values,
)
)
for nested_annotation_context in given[1]:
for nested_annotation_value in given[1]:
_merge_type_var_dicts(
type_var_dict,
_infer_type_vars(
nested_annotation_context,
value_contexts,
nested_annotation_value,
value_values,
)
)
return type_var_dict
@@ -372,7 +374,7 @@ def _find_type_from_comment_hint(context, node, varlist, name):
match = re.match(r"^#\s*type:\s*([^#]*)", comment)
if match is None:
return []
return _evaluate_annotation_string(
return _infer_annotation_string(
context, match.group(1).strip(), index
).execute_annotation()
@@ -385,7 +387,7 @@ def find_unknown_type_vars(context, node):
for subscript_node in _unpack_subscriptlist(trailer.children[1]):
check_node(subscript_node)
else:
type_var_set = context.eval_node(node)
type_var_set = context.infer_node(node)
for type_var in type_var_set:
if isinstance(type_var, TypeVar) and type_var not in found:
found.append(type_var)
jedi/evaluate/gradual/conversion.py → jedi/inference/gradual/conversion.py
+70 -69
View File
@@ -1,48 +1,49 @@
from jedi import debug
from jedi.evaluate.base_context import ContextSet, \
NO_CONTEXTS
from jedi.evaluate.utils import to_list
from jedi.evaluate.gradual.stub_context import StubModuleContext
from jedi.inference.base_value import ValueSet, \
NO_VALUES
from jedi.inference.utils import to_list
from jedi.inference.gradual.stub_value import StubModuleValue
def _stub_to_python_context_set(stub_context, ignore_compiled=False):
stub_module = stub_context.get_root_context()
if not stub_module.is_stub():
return ContextSet([stub_context])
def _stub_to_python_value_set(stub_value, ignore_compiled=False):
stub_module_context = stub_value.get_root_context()
if not stub_module_context.is_stub():
return ValueSet([stub_value])
was_instance = stub_context.is_instance()
was_instance = stub_value.is_instance()
if was_instance:
stub_context = stub_context.py__class__()
stub_value = stub_value.py__class__()
qualified_names = stub_context.get_qualified_names()
qualified_names = stub_value.get_qualified_names()
if qualified_names is None:
return NO_CONTEXTS
return NO_VALUES
was_bound_method = stub_context.is_bound_method()
was_bound_method = stub_value.is_bound_method()
if was_bound_method:
# Infer the object first. We can infer the method later.
method_name = qualified_names[-1]
qualified_names = qualified_names[:-1]
was_instance = True
contexts = _infer_from_stub(stub_module, qualified_names, ignore_compiled)
values = _infer_from_stub(stub_module_context, qualified_names, ignore_compiled)
if was_instance:
contexts = ContextSet.from_sets(
c.execute_evaluated()
for c in contexts
values = ValueSet.from_sets(
c.execute_with_values()
for c in values
if c.is_class()
)
if was_bound_method:
# Now that the instance has been properly created, we can simply get
# the method.
contexts = contexts.py__getattribute__(method_name)
return contexts
values = values.py__getattribute__(method_name)
return values
def _infer_from_stub(stub_module, qualified_names, ignore_compiled):
from jedi.evaluate.compiled.mixed import MixedObject
assert isinstance(stub_module, (StubModuleContext, MixedObject)), stub_module
non_stubs = stub_module.non_stub_context_set
def _infer_from_stub(stub_module_context, qualified_names, ignore_compiled):
from jedi.inference.compiled.mixed import MixedObject
stub_module = stub_module_context.get_value()
assert isinstance(stub_module, (StubModuleValue, MixedObject)), stub_module_context
non_stubs = stub_module.non_stub_value_set
if ignore_compiled:
non_stubs = non_stubs.filter(lambda c: not c.is_compiled())
for name in qualified_names:
@@ -53,28 +54,28 @@ def _infer_from_stub(stub_module, qualified_names, ignore_compiled):
@to_list
def _try_stub_to_python_names(names, prefer_stub_to_compiled=False):
for name in names:
module = name.get_root_context()
if not module.is_stub():
module_context = name.get_root_context()
if not module_context.is_stub():
yield name
continue
name_list = name.get_qualified_names()
if name_list is None:
contexts = NO_CONTEXTS
values = NO_VALUES
else:
contexts = _infer_from_stub(
module,
values = _infer_from_stub(
module_context,
name_list[:-1],
ignore_compiled=prefer_stub_to_compiled,
)
if contexts and name_list:
new_names = contexts.py__getattribute__(name_list[-1], is_goto=True)
if values and name_list:
new_names = values.goto(name_list[-1])
for new_name in new_names:
yield new_name
if new_names:
continue
elif contexts:
for c in contexts:
elif values:
for c in values:
yield c.name
continue
# This is the part where if we haven't found anything, just return the
@@ -85,21 +86,21 @@ def _try_stub_to_python_names(names, prefer_stub_to_compiled=False):
def _load_stub_module(module):
if module.is_stub():
return module
from jedi.evaluate.gradual.typeshed import _try_to_load_stub_cached
from jedi.inference.gradual.typeshed import _try_to_load_stub_cached
return _try_to_load_stub_cached(
module.evaluator,
module.inference_state,
import_names=module.string_names,
python_context_set=ContextSet([module]),
parent_module_context=None,
sys_path=module.evaluator.get_sys_path(),
python_value_set=ValueSet([module]),
parent_module_value=None,
sys_path=module.inference_state.get_sys_path(),
)
@to_list
def _python_to_stub_names(names, fallback_to_python=False):
for name in names:
module = name.get_root_context()
if module.is_stub():
module_context = name.get_root_context()
if module_context.is_stub():
yield name
continue
@@ -112,15 +113,15 @@ def _python_to_stub_names(names, fallback_to_python=False):
continue
name_list = name.get_qualified_names()
stubs = NO_CONTEXTS
stubs = NO_VALUES
if name_list is not None:
stub_module = _load_stub_module(module)
stub_module = _load_stub_module(module_context.get_value())
if stub_module is not None:
stubs = ContextSet({stub_module})
stubs = ValueSet({stub_module})
for name in name_list[:-1]:
stubs = stubs.py__getattribute__(name)
if stubs and name_list:
new_names = stubs.py__getattribute__(name_list[-1], is_goto=True)
new_names = stubs.goto(name_list[-1])
for new_name in new_names:
yield new_name
if new_names:
@@ -144,56 +145,56 @@ def convert_names(names, only_stubs=False, prefer_stubs=False):
return _try_stub_to_python_names(names, prefer_stub_to_compiled=True)
def convert_contexts(contexts, only_stubs=False, prefer_stubs=False, ignore_compiled=True):
def convert_values(values, only_stubs=False, prefer_stubs=False, ignore_compiled=True):
assert not (only_stubs and prefer_stubs)
with debug.increase_indent_cm('convert contexts'):
with debug.increase_indent_cm('convert values'):
if only_stubs or prefer_stubs:
return ContextSet.from_sets(
to_stub(context)
or (ContextSet({context}) if prefer_stubs else NO_CONTEXTS)
for context in contexts
return ValueSet.from_sets(
to_stub(value)
or (ValueSet({value}) if prefer_stubs else NO_VALUES)
for value in values
)
else:
return ContextSet.from_sets(
_stub_to_python_context_set(stub_context, ignore_compiled=ignore_compiled)
or ContextSet({stub_context})
for stub_context in contexts
return ValueSet.from_sets(
_stub_to_python_value_set(stub_value, ignore_compiled=ignore_compiled)
or ValueSet({stub_value})
for stub_value in values
)
# TODO merge with _python_to_stub_names?
def to_stub(context):
if context.is_stub():
return ContextSet([context])
def to_stub(value):
if value.is_stub():
return ValueSet([value])
was_instance = context.is_instance()
was_instance = value.is_instance()
if was_instance:
context = context.py__class__()
value = value.py__class__()
qualified_names = context.get_qualified_names()
stub_module = _load_stub_module(context.get_root_context())
qualified_names = value.get_qualified_names()
stub_module = _load_stub_module(value.get_root_context().get_value())
if stub_module is None or qualified_names is None:
return NO_CONTEXTS
return NO_VALUES
was_bound_method = context.is_bound_method()
was_bound_method = value.is_bound_method()
if was_bound_method:
# Infer the object first. We can infer the method later.
method_name = qualified_names[-1]
qualified_names = qualified_names[:-1]
was_instance = True
stub_contexts = ContextSet([stub_module])
stub_values = ValueSet([stub_module])
for name in qualified_names:
stub_contexts = stub_contexts.py__getattribute__(name)
stub_values = stub_values.py__getattribute__(name)
if was_instance:
stub_contexts = ContextSet.from_sets(
c.execute_evaluated()
for c in stub_contexts
stub_values = ValueSet.from_sets(
c.execute_with_values()
for c in stub_values
if c.is_class()
)
if was_bound_method:
# Now that the instance has been properly created, we can simply get
# the method.
stub_contexts = stub_contexts.py__getattribute__(method_name)
return stub_contexts
stub_values = stub_values.py__getattribute__(method_name)
return stub_values
jedi/inference/gradual/stub_value.py
+104
View File
@@ -0,0 +1,104 @@
from jedi.inference.base_value import ValueWrapper
from jedi.inference.value.module import ModuleValue
from jedi.inference.filters import ParserTreeFilter, \
TreeNameDefinition
from jedi.inference.gradual.typing import TypingModuleFilterWrapper
from jedi.inference.context import ModuleContext
class StubModuleValue(ModuleValue):
def __init__(self, non_stub_value_set, *args, **kwargs):
super(StubModuleValue, self).__init__(*args, **kwargs)
self.non_stub_value_set = non_stub_value_set
def is_stub(self):
return True
def sub_modules_dict(self):
"""
We have to overwrite this, because it's possible to have stubs that
don't have code for all the child modules. At the time of writing this
there are for example no stubs for `json.tool`.
"""
names = {}
for value in self.non_stub_value_set:
try:
method = value.sub_modules_dict
except AttributeError:
pass
else:
names.update(method())
names.update(super(StubModuleValue, self).sub_modules_dict())
return names
def _get_first_non_stub_filters(self):
for value in self.non_stub_value_set:
yield next(value.get_filters())
def _get_stub_filters(self, origin_scope):
return [StubFilter(
parent_context=self.as_context(),
origin_scope=origin_scope
)] + list(self.iter_star_filters())
def get_filters(self, origin_scope=None):
filters = super(StubModuleValue, self).get_filters(origin_scope)
next(filters) # Ignore the first filter and replace it with our own
stub_filters = self._get_stub_filters(origin_scope=origin_scope)
for f in stub_filters:
yield f
for f in filters:
yield f
class TypingModuleWrapper(StubModuleValue):
def get_filters(self, *args, **kwargs):
filters = super(TypingModuleWrapper, self).get_filters(*args, **kwargs)
yield TypingModuleFilterWrapper(next(filters))
for f in filters:
yield f
def _as_context(self):
return TypingModuleContext(self)
class TypingModuleContext(ModuleContext):
def get_filters(self, *args, **kwargs):
filters = super(TypingModuleContext, self).get_filters(*args, **kwargs)
yield TypingModuleFilterWrapper(next(filters))
for f in filters:
yield f
# From here on down we make looking up the sys.version_info fast.
class _StubName(TreeNameDefinition):
def infer(self):
inferred = super(_StubName, self).infer()
if self.string_name == 'version_info' and self.get_root_context().py__name__() == 'sys':
return [VersionInfo(c) for c in inferred]
return inferred
class StubFilter(ParserTreeFilter):
name_class = _StubName
def _is_name_reachable(self, name):
if not super(StubFilter, self)._is_name_reachable(name):
return False
# Imports in stub files are only public if they have an "as"
# export.
definition = name.get_definition()
if definition.type in ('import_from', 'import_name'):
if name.parent.type not in ('import_as_name', 'dotted_as_name'):
return False
n = name.value
# TODO rewrite direct return
if n.startswith('_') and not (n.startswith('__') and n.endswith('__')):
return False
return True
class VersionInfo(ValueWrapper):
pass
jedi/evaluate/gradual/typeshed.py → jedi/inference/gradual/typeshed.py
+63 -64
View File
@@ -5,8 +5,9 @@ from functools import wraps
from jedi.file_io import FileIO
from jedi._compatibility import FileNotFoundError, cast_path
from jedi.parser_utils import get_cached_code_lines
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS
from jedi.evaluate.gradual.stub_context import TypingModuleWrapper, StubModuleContext
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.inference.gradual.stub_value import TypingModuleWrapper, StubModuleValue
from jedi.inference.value import ModuleValue
_jedi_path = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
TYPESHED_PATH = os.path.join(_jedi_path, 'third_party', 'typeshed')
@@ -89,69 +90,72 @@ def _cache_stub_file_map(version_info):
def import_module_decorator(func):
@wraps(func)
def wrapper(evaluator, import_names, parent_module_context, sys_path, prefer_stubs):
def wrapper(inference_state, import_names, parent_module_value, sys_path, prefer_stubs):
try:
python_context_set = evaluator.module_cache.get(import_names)
python_value_set = inference_state.module_cache.get(import_names)
except KeyError:
if parent_module_context is not None and parent_module_context.is_stub():
parent_module_contexts = parent_module_context.non_stub_context_set
if parent_module_value is not None and parent_module_value.is_stub():
parent_module_values = parent_module_value.non_stub_value_set
else:
parent_module_contexts = [parent_module_context]
parent_module_values = [parent_module_value]
if import_names == ('os', 'path'):
# This is a huge exception, we follow a nested import
# ``os.path``, because it's a very important one in Python
# that is being achieved by messing with ``sys.modules`` in
# ``os``.
python_parent = next(iter(parent_module_contexts))
python_parent = next(iter(parent_module_values))
if python_parent is None:
python_parent, = evaluator.import_module(('os',), prefer_stubs=False)
python_context_set = python_parent.py__getattribute__('path')
else:
python_context_set = ContextSet.from_sets(
func(evaluator, import_names, p, sys_path,)
for p in parent_module_contexts
python_parent, = inference_state.import_module(('os',), prefer_stubs=False)
python_value_set = ValueSet.from_sets(
func(inference_state, (n,), None, sys_path,)
for n in [u'posixpath', u'ntpath', u'macpath', u'os2emxpath']
)
evaluator.module_cache.add(import_names, python_context_set)
else:
python_value_set = ValueSet.from_sets(
func(inference_state, import_names, p, sys_path,)
for p in parent_module_values
)
inference_state.module_cache.add(import_names, python_value_set)
if not prefer_stubs:
return python_context_set
return python_value_set
stub = _try_to_load_stub_cached(evaluator, import_names, python_context_set,
parent_module_context, sys_path)
stub = _try_to_load_stub_cached(inference_state, import_names, python_value_set,
parent_module_value, sys_path)
if stub is not None:
return ContextSet([stub])
return python_context_set
return ValueSet([stub])
return python_value_set
return wrapper
def _try_to_load_stub_cached(evaluator, import_names, *args, **kwargs):
def _try_to_load_stub_cached(inference_state, import_names, *args, **kwargs):
try:
return evaluator.stub_module_cache[import_names]
return inference_state.stub_module_cache[import_names]
except KeyError:
pass
# TODO is this needed? where are the exceptions coming from that make this
# necessary? Just remove this line.
evaluator.stub_module_cache[import_names] = None
evaluator.stub_module_cache[import_names] = result = \
_try_to_load_stub(evaluator, import_names, *args, **kwargs)
inference_state.stub_module_cache[import_names] = None
inference_state.stub_module_cache[import_names] = result = \
_try_to_load_stub(inference_state, import_names, *args, **kwargs)
return result
def _try_to_load_stub(evaluator, import_names, python_context_set,
parent_module_context, sys_path):
def _try_to_load_stub(inference_state, import_names, python_value_set,
parent_module_value, sys_path):
"""
Trying to load a stub for a set of import_names.
This is modelled to work like "PEP 561 -- Distributing and Packaging Type
Information", see https://www.python.org/dev/peps/pep-0561.
"""
if parent_module_context is None and len(import_names) > 1:
if parent_module_value is None and len(import_names) > 1:
try:
parent_module_context = _try_to_load_stub_cached(
evaluator, import_names[:-1], NO_CONTEXTS,
parent_module_context=None, sys_path=sys_path)
parent_module_value = _try_to_load_stub_cached(
inference_state, import_names[:-1], NO_VALUES,
parent_module_value=None, sys_path=sys_path)
except KeyError:
pass
@@ -161,8 +165,8 @@ def _try_to_load_stub(evaluator, import_names, python_context_set,
for p in sys_path:
init = os.path.join(p, *import_names) + '-stubs' + os.path.sep + '__init__.pyi'
m = _try_to_load_stub_from_file(
evaluator,
python_context_set,
inference_state,
python_value_set,
file_io=FileIO(init),
import_names=import_names,
)
@@ -170,7 +174,7 @@ def _try_to_load_stub(evaluator, import_names, python_context_set,
return m
# 2. Try to load pyi files next to py files.
for c in python_context_set:
for c in python_value_set:
try:
method = c.py__file__
except AttributeError:
@@ -185,8 +189,8 @@ def _try_to_load_stub(evaluator, import_names, python_context_set,
for file_path in file_paths:
m = _try_to_load_stub_from_file(
evaluator,
python_context_set,
inference_state,
python_value_set,
# The file path should end with .pyi
file_io=FileIO(file_path),
import_names=import_names,
@@ -195,19 +199,14 @@ def _try_to_load_stub(evaluator, import_names, python_context_set,
return m
# 3. Try to load typeshed
m = _load_from_typeshed(evaluator, python_context_set, parent_module_context, import_names)
m = _load_from_typeshed(inference_state, python_value_set, parent_module_value, import_names)
if m is not None:
return m
# 4. Try to load pyi file somewhere if python_context_set was not defined.
if not python_context_set:
if parent_module_context is not None:
try:
method = parent_module_context.py__path__
except AttributeError:
check_path = []
else:
check_path = method()
# 4. Try to load pyi file somewhere if python_value_set was not defined.
if not python_value_set:
if parent_module_value is not None:
check_path = parent_module_value.py__path__() or []
# In case import_names
names_for_path = (import_names[-1],)
else:
@@ -216,8 +215,8 @@ def _try_to_load_stub(evaluator, import_names, python_context_set,
for p in check_path:
m = _try_to_load_stub_from_file(
evaluator,
python_context_set,
inference_state,
python_value_set,
file_io=FileIO(os.path.join(p, *names_for_path) + '.pyi'),
import_names=import_names,
)
@@ -229,34 +228,34 @@ def _try_to_load_stub(evaluator, import_names, python_context_set,
return None
def _load_from_typeshed(evaluator, python_context_set, parent_module_context, import_names):
def _load_from_typeshed(inference_state, python_value_set, parent_module_value, import_names):
import_name = import_names[-1]
map_ = None
if len(import_names) == 1:
map_ = _cache_stub_file_map(evaluator.grammar.version_info)
map_ = _cache_stub_file_map(inference_state.grammar.version_info)
import_name = _IMPORT_MAP.get(import_name, import_name)
elif isinstance(parent_module_context, StubModuleContext):
if not parent_module_context.is_package:
elif isinstance(parent_module_value, ModuleValue):
if not parent_module_value.is_package:
# Only if it's a package (= a folder) something can be
# imported.
return None
path = parent_module_context.py__path__()
path = parent_module_value.py__path__()
map_ = _merge_create_stub_map(path)
if map_ is not None:
path = map_.get(import_name)
if path is not None:
return _try_to_load_stub_from_file(
evaluator,
python_context_set,
inference_state,
python_value_set,
file_io=FileIO(path),
import_names=import_names,
)
def _try_to_load_stub_from_file(evaluator, python_context_set, file_io, import_names):
def _try_to_load_stub_from_file(inference_state, python_value_set, file_io, import_names):
try:
stub_module_node = evaluator.parse(
stub_module_node = inference_state.parse(
file_io=file_io,
cache=True,
use_latest_grammar=True
@@ -266,24 +265,24 @@ def _try_to_load_stub_from_file(evaluator, python_context_set, file_io, import_n
return None
else:
return create_stub_module(
evaluator, python_context_set, stub_module_node, file_io,
inference_state, python_value_set, stub_module_node, file_io,
import_names
)
def create_stub_module(evaluator, python_context_set, stub_module_node, file_io, import_names):
def create_stub_module(inference_state, python_value_set, stub_module_node, file_io, import_names):
if import_names == ('typing',):
module_cls = TypingModuleWrapper
else:
module_cls = StubModuleContext
module_cls = StubModuleValue
file_name = os.path.basename(file_io.path)
stub_module_context = module_cls(
python_context_set, evaluator, stub_module_node,
stub_module_value = module_cls(
python_value_set, inference_state, stub_module_node,
file_io=file_io,
string_names=import_names,
# The code was loaded with latest_grammar, so use
# that.
code_lines=get_cached_code_lines(evaluator.latest_grammar, file_io.path),
code_lines=get_cached_code_lines(inference_state.latest_grammar, file_io.path),
is_package=file_name == '__init__.pyi',
)
return stub_module_context
return stub_module_value
jedi/evaluate/gradual/typing.py → jedi/inference/gradual/typing.py
+218 -210
View File
@@ -1,25 +1,26 @@
"""
We need to somehow work with the typing objects. Since the typing objects are
pretty bare we need to add all the Jedi customizations to make them work as
contexts.
values.
This file deals with all the typing.py cases.
"""
from jedi._compatibility import unicode, force_unicode
from jedi import debug
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.compiled import builtin_from_name
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS, Context, \
iterator_to_context_set, ContextWrapper, LazyContextWrapper
from jedi.evaluate.lazy_context import LazyKnownContexts
from jedi.evaluate.context.iterable import SequenceLiteralContext
from jedi.evaluate.arguments import repack_with_argument_clinic
from jedi.evaluate.utils import to_list
from jedi.evaluate.filters import FilterWrapper
from jedi.evaluate.names import NameWrapper, AbstractTreeName, \
AbstractNameDefinition, ContextName
from jedi.evaluate.helpers import is_string
from jedi.evaluate.context.klass import ClassMixin, ClassFilter
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.compiled import builtin_from_name
from jedi.inference.base_value import ValueSet, NO_VALUES, Value, \
iterator_to_value_set, ValueWrapper, LazyValueWrapper
from jedi.inference.lazy_value import LazyKnownValues
from jedi.inference.value.iterable import SequenceLiteralValue
from jedi.inference.arguments import repack_with_argument_clinic
from jedi.inference.utils import to_list
from jedi.inference.filters import FilterWrapper
from jedi.inference.names import NameWrapper, AbstractTreeName, \
AbstractNameDefinition, ValueName
from jedi.inference.helpers import is_string
from jedi.inference.value.klass import ClassMixin, ClassFilter
from jedi.inference.context import ClassContext
_PROXY_CLASS_TYPES = 'Tuple Generic Protocol Callable Type'.split()
_TYPE_ALIAS_TYPES = {
@@ -36,17 +37,17 @@ _PROXY_TYPES = 'Optional Union ClassVar'.split()
class TypingName(AbstractTreeName):
def __init__(self, context, other_name):
super(TypingName, self).__init__(context.parent_context, other_name.tree_name)
self._context = context
def __init__(self, value, other_name):
super(TypingName, self).__init__(value.parent_context, other_name.tree_name)
self._value = value
def infer(self):
return ContextSet([self._context])
return ValueSet([self._value])
class _BaseTypingContext(Context):
def __init__(self, evaluator, parent_context, tree_name):
super(_BaseTypingContext, self).__init__(evaluator, parent_context)
class _BaseTypingValue(Value):
def __init__(self, inference_state, parent_context, tree_name):
super(_BaseTypingValue, self).__init__(inference_state, parent_context)
self._tree_name = tree_name
@property
@@ -71,11 +72,11 @@ class _BaseTypingContext(Context):
# TODO this is obviously not correct, but at least gives us a class if
# we have none. Some of these objects don't really have a base class in
# typeshed.
return builtin_from_name(self.evaluator, u'object')
return builtin_from_name(self.inference_state, u'object')
@property
def name(self):
return ContextName(self, self._tree_name)
return ValueName(self, self._tree_name)
def __repr__(self):
return '%s(%s)' % (self.__class__.__name__, self._tree_name.value)
@@ -83,43 +84,43 @@ class _BaseTypingContext(Context):
class TypingModuleName(NameWrapper):
def infer(self):
return ContextSet(self._remap())
return ValueSet(self._remap())
def _remap(self):
name = self.string_name
evaluator = self.parent_context.evaluator
inference_state = self.parent_context.inference_state
try:
actual = _TYPE_ALIAS_TYPES[name]
except KeyError:
pass
else:
yield TypeAlias.create_cached(evaluator, self.parent_context, self.tree_name, actual)
yield TypeAlias.create_cached(inference_state, self.parent_context, self.tree_name, actual)
return
if name in _PROXY_CLASS_TYPES:
yield TypingClassContext.create_cached(evaluator, self.parent_context, self.tree_name)
yield TypingClassValue.create_cached(inference_state, self.parent_context, self.tree_name)
elif name in _PROXY_TYPES:
yield TypingContext.create_cached(evaluator, self.parent_context, self.tree_name)
yield TypingValue.create_cached(inference_state, self.parent_context, self.tree_name)
elif name == 'runtime':
# We don't want anything here, not sure what this function is
# supposed to do, since it just appears in the stubs and shouldn't
# have any effects there (because it's never executed).
return
elif name == 'TypeVar':
yield TypeVarClass.create_cached(evaluator, self.parent_context, self.tree_name)
yield TypeVarClass.create_cached(inference_state, self.parent_context, self.tree_name)
elif name == 'Any':
yield Any.create_cached(evaluator, self.parent_context, self.tree_name)
yield Any.create_cached(inference_state, self.parent_context, self.tree_name)
elif name == 'TYPE_CHECKING':
# This is needed for e.g. imports that are only available for type
# checking or are in cycles. The user can then check this variable.
yield builtin_from_name(evaluator, u'True')
yield builtin_from_name(inference_state, u'True')
elif name == 'overload':
yield OverloadFunction.create_cached(evaluator, self.parent_context, self.tree_name)
yield OverloadFunction.create_cached(inference_state, self.parent_context, self.tree_name)
elif name == 'NewType':
yield NewTypeFunction.create_cached(evaluator, self.parent_context, self.tree_name)
yield NewTypeFunction.create_cached(inference_state, self.parent_context, self.tree_name)
elif name == 'cast':
# TODO implement cast
yield CastFunction.create_cached(evaluator, self.parent_context, self.tree_name)
yield CastFunction.create_cached(inference_state, self.parent_context, self.tree_name)
elif name == 'TypedDict':
# TODO doesn't even exist in typeshed/typing.py, yet. But will be
# added soon.
@@ -138,122 +139,126 @@ class TypingModuleFilterWrapper(FilterWrapper):
name_wrapper_class = TypingModuleName
class _WithIndexBase(_BaseTypingContext):
def __init__(self, evaluator, parent_context, name, index_context, context_of_index):
super(_WithIndexBase, self).__init__(evaluator, parent_context, name)
self._index_context = index_context
self._context_of_index = context_of_index
class _WithIndexBase(_BaseTypingValue):
def __init__(self, inference_state, parent_context, name, index_value, value_of_index):
super(_WithIndexBase, self).__init__(inference_state, parent_context, name)
self._index_value = index_value
self._value_of_index = value_of_index
def __repr__(self):
return '<%s: %s[%s]>' % (
self.__class__.__name__,
self._tree_name.value,
self._index_context,
self._index_value,
)
class TypingContextWithIndex(_WithIndexBase):
class TypingValueWithIndex(_WithIndexBase):
def execute_annotation(self):
string_name = self._tree_name.value
if string_name == 'Union':
# This is kind of a special case, because we have Unions (in Jedi
# ContextSets).
# ValueSets).
return self.gather_annotation_classes().execute_annotation()
elif string_name == 'Optional':
# Optional is basically just saying it's either None or the actual
# type.
return self.gather_annotation_classes().execute_annotation() \
| ContextSet([builtin_from_name(self.evaluator, u'None')])
| ValueSet([builtin_from_name(self.inference_state, u'None')])
elif string_name == 'Type':
# The type is actually already given in the index_context
return ContextSet([self._index_context])
# The type is actually already given in the index_value
return ValueSet([self._index_value])
elif string_name == 'ClassVar':
# For now don't do anything here, ClassVars are always used.
return self._index_context.execute_annotation()
return self._index_value.execute_annotation()
cls = globals()[string_name]
return ContextSet([cls(
self.evaluator,
return ValueSet([cls(
self.inference_state,
self.parent_context,
self._tree_name,
self._index_context,
self._context_of_index
self._index_value,
self._value_of_index
)])
def gather_annotation_classes(self):
return ContextSet.from_sets(
_iter_over_arguments(self._index_context, self._context_of_index)
return ValueSet.from_sets(
_iter_over_arguments(self._index_value, self._value_of_index)
)
class TypingContext(_BaseTypingContext):
index_class = TypingContextWithIndex
class TypingValue(_BaseTypingValue):
index_class = TypingValueWithIndex
py__simple_getitem__ = None
def py__getitem__(self, index_context_set, contextualized_node):
return ContextSet(
def py__getitem__(self, index_value_set, contextualized_node):
return ValueSet(
self.index_class.create_cached(
self.evaluator,
self.inference_state,
self.parent_context,
self._tree_name,
index_context,
context_of_index=contextualized_node.context)
for index_context in index_context_set
index_value,
value_of_index=contextualized_node.context)
for index_value in index_value_set
)
class _TypingClassMixin(object):
class _TypingClassMixin(ClassMixin):
def py__bases__(self):
return [LazyKnownContexts(
self.evaluator.builtins_module.py__getattribute__('object')
return [LazyKnownValues(
self.inference_state.builtins_module.py__getattribute__('object')
)]
def get_metaclasses(self):
return []
@property
def name(self):
return ValueName(self, self._tree_name)
class TypingClassContextWithIndex(_TypingClassMixin, TypingContextWithIndex, ClassMixin):
class TypingClassValueWithIndex(_TypingClassMixin, TypingValueWithIndex):
pass
class TypingClassContext(_TypingClassMixin, TypingContext, ClassMixin):
index_class = TypingClassContextWithIndex
class TypingClassValue(_TypingClassMixin, TypingValue):
index_class = TypingClassValueWithIndex
def _iter_over_arguments(maybe_tuple_context, defining_context):
def _iter_over_arguments(maybe_tuple_value, defining_context):
def iterate():
if isinstance(maybe_tuple_context, SequenceLiteralContext):
for lazy_context in maybe_tuple_context.py__iter__(contextualized_node=None):
yield lazy_context.infer()
if isinstance(maybe_tuple_value, SequenceLiteralValue):
for lazy_value in maybe_tuple_value.py__iter__(contextualized_node=None):
yield lazy_value.infer()
else:
yield ContextSet([maybe_tuple_context])
yield ValueSet([maybe_tuple_value])
def resolve_forward_references(context_set):
for context in context_set:
if is_string(context):
from jedi.evaluate.gradual.annotation import _get_forward_reference_node
node = _get_forward_reference_node(defining_context, context.get_safe_value())
def resolve_forward_references(value_set):
for value in value_set:
if is_string(value):
from jedi.inference.gradual.annotation import _get_forward_reference_node
node = _get_forward_reference_node(defining_context, value.get_safe_value())
if node is not None:
for c in defining_context.eval_node(node):
for c in defining_context.infer_node(node):
yield c
else:
yield context
yield value
for context_set in iterate():
yield ContextSet(resolve_forward_references(context_set))
for value_set in iterate():
yield ValueSet(resolve_forward_references(value_set))
class TypeAlias(LazyContextWrapper):
class TypeAlias(LazyValueWrapper):
def __init__(self, parent_context, origin_tree_name, actual):
self.evaluator = parent_context.evaluator
self.inference_state = parent_context.inference_state
self.parent_context = parent_context
self._origin_tree_name = origin_tree_name
self._actual = actual # e.g. builtins.list
@property
def name(self):
return ContextName(self, self._origin_tree_name)
return ValueName(self, self._origin_tree_name)
def py__name__(self):
return self.name.string_name
@@ -261,15 +266,15 @@ class TypeAlias(LazyContextWrapper):
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self._actual)
def _get_wrapped_context(self):
def _get_wrapped_value(self):
module_name, class_name = self._actual.split('.')
if self.evaluator.environment.version_info.major == 2 and module_name == 'builtins':
if self.inference_state.environment.version_info.major == 2 and module_name == 'builtins':
module_name = '__builtin__'
# TODO use evaluator.import_module?
from jedi.evaluate.imports import Importer
# TODO use inference_state.import_module?
from jedi.inference.imports import Importer
module, = Importer(
self.evaluator, [module_name], self.evaluator.builtins_module
self.inference_state, [module_name], self.inference_state.builtins_module
).follow()
classes = module.py__getattribute__(class_name)
# There should only be one, because it's code that we control.
@@ -279,56 +284,56 @@ class TypeAlias(LazyContextWrapper):
class _ContainerBase(_WithIndexBase):
def _get_getitem_contexts(self, index):
args = _iter_over_arguments(self._index_context, self._context_of_index)
for i, contexts in enumerate(args):
def _get_getitem_values(self, index):
args = _iter_over_arguments(self._index_value, self._value_of_index)
for i, values in enumerate(args):
if i == index:
return contexts
return values
debug.warning('No param #%s found for annotation %s', index, self._index_context)
return NO_CONTEXTS
debug.warning('No param #%s found for annotation %s', index, self._index_value)
return NO_VALUES
class Callable(_ContainerBase):
def py__call__(self, arguments):
# The 0th index are the arguments.
return self._get_getitem_contexts(1).execute_annotation()
return self._get_getitem_values(1).execute_annotation()
class Tuple(_ContainerBase):
def _is_homogenous(self):
# To specify a variable-length tuple of homogeneous type, Tuple[T, ...]
# is used.
if isinstance(self._index_context, SequenceLiteralContext):
entries = self._index_context.get_tree_entries()
if isinstance(self._index_value, SequenceLiteralValue):
entries = self._index_value.get_tree_entries()
if len(entries) == 2 and entries[1] == '...':
return True
return False
def py__simple_getitem__(self, index):
if self._is_homogenous():
return self._get_getitem_contexts(0).execute_annotation()
return self._get_getitem_values(0).execute_annotation()
else:
if isinstance(index, int):
return self._get_getitem_contexts(index).execute_annotation()
return self._get_getitem_values(index).execute_annotation()
debug.dbg('The getitem type on Tuple was %s' % index)
return NO_CONTEXTS
return NO_VALUES
def py__iter__(self, contextualized_node=None):
if self._is_homogenous():
yield LazyKnownContexts(self._get_getitem_contexts(0).execute_annotation())
yield LazyKnownValues(self._get_getitem_values(0).execute_annotation())
else:
if isinstance(self._index_context, SequenceLiteralContext):
for i in range(self._index_context.py__len__()):
yield LazyKnownContexts(self._get_getitem_contexts(i).execute_annotation())
if isinstance(self._index_value, SequenceLiteralValue):
for i in range(self._index_value.py__len__()):
yield LazyKnownValues(self._get_getitem_values(i).execute_annotation())
def py__getitem__(self, index_context_set, contextualized_node):
def py__getitem__(self, index_value_set, contextualized_node):
if self._is_homogenous():
return self._get_getitem_contexts(0).execute_annotation()
return self._get_getitem_values(0).execute_annotation()
return ContextSet.from_sets(
_iter_over_arguments(self._index_context, self._context_of_index)
return ValueSet.from_sets(
_iter_over_arguments(self._index_value, self._value_of_index)
).execute_annotation()
@@ -340,49 +345,49 @@ class Protocol(_ContainerBase):
pass
class Any(_BaseTypingContext):
class Any(_BaseTypingValue):
def execute_annotation(self):
debug.warning('Used Any - returned no results')
return NO_CONTEXTS
return NO_VALUES
class TypeVarClass(_BaseTypingContext):
class TypeVarClass(_BaseTypingValue):
def py__call__(self, arguments):
unpacked = arguments.unpack()
key, lazy_context = next(unpacked, (None, None))
var_name = self._find_string_name(lazy_context)
key, lazy_value = next(unpacked, (None, None))
var_name = self._find_string_name(lazy_value)
# The name must be given, otherwise it's useless.
if var_name is None or key is not None:
debug.warning('Found a variable without a name %s', arguments)
return NO_CONTEXTS
return NO_VALUES
return ContextSet([TypeVar.create_cached(
self.evaluator,
return ValueSet([TypeVar.create_cached(
self.inference_state,
self.parent_context,
self._tree_name,
var_name,
unpacked
)])
def _find_string_name(self, lazy_context):
if lazy_context is None:
def _find_string_name(self, lazy_value):
if lazy_value is None:
return None
context_set = lazy_context.infer()
if not context_set:
value_set = lazy_value.infer()
if not value_set:
return None
if len(context_set) > 1:
debug.warning('Found multiple contexts for a type variable: %s', context_set)
if len(value_set) > 1:
debug.warning('Found multiple values for a type variable: %s', value_set)
name_context = next(iter(context_set))
name_value = next(iter(value_set))
try:
method = name_context.get_safe_value
method = name_value.get_safe_value
except AttributeError:
return None
else:
safe_value = method(default=None)
if self.evaluator.environment.version_info.major == 2:
if self.inference_state.environment.version_info.major == 2:
if isinstance(safe_value, bytes):
return force_unicode(safe_value)
if isinstance(safe_value, (str, unicode)):
@@ -390,25 +395,25 @@ class TypeVarClass(_BaseTypingContext):
return None
class TypeVar(_BaseTypingContext):
def __init__(self, evaluator, parent_context, tree_name, var_name, unpacked_args):
super(TypeVar, self).__init__(evaluator, parent_context, tree_name)
class TypeVar(_BaseTypingValue):
def __init__(self, inference_state, parent_context, tree_name, var_name, unpacked_args):
super(TypeVar, self).__init__(inference_state, parent_context, tree_name)
self._var_name = var_name
self._constraints_lazy_contexts = []
self._bound_lazy_context = None
self._covariant_lazy_context = None
self._contravariant_lazy_context = None
for key, lazy_context in unpacked_args:
self._constraints_lazy_values = []
self._bound_lazy_value = None
self._covariant_lazy_value = None
self._contravariant_lazy_value = None
for key, lazy_value in unpacked_args:
if key is None:
self._constraints_lazy_contexts.append(lazy_context)
self._constraints_lazy_values.append(lazy_value)
else:
if key == 'bound':
self._bound_lazy_context = lazy_context
self._bound_lazy_value = lazy_value
elif key == 'covariant':
self._covariant_lazy_context = lazy_context
self._covariant_lazy_value = lazy_value
elif key == 'contravariant':
self._contra_variant_lazy_context = lazy_context
self._contra_variant_lazy_value = lazy_value
else:
debug.warning('Invalid TypeVar param name %s', key)
@@ -419,12 +424,12 @@ class TypeVar(_BaseTypingContext):
return iter([])
def _get_classes(self):
if self._bound_lazy_context is not None:
return self._bound_lazy_context.infer()
if self._constraints_lazy_contexts:
if self._bound_lazy_value is not None:
return self._bound_lazy_value.infer()
if self._constraints_lazy_values:
return self.constraints
debug.warning('Tried to infer the TypeVar %s without a given type', self._var_name)
return NO_CONTEXTS
return NO_VALUES
def is_same_class(self, other):
# Everything can match an undefined type var.
@@ -432,8 +437,8 @@ class TypeVar(_BaseTypingContext):
@property
def constraints(self):
return ContextSet.from_sets(
lazy.infer() for lazy in self._constraints_lazy_contexts
return ValueSet.from_sets(
lazy.infer() for lazy in self._constraints_lazy_values
)
def define_generics(self, type_var_dict):
@@ -444,7 +449,7 @@ class TypeVar(_BaseTypingContext):
else:
if found:
return found
return self._get_classes() or ContextSet({self})
return self._get_classes() or ValueSet({self})
def execute_annotation(self):
return self._get_classes().execute_annotation()
@@ -453,70 +458,70 @@ class TypeVar(_BaseTypingContext):
return '<%s: %s>' % (self.__class__.__name__, self.py__name__())
class OverloadFunction(_BaseTypingContext):
class OverloadFunction(_BaseTypingValue):
@repack_with_argument_clinic('func, /')
def py__call__(self, func_context_set):
def py__call__(self, func_value_set):
# Just pass arguments through.
return func_context_set
return func_value_set
class NewTypeFunction(_BaseTypingContext):
class NewTypeFunction(_BaseTypingValue):
def py__call__(self, arguments):
ordered_args = arguments.unpack()
next(ordered_args, (None, None))
_, second_arg = next(ordered_args, (None, None))
if second_arg is None:
return NO_CONTEXTS
return ContextSet(
return NO_VALUES
return ValueSet(
NewType(
self.evaluator,
self.inference_state,
contextualized_node.context,
contextualized_node.node,
second_arg.infer(),
) for contextualized_node in arguments.get_calling_nodes())
class NewType(Context):
def __init__(self, evaluator, parent_context, tree_node, type_context_set):
super(NewType, self).__init__(evaluator, parent_context)
self._type_context_set = type_context_set
class NewType(Value):
def __init__(self, inference_state, parent_context, tree_node, type_value_set):
super(NewType, self).__init__(inference_state, parent_context)
self._type_value_set = type_value_set
self.tree_node = tree_node
def py__call__(self, arguments):
return self._type_context_set.execute_annotation()
return self._type_value_set.execute_annotation()
class CastFunction(_BaseTypingContext):
class CastFunction(_BaseTypingValue):
@repack_with_argument_clinic('type, object, /')
def py__call__(self, type_context_set, object_context_set):
return type_context_set.execute_annotation()
def py__call__(self, type_value_set, object_value_set):
return type_value_set.execute_annotation()
class BoundTypeVarName(AbstractNameDefinition):
"""
This type var was bound to a certain type, e.g. int.
"""
def __init__(self, type_var, context_set):
def __init__(self, type_var, value_set):
self._type_var = type_var
self.parent_context = type_var.parent_context
self._context_set = context_set
self._value_set = value_set
def infer(self):
def iter_():
for context in self._context_set:
for value in self._value_set:
# Replace any with the constraints if they are there.
if isinstance(context, Any):
if isinstance(value, Any):
for constraint in self._type_var.constraints:
yield constraint
else:
yield context
return ContextSet(iter_())
yield value
return ValueSet(iter_())
def py__name__(self):
return self._type_var.py__name__()
def __repr__(self):
return '<%s %s -> %s>' % (self.__class__.__name__, self.py__name__(), self._context_set)
return '<%s %s -> %s>' % (self.__class__.__name__, self.py__name__(), self._value_set)
class TypeVarFilter(object):
@@ -549,22 +554,22 @@ class TypeVarFilter(object):
return []
class AbstractAnnotatedClass(ClassMixin, ContextWrapper):
def get_type_var_filter(self):
return TypeVarFilter(self.get_generics(), self.list_type_vars())
def get_filters(self, search_global=False, *args, **kwargs):
filters = super(AbstractAnnotatedClass, self).get_filters(
search_global,
class AnnotatedClassContext(ClassContext):
def get_filters(self, *args, **kwargs):
filters = super(AnnotatedClassContext, self).get_filters(
*args, **kwargs
)
for f in filters:
yield f
if search_global:
# The type vars can only be looked up if it's a global search and
# not a direct lookup on the class.
yield self.get_type_var_filter()
# The type vars can only be looked up if it's a global search and
# not a direct lookup on the class.
yield self._value.get_type_var_filter()
class AbstractAnnotatedClass(ClassMixin, ValueWrapper):
def get_type_var_filter(self):
return TypeVarFilter(self.get_generics(), self.list_type_vars())
def is_same_class(self, other):
if not isinstance(other, AbstractAnnotatedClass):
@@ -593,7 +598,7 @@ class AbstractAnnotatedClass(ClassMixin, ContextWrapper):
def py__call__(self, arguments):
instance, = super(AbstractAnnotatedClass, self).py__call__(arguments)
return ContextSet([InstanceWrapper(instance)])
return ValueSet([InstanceWrapper(instance)])
def get_generics(self):
raise NotImplementedError
@@ -602,55 +607,58 @@ class AbstractAnnotatedClass(ClassMixin, ContextWrapper):
changed = False
new_generics = []
for generic_set in self.get_generics():
contexts = NO_CONTEXTS
values = NO_VALUES
for generic in generic_set:
if isinstance(generic, (AbstractAnnotatedClass, TypeVar)):
result = generic.define_generics(type_var_dict)
contexts |= result
if result != ContextSet({generic}):
values |= result
if result != ValueSet({generic}):
changed = True
else:
contexts |= ContextSet([generic])
new_generics.append(contexts)
values |= ValueSet([generic])
new_generics.append(values)
if not changed:
# There might not be any type vars that change. In that case just
# return itself, because it does not make sense to potentially lose
# cached results.
return ContextSet([self])
return ValueSet([self])
return ContextSet([GenericClass(
self._wrapped_context,
return ValueSet([GenericClass(
self._wrapped_value,
generics=tuple(new_generics)
)])
def _as_context(self):
return AnnotatedClassContext(self)
def __repr__(self):
return '<%s: %s%s>' % (
self.__class__.__name__,
self._wrapped_context,
self._wrapped_value,
list(self.get_generics()),
)
@to_list
def py__bases__(self):
for base in self._wrapped_context.py__bases__():
for base in self._wrapped_value.py__bases__():
yield LazyAnnotatedBaseClass(self, base)
class LazyGenericClass(AbstractAnnotatedClass):
def __init__(self, class_context, index_context, context_of_index):
super(LazyGenericClass, self).__init__(class_context)
self._index_context = index_context
self._context_of_index = context_of_index
def __init__(self, class_value, index_value, value_of_index):
super(LazyGenericClass, self).__init__(class_value)
self._index_value = index_value
self._value_of_index = value_of_index
@evaluator_method_cache()
@inference_state_method_cache()
def get_generics(self):
return list(_iter_over_arguments(self._index_context, self._context_of_index))
return list(_iter_over_arguments(self._index_value, self._value_of_index))
class GenericClass(AbstractAnnotatedClass):
def __init__(self, class_context, generics):
super(GenericClass, self).__init__(class_context)
def __init__(self, class_value, generics):
super(GenericClass, self).__init__(class_value)
self._generics = generics
def get_generics(self):
@@ -658,44 +666,44 @@ class GenericClass(AbstractAnnotatedClass):
class LazyAnnotatedBaseClass(object):
def __init__(self, class_context, lazy_base_class):
self._class_context = class_context
def __init__(self, class_value, lazy_base_class):
self._class_value = class_value
self._lazy_base_class = lazy_base_class
@iterator_to_context_set
@iterator_to_value_set
def infer(self):
for base in self._lazy_base_class.infer():
if isinstance(base, AbstractAnnotatedClass):
# Here we have to recalculate the given types.
yield GenericClass.create_cached(
base.evaluator,
base._wrapped_context,
base.inference_state,
base._wrapped_value,
tuple(self._remap_type_vars(base)),
)
else:
yield base
def _remap_type_vars(self, base):
filter = self._class_context.get_type_var_filter()
filter = self._class_value.get_type_var_filter()
for type_var_set in base.get_generics():
new = NO_CONTEXTS
new = NO_VALUES
for type_var in type_var_set:
if isinstance(type_var, TypeVar):
names = filter.get(type_var.py__name__())
new |= ContextSet.from_sets(
new |= ValueSet.from_sets(
name.infer() for name in names
)
else:
# Mostly will be type vars, except if in some cases
# a concrete type will already be there. In that
# case just add it to the context set.
new |= ContextSet([type_var])
# case just add it to the value set.
new |= ValueSet([type_var])
yield new
class InstanceWrapper(ContextWrapper):
class InstanceWrapper(ValueWrapper):
def py__stop_iteration_returns(self):
for cls in self._wrapped_context.class_context.py__mro__():
for cls in self._wrapped_value.class_value.py__mro__():
if cls.py__name__() == 'Generator':
generics = cls.get_generics()
try:
@@ -703,5 +711,5 @@ class InstanceWrapper(ContextWrapper):
except IndexError:
pass
elif cls.py__name__() == 'Iterator':
return ContextSet([builtin_from_name(self.evaluator, u'None')])
return self._wrapped_context.py__stop_iteration_returns()
return ValueSet([builtin_from_name(self.inference_state, u'None')])
return self._wrapped_value.py__stop_iteration_returns()
jedi/evaluate/gradual/utils.py → jedi/inference/gradual/utils.py
+6 -6
View File
@@ -1,9 +1,9 @@
import os
from jedi.evaluate.gradual.typeshed import TYPESHED_PATH, create_stub_module
from jedi.inference.gradual.typeshed import TYPESHED_PATH, create_stub_module
def load_proper_stub_module(evaluator, file_io, import_names, module_node):
def load_proper_stub_module(inference_state, file_io, import_names, module_node):
"""
This function is given a random .pyi file and should return the proper
module.
@@ -20,13 +20,13 @@ def load_proper_stub_module(evaluator, file_io, import_names, module_node):
import_names = import_names[:-1]
if import_names is not None:
actual_context_set = evaluator.import_module(import_names, prefer_stubs=False)
if not actual_context_set:
actual_value_set = inference_state.import_module(import_names, prefer_stubs=False)
if not actual_value_set:
return None
stub = create_stub_module(
evaluator, actual_context_set, module_node, file_io, import_names
inference_state, actual_value_set, module_node, file_io, import_names
)
evaluator.stub_module_cache[import_names] = stub
inference_state.stub_module_cache[import_names] = stub
return stub
return None
jedi/evaluate/helpers.py → jedi/inference/helpers.py
+23 -33
View File
@@ -44,7 +44,7 @@ def deep_ast_copy(obj):
return new_obj
def evaluate_call_of_leaf(context, leaf, cut_own_trailer=False):
def infer_call_of_leaf(context, leaf, cut_own_trailer=False):
"""
Creates a "call" node that consist of all ``trailer`` and ``power``
objects. E.g. if you call it with ``append``::
@@ -65,16 +65,16 @@ def evaluate_call_of_leaf(context, leaf, cut_own_trailer=False):
"""
trailer = leaf.parent
if trailer.type == 'fstring':
from jedi.evaluate import compiled
return compiled.get_string_context_set(context.evaluator)
from jedi.inference import compiled
return compiled.get_string_value_set(context.inference_state)
# The leaf may not be the last or first child, because there exist three
# different trailers: `( x )`, `[ x ]` and `.x`. In the first two examples
# we should not match anything more than x.
if trailer.type != 'trailer' or leaf not in (trailer.children[0], trailer.children[-1]):
if trailer.type == 'atom':
return context.eval_node(trailer)
return context.eval_node(leaf)
return context.infer_node(trailer)
return context.infer_node(leaf)
power = trailer.parent
index = power.children.index(trailer)
@@ -99,10 +99,10 @@ def evaluate_call_of_leaf(context, leaf, cut_own_trailer=False):
base = trailers[0]
trailers = trailers[1:]
values = context.eval_node(base)
from jedi.evaluate.syntax_tree import eval_trailer
values = context.infer_node(base)
from jedi.inference.syntax_tree import infer_trailer
for trailer in trailers:
values = eval_trailer(context, values, trailer)
values = infer_trailer(context, values, trailer)
return values
@@ -184,44 +184,34 @@ def get_module_names(module, all_scopes):
return names
@contextmanager
def predefine_names(context, flow_scope, dct):
predefined = context.predefined_names
predefined[flow_scope] = dct
try:
yield
finally:
del predefined[flow_scope]
def is_string(context):
if context.evaluator.environment.version_info.major == 2:
def is_string(value):
if value.inference_state.environment.version_info.major == 2:
str_classes = (unicode, bytes)
else:
str_classes = (unicode,)
return context.is_compiled() and isinstance(context.get_safe_value(default=None), str_classes)
return value.is_compiled() and isinstance(value.get_safe_value(default=None), str_classes)
def is_literal(context):
return is_number(context) or is_string(context)
def is_literal(value):
return is_number(value) or is_string(value)
def _get_safe_value_or_none(context, accept):
value = context.get_safe_value(default=None)
def _get_safe_value_or_none(value, accept):
value = value.get_safe_value(default=None)
if isinstance(value, accept):
return value
def get_int_or_none(context):
return _get_safe_value_or_none(context, int)
def get_int_or_none(value):
return _get_safe_value_or_none(value, int)
def get_str_or_none(context):
return _get_safe_value_or_none(context, (bytes, unicode))
def get_str_or_none(value):
return _get_safe_value_or_none(value, (bytes, unicode))
def is_number(context):
return _get_safe_value_or_none(context, (int, float)) is not None
def is_number(value):
return _get_safe_value_or_none(value, (int, float)) is not None
class SimpleGetItemNotFound(Exception):
@@ -265,5 +255,5 @@ def parse_dotted_names(nodes, is_import_from, until_node=None):
return level, names
def contexts_from_qualified_names(evaluator, *names):
return evaluator.import_module(names[:-1]).py__getattribute__(names[-1])
def values_from_qualified_names(inference_state, *names):
return inference_state.import_module(names[:-1]).py__getattribute__(names[-1])
jedi/evaluate/imports.py → jedi/inference/imports.py
+163 -151
View File
@@ -1,5 +1,5 @@
"""
:mod:`jedi.evaluate.imports` is here to resolve import statements and return
:mod:`jedi.inference.imports` is here to resolve import statements and return
the modules/classes/functions/whatever, which they stand for. However there's
not any actual importing done. This module is about finding modules in the
filesystem. This can be quite tricky sometimes, because Python imports are not
@@ -23,16 +23,16 @@ from jedi import debug
from jedi import settings
from jedi.file_io import KnownContentFileIO, FileIO
from jedi.parser_utils import get_cached_code_lines
from jedi.evaluate import sys_path
from jedi.evaluate import helpers
from jedi.evaluate import compiled
from jedi.evaluate import analysis
from jedi.evaluate.utils import unite
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.names import ImportName, SubModuleName
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS
from jedi.evaluate.gradual.typeshed import import_module_decorator
from jedi.evaluate.context.module import iter_module_names
from jedi.inference import sys_path
from jedi.inference import helpers
from jedi.inference import compiled
from jedi.inference import analysis
from jedi.inference.utils import unite
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.names import ImportName, SubModuleName
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.inference.gradual.typeshed import import_module_decorator
from jedi.inference.value.module import iter_module_names
from jedi.plugins import plugin_manager
@@ -41,11 +41,11 @@ class ModuleCache(object):
self._path_cache = {}
self._name_cache = {}
def add(self, string_names, context_set):
def add(self, string_names, value_set):
#path = module.py__file__()
#self._path_cache[path] = context_set
#self._path_cache[path] = value_set
if string_names is not None:
self._name_cache[string_names] = context_set
self._name_cache[string_names] = value_set
def get(self, string_names):
return self._name_cache[string_names]
@@ -56,13 +56,59 @@ class ModuleCache(object):
# This memoization is needed, because otherwise we will infinitely loop on
# certain imports.
@evaluator_method_cache(default=NO_CONTEXTS)
def infer_import(context, tree_name, is_goto=False):
@inference_state_method_cache(default=NO_VALUES)
def infer_import(context, tree_name):
module_context = context.get_root_context()
from_import_name, import_path, level, values = \
_prepare_infer_import(module_context, tree_name)
if not values:
return NO_VALUES
if from_import_name is not None:
values = values.py__getattribute__(
from_import_name,
name_context=context,
analysis_errors=False
)
if not values:
path = import_path + (from_import_name,)
importer = Importer(context.inference_state, path, module_context, level)
values = importer.follow()
debug.dbg('after import: %s', values)
return values
@inference_state_method_cache(default=[])
def goto_import(context, tree_name):
module_context = context.get_root_context()
from_import_name, import_path, level, values = \
_prepare_infer_import(module_context, tree_name)
if not values:
return []
if from_import_name is not None:
names = unite([
c.goto(
from_import_name,
name_context=context,
analysis_errors=False
) for c in values
])
# Avoid recursion on the same names.
if names and not any(n.tree_name is tree_name for n in names):
return names
path = import_path + (from_import_name,)
importer = Importer(context.inference_state, path, module_context, level)
values = importer.follow()
return set(s.name for s in values)
def _prepare_infer_import(module_context, tree_name):
import_node = search_ancestor(tree_name, 'import_name', 'import_from')
import_path = import_node.get_path_for_name(tree_name)
from_import_name = None
evaluator = context.evaluator
try:
from_names = import_node.get_from_names()
except AttributeError:
@@ -75,45 +121,12 @@ def infer_import(context, tree_name, is_goto=False):
from_import_name = import_path[-1]
import_path = from_names
importer = Importer(evaluator, tuple(import_path),
importer = Importer(module_context.inference_state, tuple(import_path),
module_context, import_node.level)
types = importer.follow()
#if import_node.is_nested() and not self.nested_resolve:
# scopes = [NestedImportModule(module, import_node)]
if not types:
return NO_CONTEXTS
if from_import_name is not None:
types = unite(
t.py__getattribute__(
from_import_name,
name_context=context,
is_goto=is_goto,
analysis_errors=False
)
for t in types
)
if not is_goto:
types = ContextSet(types)
if not types:
path = import_path + [from_import_name]
importer = Importer(evaluator, tuple(path),
module_context, import_node.level)
types = importer.follow()
# goto only accepts `Name`
if is_goto:
types = set(s.name for s in types)
else:
# goto only accepts `Name`
if is_goto:
types = set(s.name for s in types)
debug.dbg('after import: %s', types)
return types
return from_import_name, tuple(import_path), import_node.level, importer.follow()
class NestedImportModule(tree.Module):
@@ -148,9 +161,9 @@ class NestedImportModule(tree.Module):
self._nested_import)
def _add_error(context, name, message):
if hasattr(name, 'parent') and context is not None:
analysis.add(context, 'import-error', name, message)
def _add_error(value, name, message):
if hasattr(name, 'parent') and value is not None:
analysis.add(value, 'import-error', name, message)
else:
debug.warning('ImportError without origin: ' + message)
@@ -183,7 +196,7 @@ def _level_to_base_import_path(project_path, directory, level):
class Importer(object):
def __init__(self, evaluator, import_path, module_context, level=0):
def __init__(self, inference_state, import_path, module_context, level=0):
"""
An implementation similar to ``__import__``. Use `follow`
to actually follow the imports.
@@ -197,12 +210,12 @@ class Importer(object):
:param import_path: List of namespaces (strings or Names).
"""
debug.speed('import %s %s' % (import_path, module_context))
self._evaluator = evaluator
self._inference_state = inference_state
self.level = level
self.module_context = module_context
self._module_context = module_context
self._fixed_sys_path = None
self._inference_possible = True
self._infer_possible = True
if level:
base = module_context.py__package__()
# We need to care for two cases, the first one is if it's a valid
@@ -233,12 +246,12 @@ class Importer(object):
directory = os.path.dirname(path)
base_import_path, base_directory = _level_to_base_import_path(
self._evaluator.project._path, directory, level,
self._inference_state.project._path, directory, level,
)
if base_directory is None:
# Everything is lost, the relative import does point
# somewhere out of the filesystem.
self._inference_possible = False
self._infer_possible = False
else:
self._fixed_sys_path = [force_unicode(base_directory)]
@@ -265,12 +278,12 @@ class Importer(object):
return self._fixed_sys_path
sys_path_mod = (
self._evaluator.get_sys_path()
+ sys_path.check_sys_path_modifications(self.module_context)
self._inference_state.get_sys_path()
+ sys_path.check_sys_path_modifications(self._module_context)
)
if self._evaluator.environment.version_info.major == 2:
file_path = self.module_context.py__file__()
if self._inference_state.environment.version_info.major == 2:
file_path = self._module_context.py__file__()
if file_path is not None:
# Python2 uses an old strange way of importing relative imports.
sys_path_mod.append(force_unicode(os.path.dirname(file_path)))
@@ -278,8 +291,8 @@ class Importer(object):
return sys_path_mod
def follow(self):
if not self.import_path or not self._inference_possible:
return NO_CONTEXTS
if not self.import_path or not self._infer_possible:
return NO_VALUES
import_names = tuple(
force_unicode(i.value if isinstance(i, tree.Name) else i)
@@ -287,20 +300,20 @@ class Importer(object):
)
sys_path = self._sys_path_with_modifications()
context_set = [None]
value_set = [None]
for i, name in enumerate(self.import_path):
context_set = ContextSet.from_sets([
self._evaluator.import_module(
value_set = ValueSet.from_sets([
self._inference_state.import_module(
import_names[:i+1],
parent_module_context,
parent_module_value,
sys_path
) for parent_module_context in context_set
) for parent_module_value in value_set
])
if not context_set:
if not value_set:
message = 'No module named ' + '.'.join(import_names)
_add_error(self.module_context, name, message)
return NO_CONTEXTS
return context_set
_add_error(self._module_context, name, message)
return NO_VALUES
return value_set
def _get_module_names(self, search_path=None, in_module=None):
"""
@@ -310,26 +323,26 @@ class Importer(object):
names = []
# add builtin module names
if search_path is None and in_module is None:
names += [ImportName(self.module_context, name)
for name in self._evaluator.compiled_subprocess.get_builtin_module_names()]
names += [ImportName(self._module_context, name)
for name in self._inference_state.compiled_subprocess.get_builtin_module_names()]
if search_path is None:
search_path = self._sys_path_with_modifications()
for name in iter_module_names(self._evaluator, search_path):
for name in iter_module_names(self._inference_state, search_path):
if in_module is None:
n = ImportName(self.module_context, name)
n = ImportName(self._module_context, name)
else:
n = SubModuleName(in_module, name)
names.append(n)
return names
def completion_names(self, evaluator, only_modules=False):
def completion_names(self, inference_state, only_modules=False):
"""
:param only_modules: Indicates wheter it's possible to import a
definition that is not defined in a module.
"""
if not self._inference_possible:
if not self._infer_possible:
return []
names = []
@@ -341,26 +354,26 @@ class Importer(object):
modname = mod.string_name
if modname.startswith('flask_'):
extname = modname[len('flask_'):]
names.append(ImportName(self.module_context, extname))
names.append(ImportName(self._module_context, extname))
# Now the old style: ``flaskext.foo``
for dir in self._sys_path_with_modifications():
flaskext = os.path.join(dir, 'flaskext')
if os.path.isdir(flaskext):
names += self._get_module_names([flaskext])
contexts = self.follow()
for context in contexts:
values = self.follow()
for value in values:
# Non-modules are not completable.
if context.api_type != 'module': # not a module
if value.api_type != 'module': # not a module
continue
names += context.sub_modules_dict().values()
names += value.sub_modules_dict().values()
if not only_modules:
from jedi.evaluate.gradual.conversion import convert_contexts
from jedi.inference.gradual.conversion import convert_values
both_contexts = contexts | convert_contexts(contexts)
for c in both_contexts:
for filter in c.get_filters(search_global=False):
both_values = values | convert_values(values)
for c in both_values:
for filter in c.get_filters():
names += filter.values()
else:
if self.level:
@@ -374,108 +387,106 @@ class Importer(object):
@plugin_manager.decorate()
@import_module_decorator
def import_module(evaluator, import_names, parent_module_context, sys_path):
def import_module(inference_state, import_names, parent_module_value, sys_path):
"""
This method is very similar to importlib's `_gcd_import`.
"""
if import_names[0] in settings.auto_import_modules:
module = _load_builtin_module(evaluator, import_names, sys_path)
module = _load_builtin_module(inference_state, import_names, sys_path)
if module is None:
return NO_CONTEXTS
return ContextSet([module])
return NO_VALUES
return ValueSet([module])
module_name = '.'.join(import_names)
if parent_module_context is None:
if parent_module_value is None:
# Override the sys.path. It works only good that way.
# Injecting the path directly into `find_module` did not work.
file_io_or_ns, is_pkg = evaluator.compiled_subprocess.get_module_info(
file_io_or_ns, is_pkg = inference_state.compiled_subprocess.get_module_info(
string=import_names[-1],
full_name=module_name,
sys_path=sys_path,
is_global_search=True,
)
if is_pkg is None:
return NO_CONTEXTS
return NO_VALUES
else:
try:
method = parent_module_context.py__path__
except AttributeError:
# The module is not a package.
return NO_CONTEXTS
paths = parent_module_value.py__path__()
if paths is None:
# The module might not be a package.
return NO_VALUES
for path in paths:
# At the moment we are only using one path. So this is
# not important to be correct.
if not isinstance(path, list):
path = [path]
file_io_or_ns, is_pkg = inference_state.compiled_subprocess.get_module_info(
string=import_names[-1],
path=path,
full_name=module_name,
is_global_search=False,
)
if is_pkg is not None:
break
else:
paths = method()
for path in paths:
# At the moment we are only using one path. So this is
# not important to be correct.
if not isinstance(path, list):
path = [path]
file_io_or_ns, is_pkg = evaluator.compiled_subprocess.get_module_info(
string=import_names[-1],
path=path,
full_name=module_name,
is_global_search=False,
)
if is_pkg is not None:
break
else:
return NO_CONTEXTS
return NO_VALUES
if isinstance(file_io_or_ns, ImplicitNSInfo):
from jedi.evaluate.context.namespace import ImplicitNamespaceContext
module = ImplicitNamespaceContext(
evaluator,
from jedi.inference.value.namespace import ImplicitNamespaceValue
module = ImplicitNamespaceValue(
inference_state,
fullname=file_io_or_ns.name,
paths=file_io_or_ns.paths,
)
elif file_io_or_ns is None:
module = _load_builtin_module(evaluator, import_names, sys_path)
module = _load_builtin_module(inference_state, import_names, sys_path)
if module is None:
return NO_CONTEXTS
return NO_VALUES
else:
module = _load_python_module(
evaluator, file_io_or_ns, sys_path,
inference_state, file_io_or_ns, sys_path,
import_names=import_names,
is_package=is_pkg,
)
if parent_module_context is None:
if parent_module_value is None:
debug.dbg('global search_module %s: %s', import_names[-1], module)
else:
debug.dbg('search_module %s in paths %s: %s', module_name, paths, module)
return ContextSet([module])
return ValueSet([module])
def _load_python_module(evaluator, file_io, sys_path=None,
def _load_python_module(inference_state, file_io, sys_path=None,
import_names=None, is_package=False):
try:
return evaluator.module_cache.get_from_path(file_io.path)
return inference_state.module_cache.get_from_path(file_io.path)
except KeyError:
pass
module_node = evaluator.parse(
module_node = inference_state.parse(
file_io=file_io,
cache=True,
diff_cache=settings.fast_parser,
cache_path=settings.cache_directory
)
from jedi.evaluate.context import ModuleContext
return ModuleContext(
evaluator, module_node,
from jedi.inference.value import ModuleValue
return ModuleValue(
inference_state, module_node,
file_io=file_io,
string_names=import_names,
code_lines=get_cached_code_lines(evaluator.grammar, file_io.path),
code_lines=get_cached_code_lines(inference_state.grammar, file_io.path),
is_package=is_package,
)
def _load_builtin_module(evaluator, import_names=None, sys_path=None):
def _load_builtin_module(inference_state, import_names=None, sys_path=None):
if sys_path is None:
sys_path = evaluator.get_sys_path()
sys_path = inference_state.get_sys_path()
dotted_name = '.'.join(import_names)
assert dotted_name is not None
module = compiled.load_module(evaluator, dotted_name=dotted_name, sys_path=sys_path)
module = compiled.load_module(inference_state, dotted_name=dotted_name, sys_path=sys_path)
if module is None:
# The file might raise an ImportError e.g. and therefore not be
# importable.
@@ -483,13 +494,13 @@ def _load_builtin_module(evaluator, import_names=None, sys_path=None):
return module
def _load_module_from_path(evaluator, file_io, base_names):
def _load_module_from_path(inference_state, file_io, base_names):
"""
This should pretty much only be used for get_modules_containing_name. It's
here to ensure that a random path is still properly loaded into the Jedi
module structure.
"""
e_sys_path = evaluator.get_sys_path()
e_sys_path = inference_state.get_sys_path()
path = file_io.path
if base_names:
module_name = os.path.basename(path)
@@ -503,16 +514,16 @@ def _load_module_from_path(evaluator, file_io, base_names):
import_names, is_package = sys_path.transform_path_to_dotted(e_sys_path, path)
module = _load_python_module(
evaluator, file_io,
inference_state, file_io,
sys_path=e_sys_path,
import_names=import_names,
is_package=is_package,
)
evaluator.module_cache.add(import_names, ContextSet([module]))
inference_state.module_cache.add(import_names, ValueSet([module]))
return module
def get_modules_containing_name(evaluator, modules, name):
def get_module_contexts_containing_name(inference_state, module_contexts, name):
"""
Search a name in the directories of modules.
"""
@@ -530,24 +541,25 @@ def get_modules_containing_name(evaluator, modules, name):
if name not in code:
return None
new_file_io = KnownContentFileIO(file_io.path, code)
m = _load_module_from_path(evaluator, new_file_io, base_names)
m = _load_module_from_path(inference_state, new_file_io, base_names)
if isinstance(m, compiled.CompiledObject):
return None
return m
return m.as_context()
# skip non python modules
used_mod_paths = set()
folders_with_names_to_be_checked = []
for m in modules:
if m.file_io is not None:
path = m.file_io.path
if path not in used_mod_paths:
for module_context in module_contexts:
path = module_context.py__file__()
if path not in used_mod_paths:
file_io = module_context.get_value().file_io
if file_io is not None:
used_mod_paths.add(path)
folders_with_names_to_be_checked.append((
m.file_io.get_parent_folder(),
m.py__package__()
file_io.get_parent_folder(),
module_context.py__package__()
))
yield m
yield module_context
if not settings.dynamic_params_for_other_modules:
return
jedi/inference/lazy_value.py
+59
View File
@@ -0,0 +1,59 @@
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.common.utils import monkeypatch
class AbstractLazyValue(object):
def __init__(self, data):
self.data = data
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self.data)
def infer(self):
raise NotImplementedError
class LazyKnownValue(AbstractLazyValue):
"""data is a Value."""
def infer(self):
return ValueSet([self.data])
class LazyKnownValues(AbstractLazyValue):
"""data is a ValueSet."""
def infer(self):
return self.data
class LazyUnknownValue(AbstractLazyValue):
def __init__(self):
super(LazyUnknownValue, self).__init__(None)
def infer(self):
return NO_VALUES
class LazyTreeValue(AbstractLazyValue):
def __init__(self, context, node):
super(LazyTreeValue, self).__init__(node)
self.context = context
# We need to save the predefined names. It's an unfortunate side effect
# that needs to be tracked otherwise results will be wrong.
self._predefined_names = dict(context.predefined_names)
def infer(self):
with monkeypatch(self.context, 'predefined_names', self._predefined_names):
return self.context.infer_node(self.data)
def get_merged_lazy_value(lazy_values):
if len(lazy_values) > 1:
return MergedLazyValues(lazy_values)
else:
return lazy_values[0]
class MergedLazyValues(AbstractLazyValue):
"""data is a list of lazy values."""
def infer(self):
return ValueSet.from_sets(l.infer() for l in self.data)
jedi/evaluate/names.py → jedi/inference/names.py
+70 -50
View File
@@ -3,7 +3,8 @@ from abc import abstractmethod
from parso.tree import search_ancestor
from jedi._compatibility import Parameter
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS
from jedi.inference.base_value import ValueSet, NO_VALUES
from jedi.inference import docstrings
from jedi.cache import memoize_method
@@ -12,7 +13,7 @@ class AbstractNameDefinition(object):
string_name = None
parent_context = None
tree_name = None
is_context_name = True
is_value_name = True
"""
Used for the Jedi API to know if it's a keyword or an actual name.
"""
@@ -44,6 +45,9 @@ class AbstractNameDefinition(object):
def get_root_context(self):
return self.parent_context.get_root_context()
def get_public_name(self):
return self.string_name
def __repr__(self):
if self.start_pos is None:
return '<%s: string_name=%s>' % (self.__class__.__name__, self.string_name)
@@ -64,15 +68,15 @@ class AbstractArbitraryName(AbstractNameDefinition):
string literals, which is not really a name, but for Jedi we use this
concept of Name for completions as well.
"""
is_context_name = False
is_value_name = False
def __init__(self, evaluator, string):
self.evaluator = evaluator
def __init__(self, inference_state, string):
self.inference_state = inference_state
self.string_name = string
self.parent_context = evaluator.builtins_module
self.parent_context = inference_state.builtins_module
def infer(self):
return NO_CONTEXTS
return NO_VALUES
class AbstractTreeName(AbstractNameDefinition):
@@ -103,7 +107,9 @@ class AbstractTreeName(AbstractNameDefinition):
return parent_names + (self.tree_name.value,)
def goto(self, **kwargs):
return self.parent_context.evaluator.goto(self.parent_context, self.tree_name, **kwargs)
return self.parent_context.inference_state.goto(
self.parent_context, self.tree_name, **kwargs
)
def is_import(self):
imp = search_ancestor(self.tree_name, 'import_from', 'import_name')
@@ -118,30 +124,30 @@ class AbstractTreeName(AbstractNameDefinition):
return self.tree_name.start_pos
class ContextNameMixin(object):
class ValueNameMixin(object):
def infer(self):
return ContextSet([self._context])
return ValueSet([self._value])
def _get_qualified_names(self):
return self._context.get_qualified_names()
return self._value.get_qualified_names()
def get_root_context(self):
if self.parent_context is None: # A module
return self._context
return super(ContextNameMixin, self).get_root_context()
return self._value.as_context()
return super(ValueNameMixin, self).get_root_context()
@property
def api_type(self):
return self._context.api_type
return self._value.api_type
class ContextName(ContextNameMixin, AbstractTreeName):
def __init__(self, context, tree_name):
super(ContextName, self).__init__(context.parent_context, tree_name)
self._context = context
class ValueName(ValueNameMixin, AbstractTreeName):
def __init__(self, value, tree_name):
super(ValueName, self).__init__(value.parent_context, tree_name)
self._value = value
def goto(self):
return ContextSet([self._context.name])
return ValueSet([self._value.name])
class TreeNameDefinition(AbstractTreeName):
@@ -155,9 +161,12 @@ class TreeNameDefinition(AbstractTreeName):
def infer(self):
# Refactor this, should probably be here.
from jedi.evaluate.syntax_tree import tree_name_to_contexts
parent = self.parent_context
return tree_name_to_contexts(parent.evaluator, parent, self.tree_name)
from jedi.inference.syntax_tree import tree_name_to_values
return tree_name_to_values(
self.parent_context.inference_state,
self.parent_context,
self.tree_name
)
@property
def api_type(self):
@@ -198,10 +207,15 @@ class ParamNameInterface(_ParamMixin):
def to_string(self):
raise NotImplementedError
def get_param(self):
# TODO document better where this is used and when. Currently it has
# very limited use, but is still in use. It's currently not even
# clear what values would be allowed.
def get_executed_param_name(self):
"""
For dealing with type inference and working around the graph, we
sometimes want to have the param name of the execution. This feels a
bit strange and we might have to refactor at some point.
For now however it exists to avoid infering params when we don't really
need them (e.g. when we can just instead use annotations.
"""
return None
@property
@@ -219,7 +233,7 @@ class BaseTreeParamName(ParamNameInterface, AbstractTreeName):
default_node = None
def to_string(self):
output = self._kind_string() + self.string_name
output = self._kind_string() + self.get_public_name()
annotation = self.annotation_node
default = self.default_node
if annotation is not None:
@@ -237,28 +251,27 @@ class ParamName(BaseTreeParamName):
def annotation_node(self):
return self._get_param_node().annotation
def infer_annotation(self, execute_annotation=True):
node = self.annotation_node
if node is None:
return NO_CONTEXTS
contexts = self.parent_context.parent_context.eval_node(node)
def infer_annotation(self, execute_annotation=True, ignore_stars=False):
from jedi.inference.gradual.annotation import infer_param
values = infer_param(
self.parent_context, self._get_param_node(),
ignore_stars=ignore_stars)
if execute_annotation:
contexts = contexts.execute_annotation()
return contexts
values = values.execute_annotation()
return values
def infer_default(self):
node = self.default_node
if node is None:
return NO_CONTEXTS
return self.parent_context.parent_context.eval_node(node)
return NO_VALUES
return self.parent_context.parent_context.infer_node(node)
@property
def default_node(self):
return self._get_param_node().default
@property
def string_name(self):
name = self.tree_name.value
def get_public_name(self):
name = self.string_name
if name.startswith('__'):
# Params starting with __ are an equivalent to positional only
# variables in typeshed.
@@ -294,12 +307,19 @@ class ParamName(BaseTreeParamName):
return Parameter.POSITIONAL_OR_KEYWORD
def infer(self):
return self.get_param().infer()
values = self.infer_annotation()
if values:
return values
def get_param(self):
params, _ = self.parent_context.get_executed_params_and_issues()
param_node = search_ancestor(self.tree_name, 'param')
return params[param_node.position_index]
doc_params = docstrings.infer_param(self.parent_context, self._get_param_node())
if doc_params:
return doc_params
return self.get_executed_param_name().infer()
def get_executed_param_name(self):
params_names, _ = self.parent_context.get_executed_param_names_and_issues()
return params_names[self._get_param_node().position_index]
class ParamNameWrapper(_ParamMixin):
@@ -335,18 +355,18 @@ class ImportName(AbstractNameDefinition):
@property
def parent_context(self):
m = self._from_module_context
import_contexts = self.infer()
if not import_contexts:
import_values = self.infer()
if not import_values:
return m
# It's almost always possible to find the import or to not find it. The
# importing returns only one context, pretty much always.
return next(iter(import_contexts))
# importing returns only one value, pretty much always.
return next(iter(import_values))
@memoize_method
def infer(self):
from jedi.evaluate.imports import Importer
from jedi.inference.imports import Importer
m = self._from_module_context
return Importer(m.evaluator, [self.string_name], m, level=self._level).follow()
return Importer(m.inference_state, [self.string_name], m, level=self._level).follow()
def goto(self):
return [m.name for m in self.infer()]
jedi/evaluate/param.py → jedi/inference/param.py
+49 -60
View File
@@ -1,71 +1,60 @@
from collections import defaultdict
from jedi import debug
from jedi.evaluate.utils import PushBackIterator
from jedi.evaluate import analysis
from jedi.evaluate.lazy_context import LazyKnownContext, \
LazyTreeContext, LazyUnknownContext
from jedi.evaluate import docstrings
from jedi.evaluate.context import iterable
from jedi.inference.utils import PushBackIterator
from jedi.inference import analysis
from jedi.inference.lazy_value import LazyKnownValue, \
LazyTreeValue, LazyUnknownValue
from jedi.inference.value import iterable
from jedi._compatibility import Parameter
from jedi.inference.names import ParamName
def _add_argument_issue(error_name, lazy_context, message):
if isinstance(lazy_context, LazyTreeContext):
node = lazy_context.data
def _add_argument_issue(error_name, lazy_value, message):
if isinstance(lazy_value, LazyTreeValue):
node = lazy_value.data
if node.parent.type == 'argument':
node = node.parent
return analysis.add(lazy_context.context, error_name, node, message)
return analysis.add(lazy_value.context, error_name, node, message)
class ExecutedParam(object):
class ExecutedParamName(ParamName):
"""Fake a param and give it values."""
def __init__(self, execution_context, param_node, lazy_context, is_default=False):
self._execution_context = execution_context
self._param_node = param_node
self._lazy_context = lazy_context
self.string_name = param_node.name.value
def __init__(self, execution_context, param_node, lazy_value, is_default=False):
super(ExecutedParamName, self).__init__(execution_context, param_node.name)
self._lazy_value = lazy_value
self._is_default = is_default
def infer_annotations(self):
from jedi.evaluate.gradual.annotation import infer_param
return infer_param(self._execution_context, self._param_node)
def infer(self, use_hints=True):
if use_hints:
doc_params = docstrings.infer_param(self._execution_context, self._param_node)
ann = self.infer_annotations().execute_annotation()
if ann or doc_params:
return ann | doc_params
return self._lazy_context.infer()
def infer(self):
return self._lazy_value.infer()
def matches_signature(self):
if self._is_default:
return True
argument_contexts = self.infer(use_hints=False).py__class__()
if self._param_node.star_count:
argument_values = self.infer().py__class__()
if self.get_kind() in (Parameter.VAR_POSITIONAL, Parameter.VAR_KEYWORD):
return True
annotations = self.infer_annotations()
annotations = self.infer_annotation(execute_annotation=False)
if not annotations:
# If we cannot infer annotations - or there aren't any - pretend
# that the signature matches.
return True
matches = any(c1.is_sub_class_of(c2)
for c1 in argument_contexts
for c1 in argument_values
for c2 in annotations.gather_annotation_classes())
debug.dbg("signature compare %s: %s <=> %s",
matches, argument_contexts, annotations, color='BLUE')
matches, argument_values, annotations, color='BLUE')
return matches
@property
def var_args(self):
return self._execution_context.var_args
return self.parent_context.var_args
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self.string_name)
def get_executed_params_and_issues(execution_context, arguments):
def get_executed_param_names_and_issues(execution_context, arguments):
def too_many_args(argument):
m = _error_argument_count(funcdef, len(unpacked_va))
# Just report an error for the first param that is not needed (like
@@ -86,10 +75,10 @@ def get_executed_params_and_issues(execution_context, arguments):
result_params = []
param_dict = {}
funcdef = execution_context.tree_node
# Default params are part of the context where the function was defined.
# Default params are part of the value where the function was defined.
# This means that they might have access on class variables that the
# function itself doesn't have.
default_param_context = execution_context.function_context.get_default_param_context()
default_param_context = execution_context.function_value.get_default_param_context()
for param in funcdef.get_params():
param_dict[param.name.value] = param
@@ -125,7 +114,7 @@ def get_executed_params_and_issues(execution_context, arguments):
contextualized_node.node, message=m)
)
else:
keys_used[key] = ExecutedParam(execution_context, key_param, argument)
keys_used[key] = ExecutedParamName(execution_context, key_param, argument)
key, argument = next(var_arg_iterator, (None, None))
try:
@@ -136,30 +125,30 @@ def get_executed_params_and_issues(execution_context, arguments):
if param.star_count == 1:
# *args param
lazy_context_list = []
lazy_value_list = []
if argument is not None:
lazy_context_list.append(argument)
lazy_value_list.append(argument)
for key, argument in var_arg_iterator:
# Iterate until a key argument is found.
if key:
var_arg_iterator.push_back((key, argument))
break
lazy_context_list.append(argument)
seq = iterable.FakeSequence(execution_context.evaluator, u'tuple', lazy_context_list)
result_arg = LazyKnownContext(seq)
lazy_value_list.append(argument)
seq = iterable.FakeSequence(execution_context.inference_state, u'tuple', lazy_value_list)
result_arg = LazyKnownValue(seq)
elif param.star_count == 2:
if argument is not None:
too_many_args(argument)
# **kwargs param
dct = iterable.FakeDict(execution_context.evaluator, dict(non_matching_keys))
result_arg = LazyKnownContext(dct)
dct = iterable.FakeDict(execution_context.inference_state, dict(non_matching_keys))
result_arg = LazyKnownValue(dct)
non_matching_keys = {}
else:
# normal param
if argument is None:
# No value: Return an empty container
if param.default is None:
result_arg = LazyUnknownContext()
result_arg = LazyUnknownValue()
if not keys_only:
for contextualized_node in arguments.get_calling_nodes():
m = _error_argument_count(funcdef, len(unpacked_va))
@@ -172,16 +161,16 @@ def get_executed_params_and_issues(execution_context, arguments):
)
)
else:
result_arg = LazyTreeContext(default_param_context, param.default)
result_arg = LazyTreeValue(default_param_context, param.default)
is_default = True
else:
result_arg = argument
result_params.append(ExecutedParam(
result_params.append(ExecutedParamName(
execution_context, param, result_arg,
is_default=is_default
))
if not isinstance(result_arg, LazyUnknownContext):
if not isinstance(result_arg, LazyUnknownValue):
keys_used[param.name.value] = result_params[-1]
if keys_only:
@@ -202,21 +191,21 @@ def get_executed_params_and_issues(execution_context, arguments):
contextualized_node.node, message=m)
)
for key, lazy_context in non_matching_keys.items():
for key, lazy_value in non_matching_keys.items():
m = "TypeError: %s() got an unexpected keyword argument '%s'." \
% (funcdef.name, key)
issues.append(
_add_argument_issue(
'type-error-keyword-argument',
lazy_context,
lazy_value,
message=m
)
)
remaining_arguments = list(var_arg_iterator)
if remaining_arguments:
first_key, lazy_context = remaining_arguments[0]
too_many_args(lazy_context)
first_key, lazy_value = remaining_arguments[0]
too_many_args(lazy_value)
return result_params, issues
@@ -234,18 +223,18 @@ def _error_argument_count(funcdef, actual_count):
def _create_default_param(execution_context, param):
if param.star_count == 1:
result_arg = LazyKnownContext(
iterable.FakeSequence(execution_context.evaluator, u'tuple', [])
result_arg = LazyKnownValue(
iterable.FakeSequence(execution_context.inference_state, u'tuple', [])
)
elif param.star_count == 2:
result_arg = LazyKnownContext(
iterable.FakeDict(execution_context.evaluator, {})
result_arg = LazyKnownValue(
iterable.FakeDict(execution_context.inference_state, {})
)
elif param.default is None:
result_arg = LazyUnknownContext()
result_arg = LazyUnknownValue()
else:
result_arg = LazyTreeContext(execution_context.parent_context, param.default)
return ExecutedParam(execution_context, param, result_arg)
result_arg = LazyTreeValue(execution_context.parent_context, param.default)
return ExecutedParamName(execution_context, param, result_arg)
def create_default_params(execution_context, funcdef):
jedi/inference/parser_cache.py
+6
View File
@@ -0,0 +1,6 @@
from jedi.inference.cache import inference_state_function_cache
@inference_state_function_cache()
def get_yield_exprs(inference_state, funcdef):
return list(funcdef.iter_yield_exprs())
jedi/evaluate/recursion.py → jedi/inference/recursion.py
+12 -11
View File
@@ -3,7 +3,7 @@ Recursions are the recipe of |jedi| to conquer Python code. However, someone
must stop recursions going mad. Some settings are here to make |jedi| stop at
the right time. You can read more about them :ref:`here <settings-recursion>`.
Next to :mod:`jedi.evaluate.cache` this module also makes |jedi| not
Next to :mod:`jedi.inference.cache` this module also makes |jedi| not
thread-safe. Why? ``execution_recursion_decorator`` uses class variables to
count the function calls.
@@ -29,7 +29,7 @@ therefore the quality might not always be maximal.
from contextlib import contextmanager
from jedi import debug
from jedi.evaluate.base_context import NO_CONTEXTS
from jedi.inference.base_value import NO_VALUES
recursion_limit = 15
@@ -56,12 +56,12 @@ class RecursionDetector(object):
@contextmanager
def execution_allowed(evaluator, node):
def execution_allowed(inference_state, node):
"""
A decorator to detect recursions in statements. In a recursion a statement
at the same place, in the same module may not be executed two times.
"""
pushed_nodes = evaluator.recursion_detector.pushed_nodes
pushed_nodes = inference_state.recursion_detector.pushed_nodes
if node in pushed_nodes:
debug.warning('catched stmt recursion: %s @%s', node,
@@ -75,10 +75,10 @@ def execution_allowed(evaluator, node):
pushed_nodes.pop()
def execution_recursion_decorator(default=NO_CONTEXTS):
def execution_recursion_decorator(default=NO_VALUES):
def decorator(func):
def wrapper(self, **kwargs):
detector = self.evaluator.execution_recursion_detector
detector = self.inference_state.execution_recursion_detector
limit_reached = detector.push_execution(self)
try:
if limit_reached:
@@ -96,8 +96,8 @@ class ExecutionRecursionDetector(object):
"""
Catches recursions of executions.
"""
def __init__(self, evaluator):
self._evaluator = evaluator
def __init__(self, inference_state):
self._inference_state = inference_state
self._recursion_level = 0
self._parent_execution_funcs = []
@@ -115,9 +115,9 @@ class ExecutionRecursionDetector(object):
self._recursion_level += 1
self._parent_execution_funcs.append(funcdef)
module = execution.get_root_context()
module_context = execution.get_root_context()
if module == self._evaluator.builtins_module:
if module_context.is_builtins_module():
# We have control over builtins so we know they are not recursing
# like crazy. Therefore we just let them execute always, because
# they usually just help a lot with getting good results.
@@ -133,7 +133,8 @@ class ExecutionRecursionDetector(object):
self._execution_count += 1
if self._funcdef_execution_counts.setdefault(funcdef, 0) >= per_function_execution_limit:
if module.py__name__() in ('builtins', 'typing'):
# TODO why check for builtins here again?
if module_context.py__name__() in ('builtins', 'typing'):
return False
debug.warning(
'Per function execution limit (%s) reached: %s',
jedi/evaluate/signature.py → jedi/inference/signature.py
+20 -20
View File
@@ -33,46 +33,46 @@ class _SignatureMixin(object):
class AbstractSignature(_SignatureMixin):
def __init__(self, context, is_bound=False):
self.context = context
def __init__(self, value, is_bound=False):
self.value = value
self.is_bound = is_bound
@property
def name(self):
return self.context.name
return self.value.name
@property
def annotation_string(self):
return ''
def get_param_names(self, resolve_stars=False):
param_names = self._function_context.get_param_names()
param_names = self._function_value.get_param_names()
if self.is_bound:
return param_names[1:]
return param_names
def bind(self, context):
def bind(self, value):
raise NotImplementedError
def __repr__(self):
return '<%s: %s, %s>' % (self.__class__.__name__, self.context, self._function_context)
return '<%s: %s, %s>' % (self.__class__.__name__, self.value, self._function_value)
class TreeSignature(AbstractSignature):
def __init__(self, context, function_context=None, is_bound=False):
super(TreeSignature, self).__init__(context, is_bound)
self._function_context = function_context or context
def __init__(self, value, function_value=None, is_bound=False):
super(TreeSignature, self).__init__(value, is_bound)
self._function_value = function_value or value
def bind(self, context):
return TreeSignature(context, self._function_context, is_bound=True)
def bind(self, value):
return TreeSignature(value, self._function_value, is_bound=True)
@property
def _annotation(self):
# Classes don't need annotations, even if __init__ has one. They always
# return themselves.
if self.context.is_class():
if self.value.is_class():
return None
return self._function_context.tree_node.annotation
return self._function_value.tree_node.annotation
@property
def annotation_string(self):
@@ -85,14 +85,14 @@ class TreeSignature(AbstractSignature):
def get_param_names(self, resolve_stars=False):
params = super(TreeSignature, self).get_param_names(resolve_stars=False)
if resolve_stars:
from jedi.evaluate.star_args import process_params
from jedi.inference.star_args import process_params
params = process_params(params)
return params
class BuiltinSignature(AbstractSignature):
def __init__(self, context, return_string, is_bound=False):
super(BuiltinSignature, self).__init__(context, is_bound)
def __init__(self, value, return_string, is_bound=False):
super(BuiltinSignature, self).__init__(value, is_bound)
self._return_string = return_string
@property
@@ -100,12 +100,12 @@ class BuiltinSignature(AbstractSignature):
return self._return_string
@property
def _function_context(self):
return self.context
def _function_value(self):
return self.value
def bind(self, context):
def bind(self, value):
assert not self.is_bound
return BuiltinSignature(context, self._return_string, is_bound=True)
return BuiltinSignature(value, self._return_string, is_bound=True)
class SignatureWrapper(_SignatureMixin):
jedi/evaluate/star_args.py → jedi/inference/star_args.py
+13 -13
View File
@@ -1,5 +1,5 @@
"""
This module is responsible for evaluating *args and **kwargs for signatures.
This module is responsible for inferring *args and **kwargs for signatures.
This means for example in this case::
@@ -12,13 +12,13 @@ The signature here for bar should be `bar(b, c)` instead of bar(*args).
"""
from jedi._compatibility import Parameter
from jedi.evaluate.utils import to_list
from jedi.evaluate.names import ParamNameWrapper
from jedi.inference.utils import to_list
from jedi.inference.names import ParamNameWrapper
def _iter_nodes_for_param(param_name):
from parso.python.tree import search_ancestor
from jedi.evaluate.arguments import TreeArguments
from jedi.inference.arguments import TreeArguments
execution_context = param_name.parent_context
function_node = execution_context.tree_node
@@ -37,15 +37,15 @@ def _iter_nodes_for_param(param_name):
if trailer is not None: # Make sure we're in a function
context = execution_context.create_context(trailer)
if _goes_to_param_name(param_name, context, name):
contexts = _to_callables(context, trailer)
values = _to_callables(context, trailer)
args = TreeArguments.create_cached(
execution_context.evaluator,
execution_context.inference_state,
context=context,
argument_node=trailer.children[1],
trailer=trailer,
)
for c in contexts:
for c in values:
yield c, args
else:
assert False
@@ -54,7 +54,7 @@ def _iter_nodes_for_param(param_name):
def _goes_to_param_name(param_name, context, potential_name):
if potential_name.type != 'name':
return False
from jedi.evaluate.names import TreeNameDefinition
from jedi.inference.names import TreeNameDefinition
found = TreeNameDefinition(context, potential_name).goto()
return any(param_name.parent_context == p.parent_context
and param_name.start_pos == p.start_pos
@@ -62,17 +62,17 @@ def _goes_to_param_name(param_name, context, potential_name):
def _to_callables(context, trailer):
from jedi.evaluate.syntax_tree import eval_trailer
from jedi.inference.syntax_tree import infer_trailer
atom_expr = trailer.parent
index = atom_expr.children[0] == 'await'
# Eval atom first
contexts = context.eval_node(atom_expr.children[index])
# Infer atom first
values = context.infer_node(atom_expr.children[index])
for trailer2 in atom_expr.children[index + 1:]:
if trailer == trailer2:
break
contexts = eval_trailer(context, contexts, trailer2)
return contexts
values = infer_trailer(context, values, trailer2)
return values
def _remove_given_params(arguments, param_names):
jedi/evaluate/syntax_tree.py → jedi/inference/syntax_tree.py
+223 -226
View File
@@ -1,5 +1,5 @@
"""
Functions evaluating the syntax tree.
Functions inferring the syntax tree.
"""
import copy
@@ -8,29 +8,27 @@ from parso.python import tree
from jedi._compatibility import force_unicode, unicode
from jedi import debug
from jedi import parser_utils
from jedi.evaluate.base_context import ContextSet, NO_CONTEXTS, ContextualizedNode, \
ContextualizedName, iterator_to_context_set, iterate_contexts
from jedi.evaluate.lazy_context import LazyTreeContext
from jedi.evaluate import compiled
from jedi.evaluate import recursion
from jedi.evaluate import helpers
from jedi.evaluate import analysis
from jedi.evaluate import imports
from jedi.evaluate import arguments
from jedi.evaluate.context import ClassContext, FunctionContext
from jedi.evaluate.context import iterable
from jedi.evaluate.context import TreeInstance
from jedi.evaluate.finder import NameFinder
from jedi.evaluate.helpers import is_string, is_literal, is_number
from jedi.evaluate.compiled.access import COMPARISON_OPERATORS
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.gradual.stub_context import VersionInfo
from jedi.evaluate.gradual import annotation
from jedi.evaluate.context.decorator import Decoratee
from jedi.inference.base_value import ValueSet, NO_VALUES, ContextualizedNode, \
ContextualizedName, iterator_to_value_set, iterate_values
from jedi.inference.lazy_value import LazyTreeValue
from jedi.inference import compiled
from jedi.inference import recursion
from jedi.inference import analysis
from jedi.inference import imports
from jedi.inference import arguments
from jedi.inference.value import ClassValue, FunctionValue
from jedi.inference.value import iterable
from jedi.inference.value import TreeInstance
from jedi.inference.helpers import is_string, is_literal, is_number
from jedi.inference.compiled.access import COMPARISON_OPERATORS
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.gradual.stub_value import VersionInfo
from jedi.inference.gradual import annotation
from jedi.inference.value.decorator import Decoratee
from jedi.plugins import plugin_manager
def _limit_context_infers(func):
def _limit_value_infers(func):
"""
This is for now the way how we limit type inference going wild. There are
other ways to ensure recursion limits as well. This is mostly necessary
@@ -41,21 +39,21 @@ def _limit_context_infers(func):
"""
def wrapper(context, *args, **kwargs):
n = context.tree_node
evaluator = context.evaluator
inference_state = context.inference_state
try:
evaluator.inferred_element_counts[n] += 1
if evaluator.inferred_element_counts[n] > 300:
debug.warning('In context %s there were too many inferences.', n)
return NO_CONTEXTS
inference_state.inferred_element_counts[n] += 1
if inference_state.inferred_element_counts[n] > 300:
debug.warning('In value %s there were too many inferences.', n)
return NO_VALUES
except KeyError:
evaluator.inferred_element_counts[n] = 1
inference_state.inferred_element_counts[n] = 1
return func(context, *args, **kwargs)
return wrapper
def _py__stop_iteration_returns(generators):
results = NO_CONTEXTS
results = NO_VALUES
for generator in generators:
try:
method = generator.py__stop_iteration_returns
@@ -67,17 +65,17 @@ def _py__stop_iteration_returns(generators):
@debug.increase_indent
@_limit_context_infers
def eval_node(context, element):
debug.dbg('eval_node %s@%s in %s', element, element.start_pos, context)
evaluator = context.evaluator
@_limit_value_infers
def infer_node(context, element):
debug.dbg('infer_node %s@%s in %s', element, element.start_pos, context)
inference_state = context.inference_state
typ = element.type
if typ in ('name', 'number', 'string', 'atom', 'strings', 'keyword', 'fstring'):
return eval_atom(context, element)
return infer_atom(context, element)
elif typ == 'lambdef':
return ContextSet([FunctionContext.from_context(context, element)])
return ValueSet([FunctionValue.from_context(context, element)])
elif typ == 'expr_stmt':
return eval_expr_stmt(context, element)
return infer_expr_stmt(context, element)
elif typ in ('power', 'atom_expr'):
first_child = element.children[0]
children = element.children[1:]
@@ -86,104 +84,103 @@ def eval_node(context, element):
had_await = True
first_child = children.pop(0)
context_set = context.eval_node(first_child)
value_set = context.infer_node(first_child)
for (i, trailer) in enumerate(children):
if trailer == '**': # has a power operation.
right = context.eval_node(children[i + 1])
context_set = _eval_comparison(
evaluator,
right = context.infer_node(children[i + 1])
value_set = _infer_comparison(
context,
context_set,
value_set,
trailer,
right
)
break
context_set = eval_trailer(context, context_set, trailer)
value_set = infer_trailer(context, value_set, trailer)
if had_await:
return context_set.py__await__().py__stop_iteration_returns()
return context_set
return value_set.py__await__().py__stop_iteration_returns()
return value_set
elif typ in ('testlist_star_expr', 'testlist',):
# The implicit tuple in statements.
return ContextSet([iterable.SequenceLiteralContext(evaluator, context, element)])
return ValueSet([iterable.SequenceLiteralValue(inference_state, context, element)])
elif typ in ('not_test', 'factor'):
context_set = context.eval_node(element.children[-1])
value_set = context.infer_node(element.children[-1])
for operator in element.children[:-1]:
context_set = eval_factor(context_set, operator)
return context_set
value_set = infer_factor(value_set, operator)
return value_set
elif typ == 'test':
# `x if foo else y` case.
return (context.eval_node(element.children[0]) |
context.eval_node(element.children[-1]))
return (context.infer_node(element.children[0]) |
context.infer_node(element.children[-1]))
elif typ == 'operator':
# Must be an ellipsis, other operators are not evaluated.
# Must be an ellipsis, other operators are not inferred.
# In Python 2 ellipsis is coded as three single dot tokens, not
# as one token 3 dot token.
if element.value not in ('.', '...'):
origin = element.parent
raise AssertionError("unhandled operator %s in %s " % (repr(element.value), origin))
return ContextSet([compiled.builtin_from_name(evaluator, u'Ellipsis')])
return ValueSet([compiled.builtin_from_name(inference_state, u'Ellipsis')])
elif typ == 'dotted_name':
context_set = eval_atom(context, element.children[0])
value_set = infer_atom(context, element.children[0])
for next_name in element.children[2::2]:
# TODO add search_global=True?
context_set = context_set.py__getattribute__(next_name, name_context=context)
return context_set
value_set = value_set.py__getattribute__(next_name, name_context=context)
return value_set
elif typ == 'eval_input':
return eval_node(context, element.children[0])
return infer_node(context, element.children[0])
elif typ == 'annassign':
return annotation.eval_annotation(context, element.children[1]) \
return annotation.infer_annotation(context, element.children[1]) \
.execute_annotation()
elif typ == 'yield_expr':
if len(element.children) and element.children[1].type == 'yield_arg':
# Implies that it's a yield from.
element = element.children[1].children[1]
generators = context.eval_node(element) \
.py__getattribute__('__iter__').execute_evaluated()
generators = context.infer_node(element) \
.py__getattribute__('__iter__').execute_with_values()
return generators.py__stop_iteration_returns()
# Generator.send() is not implemented.
return NO_CONTEXTS
return NO_VALUES
elif typ == 'namedexpr_test':
return eval_node(context, element.children[2])
return infer_node(context, element.children[2])
else:
return eval_or_test(context, element)
return infer_or_test(context, element)
def eval_trailer(context, atom_contexts, trailer):
def infer_trailer(context, atom_values, trailer):
trailer_op, node = trailer.children[:2]
if node == ')': # `arglist` is optional.
node = None
if trailer_op == '[':
trailer_op, node, _ = trailer.children
return atom_contexts.get_item(
eval_subscript_list(context.evaluator, context, node),
return atom_values.get_item(
_infer_subscript_list(context, node),
ContextualizedNode(context, trailer)
)
else:
debug.dbg('eval_trailer: %s in %s', trailer, atom_contexts)
debug.dbg('infer_trailer: %s in %s', trailer, atom_values)
if trailer_op == '.':
return atom_contexts.py__getattribute__(
return atom_values.py__getattribute__(
name_context=context,
name_or_str=node
)
else:
assert trailer_op == '(', 'trailer_op is actually %s' % trailer_op
args = arguments.TreeArguments(context.evaluator, context, node, trailer)
return atom_contexts.execute(args)
args = arguments.TreeArguments(context.inference_state, context, node, trailer)
return atom_values.execute(args)
def eval_atom(context, atom):
def infer_atom(context, atom):
"""
Basically to process ``atom`` nodes. The parser sometimes doesn't
generate the node (because it has just one child). In that case an atom
might be a name or a literal as well.
"""
state = context.inference_state
if atom.type == 'name':
if atom.value in ('True', 'False', 'None'):
# Python 2...
return ContextSet([compiled.builtin_from_name(context.evaluator, atom.value)])
return ValueSet([compiled.builtin_from_name(state, atom.value)])
# This is the first global lookup.
stmt = tree.search_ancestor(
@@ -199,43 +196,39 @@ def eval_atom(context, atom):
# position to None, so the finder will not try to stop at a certain
# position in the module.
position = None
return context.py__getattribute__(
name_or_str=atom,
position=position,
search_global=True
)
return context.py__getattribute__(atom, position=position)
elif atom.type == 'keyword':
# For False/True/None
if atom.value in ('False', 'True', 'None'):
return ContextSet([compiled.builtin_from_name(context.evaluator, atom.value)])
return ValueSet([compiled.builtin_from_name(state, atom.value)])
elif atom.value == 'print':
# print e.g. could be evaluated like this in Python 2.7
return NO_CONTEXTS
# print e.g. could be inferred like this in Python 2.7
return NO_VALUES
elif atom.value == 'yield':
# Contrary to yield from, yield can just appear alone to return a
# value when used with `.send()`.
return NO_CONTEXTS
assert False, 'Cannot evaluate the keyword %s' % atom
return NO_VALUES
assert False, 'Cannot infer the keyword %s' % atom
elif isinstance(atom, tree.Literal):
string = context.evaluator.compiled_subprocess.safe_literal_eval(atom.value)
return ContextSet([compiled.create_simple_object(context.evaluator, string)])
string = state.compiled_subprocess.safe_literal_eval(atom.value)
return ValueSet([compiled.create_simple_object(state, string)])
elif atom.type == 'strings':
# Will be multiple string.
context_set = eval_atom(context, atom.children[0])
value_set = infer_atom(context, atom.children[0])
for string in atom.children[1:]:
right = eval_atom(context, string)
context_set = _eval_comparison(context.evaluator, context, context_set, u'+', right)
return context_set
right = infer_atom(context, string)
value_set = _infer_comparison(context, value_set, u'+', right)
return value_set
elif atom.type == 'fstring':
return compiled.get_string_context_set(context.evaluator)
return compiled.get_string_value_set(state)
else:
c = atom.children
# Parentheses without commas are not tuples.
if c[0] == '(' and not len(c) == 2 \
and not(c[1].type == 'testlist_comp' and
len(c[1].children) > 1):
return context.eval_node(c[1])
return context.infer_node(c[1])
try:
comp_for = c[1].children[1]
@@ -250,8 +243,8 @@ def eval_atom(context, atom):
pass
if comp_for.type in ('comp_for', 'sync_comp_for'):
return ContextSet([iterable.comprehension_from_atom(
context.evaluator, context, atom
return ValueSet([iterable.comprehension_from_atom(
state, context, atom
)])
# It's a dict/list/tuple literal.
@@ -262,36 +255,36 @@ def eval_atom(context, atom):
array_node_c = []
if c[0] == '{' and (array_node == '}' or ':' in array_node_c or
'**' in array_node_c):
context = iterable.DictLiteralContext(context.evaluator, context, atom)
new_value = iterable.DictLiteralValue(state, context, atom)
else:
context = iterable.SequenceLiteralContext(context.evaluator, context, atom)
return ContextSet([context])
new_value = iterable.SequenceLiteralValue(state, context, atom)
return ValueSet([new_value])
@_limit_context_infers
def eval_expr_stmt(context, stmt, seek_name=None):
with recursion.execution_allowed(context.evaluator, stmt) as allowed:
@_limit_value_infers
def infer_expr_stmt(context, stmt, seek_name=None):
with recursion.execution_allowed(context.inference_state, stmt) as allowed:
# Here we allow list/set to recurse under certain conditions. To make
# it possible to resolve stuff like list(set(list(x))), this is
# necessary.
if not allowed and context.get_root_context() == context.evaluator.builtins_module:
if not allowed and context.get_root_context().is_builtins_module():
try:
instance = context.var_args.instance
except AttributeError:
pass
else:
if instance.name.string_name in ('list', 'set'):
c = instance.get_first_non_keyword_argument_contexts()
c = instance.get_first_non_keyword_argument_values()
if instance not in c:
allowed = True
if allowed:
return _eval_expr_stmt(context, stmt, seek_name)
return NO_CONTEXTS
return _infer_expr_stmt(context, stmt, seek_name)
return NO_VALUES
@debug.increase_indent
def _eval_expr_stmt(context, stmt, seek_name=None):
def _infer_expr_stmt(context, stmt, seek_name=None):
"""
The starting point of the completion. A statement always owns a call
list, which are the calls, that a statement does. In case multiple
@@ -300,13 +293,13 @@ def _eval_expr_stmt(context, stmt, seek_name=None):
:param stmt: A `tree.ExprStmt`.
"""
debug.dbg('eval_expr_stmt %s (%s)', stmt, seek_name)
debug.dbg('infer_expr_stmt %s (%s)', stmt, seek_name)
rhs = stmt.get_rhs()
context_set = context.eval_node(rhs)
value_set = context.infer_node(rhs)
if seek_name:
c_node = ContextualizedName(context, seek_name)
context_set = check_tuple_assignments(context.evaluator, c_node, context_set)
value_set = check_tuple_assignments(c_node, value_set)
first_operator = next(stmt.yield_operators(), None)
if first_operator not in ('=', None) and first_operator.type == 'operator':
@@ -314,11 +307,10 @@ def _eval_expr_stmt(context, stmt, seek_name=None):
operator = copy.copy(first_operator)
operator.value = operator.value[:-1]
name = stmt.get_defined_names()[0].value
left = context.py__getattribute__(
name, position=stmt.start_pos, search_global=True)
left = context.py__getattribute__(name, position=stmt.start_pos)
for_stmt = tree.search_ancestor(stmt, 'for_stmt')
if for_stmt is not None and for_stmt.type == 'for_stmt' and context_set \
if for_stmt is not None and for_stmt.type == 'for_stmt' and value_set \
and parser_utils.for_stmt_defines_one_name(for_stmt):
# Iterate through result and add the values, that's possible
# only in for loops without clutter, because they are
@@ -327,92 +319,93 @@ def _eval_expr_stmt(context, stmt, seek_name=None):
cn = ContextualizedNode(context, node)
ordered = list(cn.infer().iterate(cn))
for lazy_context in ordered:
dct = {for_stmt.children[1].value: lazy_context.infer()}
with helpers.predefine_names(context, for_stmt, dct):
t = context.eval_node(rhs)
left = _eval_comparison(context.evaluator, context, left, operator, t)
context_set = left
for lazy_value in ordered:
dct = {for_stmt.children[1].value: lazy_value.infer()}
with context.predefine_names(for_stmt, dct):
t = context.infer_node(rhs)
left = _infer_comparison(context, left, operator, t)
value_set = left
else:
context_set = _eval_comparison(context.evaluator, context, left, operator, context_set)
debug.dbg('eval_expr_stmt result %s', context_set)
return context_set
value_set = _infer_comparison(context, left, operator, value_set)
debug.dbg('infer_expr_stmt result %s', value_set)
return value_set
def eval_or_test(context, or_test):
def infer_or_test(context, or_test):
iterator = iter(or_test.children)
types = context.eval_node(next(iterator))
types = context.infer_node(next(iterator))
for operator in iterator:
right = next(iterator)
if operator.type == 'comp_op': # not in / is not
operator = ' '.join(c.value for c in operator.children)
# handle lazy evaluation of and/or here.
# handle type inference of and/or here.
if operator in ('and', 'or'):
left_bools = set(left.py__bool__() for left in types)
if left_bools == {True}:
if operator == 'and':
types = context.eval_node(right)
types = context.infer_node(right)
elif left_bools == {False}:
if operator != 'and':
types = context.eval_node(right)
types = context.infer_node(right)
# Otherwise continue, because of uncertainty.
else:
types = _eval_comparison(context.evaluator, context, types, operator,
context.eval_node(right))
debug.dbg('eval_or_test types %s', types)
types = _infer_comparison(context, types, operator,
context.infer_node(right))
debug.dbg('infer_or_test types %s', types)
return types
@iterator_to_context_set
def eval_factor(context_set, operator):
@iterator_to_value_set
def infer_factor(value_set, operator):
"""
Calculates `+`, `-`, `~` and `not` prefixes.
"""
for context in context_set:
for value in value_set:
if operator == '-':
if is_number(context):
yield context.negate()
if is_number(value):
yield value.negate()
elif operator == 'not':
value = context.py__bool__()
if value is None: # Uncertainty.
b = value.py__bool__()
if b is None: # Uncertainty.
return
yield compiled.create_simple_object(context.evaluator, not value)
yield compiled.create_simple_object(value.inference_state, not b)
else:
yield context
yield value
def _literals_to_types(evaluator, result):
def _literals_to_types(inference_state, result):
# Changes literals ('a', 1, 1.0, etc) to its type instances (str(),
# int(), float(), etc).
new_result = NO_CONTEXTS
new_result = NO_VALUES
for typ in result:
if is_literal(typ):
# Literals are only valid as long as the operations are
# correct. Otherwise add a value-free instance.
cls = compiled.builtin_from_name(evaluator, typ.name.string_name)
new_result |= cls.execute_evaluated()
cls = compiled.builtin_from_name(inference_state, typ.name.string_name)
new_result |= cls.execute_with_values()
else:
new_result |= ContextSet([typ])
new_result |= ValueSet([typ])
return new_result
def _eval_comparison(evaluator, context, left_contexts, operator, right_contexts):
if not left_contexts or not right_contexts:
def _infer_comparison(context, left_values, operator, right_values):
state = context.inference_state
if not left_values or not right_values:
# illegal slices e.g. cause left/right_result to be None
result = (left_contexts or NO_CONTEXTS) | (right_contexts or NO_CONTEXTS)
return _literals_to_types(evaluator, result)
result = (left_values or NO_VALUES) | (right_values or NO_VALUES)
return _literals_to_types(state, result)
else:
# I don't think there's a reasonable chance that a string
# operation is still correct, once we pass something like six
# objects.
if len(left_contexts) * len(right_contexts) > 6:
return _literals_to_types(evaluator, left_contexts | right_contexts)
if len(left_values) * len(right_values) > 6:
return _literals_to_types(state, left_values | right_values)
else:
return ContextSet.from_sets(
_eval_comparison_part(evaluator, context, left, operator, right)
for left in left_contexts
for right in right_contexts
return ValueSet.from_sets(
_infer_comparison_part(state, context, left, operator, right)
for left in left_values
for right in right_values
)
@@ -432,26 +425,26 @@ def _is_annotation_name(name):
return False
def _is_tuple(context):
return isinstance(context, iterable.Sequence) and context.array_type == 'tuple'
def _is_tuple(value):
return isinstance(value, iterable.Sequence) and value.array_type == 'tuple'
def _is_list(context):
return isinstance(context, iterable.Sequence) and context.array_type == 'list'
def _is_list(value):
return isinstance(value, iterable.Sequence) and value.array_type == 'list'
def _bool_to_context(evaluator, bool_):
return compiled.builtin_from_name(evaluator, force_unicode(str(bool_)))
def _bool_to_value(inference_state, bool_):
return compiled.builtin_from_name(inference_state, force_unicode(str(bool_)))
def _get_tuple_ints(context):
if not isinstance(context, iterable.SequenceLiteralContext):
def _get_tuple_ints(value):
if not isinstance(value, iterable.SequenceLiteralValue):
return None
numbers = []
for lazy_context in context.py__iter__():
if not isinstance(lazy_context, LazyTreeContext):
for lazy_value in value.py__iter__():
if not isinstance(lazy_value, LazyTreeValue):
return None
node = lazy_context.data
node = lazy_value.data
if node.type != 'number':
return None
try:
@@ -461,7 +454,7 @@ def _get_tuple_ints(context):
return numbers
def _eval_comparison_part(evaluator, context, left, operator, right):
def _infer_comparison_part(inference_state, context, left, operator, right):
l_is_num = is_number(left)
r_is_num = is_number(right)
if isinstance(operator, unicode):
@@ -472,26 +465,26 @@ def _eval_comparison_part(evaluator, context, left, operator, right):
if str_operator == '*':
# for iterables, ignore * operations
if isinstance(left, iterable.Sequence) or is_string(left):
return ContextSet([left])
return ValueSet([left])
elif isinstance(right, iterable.Sequence) or is_string(right):
return ContextSet([right])
return ValueSet([right])
elif str_operator == '+':
if l_is_num and r_is_num or is_string(left) and is_string(right):
return ContextSet([left.execute_operation(right, str_operator)])
return ValueSet([left.execute_operation(right, str_operator)])
elif _is_tuple(left) and _is_tuple(right) or _is_list(left) and _is_list(right):
return ContextSet([iterable.MergedArray(evaluator, (left, right))])
return ValueSet([iterable.MergedArray(inference_state, (left, right))])
elif str_operator == '-':
if l_is_num and r_is_num:
return ContextSet([left.execute_operation(right, str_operator)])
return ValueSet([left.execute_operation(right, str_operator)])
elif str_operator == '%':
# With strings and numbers the left type typically remains. Except for
# `int() % float()`.
return ContextSet([left])
return ValueSet([left])
elif str_operator in COMPARISON_OPERATORS:
if left.is_compiled() and right.is_compiled():
# Possible, because the return is not an option. Just compare.
try:
return ContextSet([left.execute_operation(right, str_operator)])
return ValueSet([left.execute_operation(right, str_operator)])
except TypeError:
# Could be True or False.
pass
@@ -499,20 +492,23 @@ def _eval_comparison_part(evaluator, context, left, operator, right):
if str_operator in ('is', '!=', '==', 'is not'):
operation = COMPARISON_OPERATORS[str_operator]
bool_ = operation(left, right)
return ContextSet([_bool_to_context(evaluator, bool_)])
return ValueSet([_bool_to_value(inference_state, bool_)])
if isinstance(left, VersionInfo):
version_info = _get_tuple_ints(right)
if version_info is not None:
bool_result = compiled.access.COMPARISON_OPERATORS[operator](
evaluator.environment.version_info,
inference_state.environment.version_info,
tuple(version_info)
)
return ContextSet([_bool_to_context(evaluator, bool_result)])
return ValueSet([_bool_to_value(inference_state, bool_result)])
return ContextSet([_bool_to_context(evaluator, True), _bool_to_context(evaluator, False)])
return ValueSet([
_bool_to_value(inference_state, True),
_bool_to_value(inference_state, False)
])
elif str_operator == 'in':
return NO_CONTEXTS
return NO_VALUES
def check(obj):
"""Checks if a Jedi object is either a float or an int."""
@@ -526,29 +522,31 @@ def _eval_comparison_part(evaluator, context, left, operator, right):
analysis.add(context, 'type-error-operation', operator,
message % (left, right))
result = ContextSet([left, right])
result = ValueSet([left, right])
debug.dbg('Used operator %s resulting in %s', operator, result)
return result
def _remove_statements(evaluator, context, stmt, name):
def _remove_statements(context, stmt, name):
"""
This is the part where statements are being stripped.
Due to lazy evaluation, statements like a = func; b = a; b() have to be
evaluated.
"""
pep0484_contexts = \
annotation.find_type_from_comment_hint_assign(context, stmt, name)
if pep0484_contexts:
return pep0484_contexts
Due to lazy type inference, statements like a = func; b = a; b() have to be
inferred.
return eval_expr_stmt(context, stmt, seek_name=name)
TODO merge with infer_expr_stmt?
"""
pep0484_values = \
annotation.find_type_from_comment_hint_assign(context, stmt, name)
if pep0484_values:
return pep0484_values
return infer_expr_stmt(context, stmt, seek_name=name)
@plugin_manager.decorate()
def tree_name_to_contexts(evaluator, context, tree_name):
context_set = NO_CONTEXTS
def tree_name_to_values(inference_state, context, tree_name):
value_set = NO_VALUES
module_node = context.get_root_context().tree_node
# First check for annotations, like: `foo: int = 3`
if module_node is not None:
@@ -559,27 +557,26 @@ def tree_name_to_contexts(evaluator, context, tree_name):
if expr_stmt.type == "expr_stmt" and expr_stmt.children[1].type == "annassign":
correct_scope = parser_utils.get_parent_scope(name) == context.tree_node
if correct_scope:
context_set |= annotation.eval_annotation(
value_set |= annotation.infer_annotation(
context, expr_stmt.children[1].children[1]
).execute_annotation()
if context_set:
return context_set
if value_set:
return value_set
types = []
node = tree_name.get_definition(import_name_always=True)
if node is None:
node = tree_name.parent
if node.type == 'global_stmt':
context = evaluator.create_context(context, tree_name)
finder = NameFinder(evaluator, context, context, tree_name.value)
filters = finder.get_filters(search_global=True)
c = context.create_context(tree_name)
# For global_stmt lookups, we only need the first possible scope,
# which means the function itself.
filters = [next(filters)]
return finder.find(filters, attribute_lookup=False)
filter = next(c.get_filters())
names = filter.get(tree_name.value)
return ValueSet.from_sets(name.infer() for name in names)
elif node.type not in ('import_from', 'import_name'):
context = evaluator.create_context(context, tree_name)
return eval_atom(context, tree_name)
c = context.create_context(tree_name)
return infer_atom(c, tree_name)
typ = node.type
if typ == 'for_stmt':
@@ -596,19 +593,19 @@ def tree_name_to_contexts(evaluator, context, tree_name):
types = context.predefined_names[node][tree_name.value]
except KeyError:
cn = ContextualizedNode(context, node.children[3])
for_types = iterate_contexts(
for_types = iterate_values(
cn.infer(),
contextualized_node=cn,
is_async=node.parent.type == 'async_stmt',
)
c_node = ContextualizedName(context, tree_name)
types = check_tuple_assignments(evaluator, c_node, for_types)
types = check_tuple_assignments(c_node, for_types)
elif typ == 'expr_stmt':
types = _remove_statements(evaluator, context, node, tree_name)
types = _remove_statements(context, node, tree_name)
elif typ == 'with_stmt':
context_managers = context.eval_node(node.get_test_node_from_name(tree_name))
enter_methods = context_managers.py__getattribute__(u'__enter__')
return enter_methods.execute_evaluated()
value_managers = context.infer_node(node.get_test_node_from_name(tree_name))
enter_methods = value_managers.py__getattribute__(u'__enter__')
return enter_methods.execute_with_values()
elif typ in ('import_from', 'import_name'):
types = imports.infer_import(context, tree_name)
elif typ in ('funcdef', 'classdef'):
@@ -617,10 +614,10 @@ def tree_name_to_contexts(evaluator, context, tree_name):
# TODO an exception can also be a tuple. Check for those.
# TODO check for types that are not classes and add it to
# the static analysis report.
exceptions = context.eval_node(tree_name.get_previous_sibling().get_previous_sibling())
types = exceptions.execute_evaluated()
exceptions = context.infer_node(tree_name.get_previous_sibling().get_previous_sibling())
types = exceptions.execute_with_values()
elif node.type == 'param':
types = NO_CONTEXTS
types = NO_VALUES
else:
raise ValueError("Should not happen. type: %s" % typ)
return types
@@ -628,31 +625,31 @@ def tree_name_to_contexts(evaluator, context, tree_name):
# We don't want to have functions/classes that are created by the same
# tree_node.
@evaluator_method_cache()
@inference_state_method_cache()
def _apply_decorators(context, node):
"""
Returns the function, that should to be executed in the end.
This is also the places where the decorators are processed.
"""
if node.type == 'classdef':
decoratee_context = ClassContext(
context.evaluator,
decoratee_value = ClassValue(
context.inference_state,
parent_context=context,
tree_node=node
)
else:
decoratee_context = FunctionContext.from_context(context, node)
initial = values = ContextSet([decoratee_context])
decoratee_value = FunctionValue.from_context(context, node)
initial = values = ValueSet([decoratee_value])
for dec in reversed(node.get_decorators()):
debug.dbg('decorator: %s %s', dec, values, color="MAGENTA")
with debug.increase_indent_cm():
dec_values = context.eval_node(dec.children[1])
dec_values = context.infer_node(dec.children[1])
trailer_nodes = dec.children[2:-1]
if trailer_nodes:
# Create a trailer and evaluate it.
# Create a trailer and infer it.
trailer = tree.PythonNode('trailer', trailer_nodes)
trailer.parent = dec
dec_values = eval_trailer(context, dec_values, trailer)
dec_values = infer_trailer(context, dec_values, trailer)
if not len(dec_values):
code = dec.get_code(include_prefix=False)
@@ -670,41 +667,41 @@ def _apply_decorators(context, node):
debug.dbg('decorator end %s', values, color="MAGENTA")
if values != initial:
return ContextSet([Decoratee(c, decoratee_context) for c in values])
return ValueSet([Decoratee(c, decoratee_value) for c in values])
return values
def check_tuple_assignments(evaluator, contextualized_name, context_set):
def check_tuple_assignments(contextualized_name, value_set):
"""
Checks if tuples are assigned.
"""
lazy_context = None
lazy_value = None
for index, node in contextualized_name.assignment_indexes():
cn = ContextualizedNode(contextualized_name.context, node)
iterated = context_set.iterate(cn)
iterated = value_set.iterate(cn)
if isinstance(index, slice):
# For no star unpacking is not possible.
return NO_CONTEXTS
return NO_VALUES
for _ in range(index + 1):
try:
lazy_context = next(iterated)
lazy_value = next(iterated)
except StopIteration:
# We could do this with the default param in next. But this
# would allow this loop to run for a very long time if the
# index number is high. Therefore break if the loop is
# finished.
return NO_CONTEXTS
context_set = lazy_context.infer()
return context_set
return NO_VALUES
value_set = lazy_value.infer()
return value_set
def eval_subscript_list(evaluator, context, index):
def _infer_subscript_list(context, index):
"""
Handles slices in subscript nodes.
"""
if index == ':':
# Like array[:]
return ContextSet([iterable.Slice(context, None, None, None)])
return ValueSet([iterable.Slice(context, None, None, None)])
elif index.type == 'subscript' and not index.children[0] == '.':
# subscript basically implies a slice operation, except for Python 2's
@@ -722,9 +719,9 @@ def eval_subscript_list(evaluator, context, index):
result.append(el)
result += [None] * (3 - len(result))
return ContextSet([iterable.Slice(context, *result)])
return ValueSet([iterable.Slice(context, *result)])
elif index.type == 'subscriptlist':
return ContextSet([iterable.SequenceLiteralContext(evaluator, context, index)])
return ValueSet([iterable.SequenceLiteralValue(context.inference_state, context, index)])
# No slices
return context.eval_node(index)
return context.infer_node(index)
jedi/evaluate/sys_path.py → jedi/inference/sys_path.py
+21 -21
View File
@@ -1,9 +1,9 @@
import os
from jedi._compatibility import unicode, force_unicode, all_suffixes
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.base_context import ContextualizedNode
from jedi.evaluate.helpers import is_string
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.base_value import ContextualizedNode
from jedi.inference.helpers import is_string
from jedi.common.utils import traverse_parents
from jedi.parser_utils import get_cached_code_lines
from jedi.file_io import FileIO
@@ -61,10 +61,10 @@ def _paths_from_assignment(module_context, expr_stmt):
continue
cn = ContextualizedNode(module_context.create_context(expr_stmt), expr_stmt)
for lazy_context in cn.infer().iterate(cn):
for context in lazy_context.infer():
if is_string(context):
abs_path = _abs_path(module_context, context.get_safe_value())
for lazy_value in cn.infer().iterate(cn):
for value in lazy_value.infer():
if is_string(value):
abs_path = _abs_path(module_context, value.get_safe_value())
if abs_path is not None:
yield abs_path
@@ -85,14 +85,14 @@ def _paths_from_list_modifications(module_context, trailer1, trailer2):
if name == 'insert' and len(arg.children) in (3, 4): # Possible trailing comma.
arg = arg.children[2]
for context in module_context.create_context(arg).eval_node(arg):
if is_string(context):
abs_path = _abs_path(module_context, context.get_safe_value())
for value in module_context.create_context(arg).infer_node(arg):
if is_string(value):
abs_path = _abs_path(module_context, value.get_safe_value())
if abs_path is not None:
yield abs_path
@evaluator_method_cache(default=[])
@inference_state_method_cache(default=[])
def check_sys_path_modifications(module_context):
"""
Detect sys.path modifications within module.
@@ -130,20 +130,20 @@ def check_sys_path_modifications(module_context):
return added
def discover_buildout_paths(evaluator, script_path):
def discover_buildout_paths(inference_state, script_path):
buildout_script_paths = set()
for buildout_script_path in _get_buildout_script_paths(script_path):
for path in _get_paths_from_buildout_script(evaluator, buildout_script_path):
for path in _get_paths_from_buildout_script(inference_state, buildout_script_path):
buildout_script_paths.add(path)
return buildout_script_paths
def _get_paths_from_buildout_script(evaluator, buildout_script_path):
def _get_paths_from_buildout_script(inference_state, buildout_script_path):
file_io = FileIO(buildout_script_path)
try:
module_node = evaluator.parse(
module_node = inference_state.parse(
file_io=file_io,
cache=True,
cache_path=settings.cache_directory
@@ -152,13 +152,13 @@ def _get_paths_from_buildout_script(evaluator, buildout_script_path):
debug.warning('Error trying to read buildout_script: %s', buildout_script_path)
return
from jedi.evaluate.context import ModuleContext
module = ModuleContext(
evaluator, module_node, file_io,
from jedi.inference.value import ModuleValue
module_context = ModuleValue(
inference_state, module_node, file_io,
string_names=None,
code_lines=get_cached_code_lines(evaluator.grammar, buildout_script_path),
)
for path in check_sys_path_modifications(module):
code_lines=get_cached_code_lines(inference_state.grammar, buildout_script_path),
).as_context()
for path in check_sys_path_modifications(module_context):
yield path
jedi/evaluate/usages.py → jedi/inference/usages.py
+11 -7
View File
@@ -1,5 +1,5 @@
from jedi.evaluate import imports
from jedi.evaluate.names import TreeNameDefinition
from jedi.inference import imports
from jedi.inference.names import TreeNameDefinition
def _resolve_names(definition_names, avoid_names=()):
@@ -37,13 +37,17 @@ def _find_names(module_context, tree_name):
def usages(module_context, tree_name):
search_name = tree_name.value
found_names = _find_names(module_context, tree_name)
modules = set(d.get_root_context() for d in found_names.values())
modules = set(m for m in modules if m.is_module() and not m.is_compiled())
module_contexts = set(d.get_root_context() for d in found_names.values())
module_contexts = set(m for m in module_contexts if not m.is_compiled())
non_matching_usage_maps = {}
for m in imports.get_modules_containing_name(module_context.evaluator, modules, search_name):
for name_leaf in m.tree_node.get_used_names().get(search_name, []):
new = _find_names(m, name_leaf)
inf = module_context.inference_state
potential_modules = imports.get_module_contexts_containing_name(
inf, module_contexts, search_name
)
for module_context in potential_modules:
for name_leaf in module_context.tree_node.get_used_names().get(search_name, []):
new = _find_names(module_context, name_leaf)
if any(tree_name in found_names for tree_name in new):
found_names.update(new)
for tree_name in new:
jedi/evaluate/utils.py → jedi/inference/utils.py
+1 -1
View File
@@ -100,7 +100,7 @@ class PushBackIterator(object):
@contextlib.contextmanager
def ignored(*exceptions):
"""
Context manager that ignores all of the specified exceptions. This will
Value manager that ignores all of the specified exceptions. This will
be in the standard library starting with Python 3.4.
"""
try:
jedi/inference/value/__init__.py
+6
View File
@@ -0,0 +1,6 @@
from jedi.inference.value.module import ModuleValue
from jedi.inference.value.klass import ClassValue
from jedi.inference.value.function import FunctionValue, \
MethodValue, FunctionExecutionContext
from jedi.inference.value.instance import AnonymousInstance, BoundMethod, \
CompiledInstance, AbstractInstanceValue, TreeInstance
jedi/inference/value/decorator.py
+15
View File
@@ -0,0 +1,15 @@
'''
Decorators are not really values, however we need some wrappers to improve
docstrings and other things around decorators.
'''
from jedi.inference.base_value import ValueWrapper
class Decoratee(ValueWrapper):
def __init__(self, wrapped_value, original_value):
self._wrapped_value = wrapped_value
self._original_value = original_value
def py__doc__(self):
return self._original_value.py__doc__()
jedi/evaluate/context/function.py → jedi/inference/value/function.py
+120 -145
View File
@@ -2,43 +2,43 @@ from parso.python import tree
from jedi._compatibility import use_metaclass
from jedi import debug
from jedi.evaluate.cache import evaluator_method_cache, CachedMetaClass
from jedi.evaluate import compiled
from jedi.evaluate import recursion
from jedi.evaluate import docstrings
from jedi.evaluate import flow_analysis
from jedi.evaluate import helpers
from jedi.evaluate.signature import TreeSignature
from jedi.evaluate.arguments import AnonymousArguments
from jedi.evaluate.filters import ParserTreeFilter, FunctionExecutionFilter
from jedi.evaluate.names import ContextName, AbstractNameDefinition, ParamName
from jedi.evaluate.base_context import ContextualizedNode, NO_CONTEXTS, \
ContextSet, TreeContext, ContextWrapper
from jedi.evaluate.lazy_context import LazyKnownContexts, LazyKnownContext, \
LazyTreeContext
from jedi.evaluate.context import iterable
from jedi.inference.cache import inference_state_method_cache, CachedMetaClass
from jedi.inference import compiled
from jedi.inference import recursion
from jedi.inference import docstrings
from jedi.inference import flow_analysis
from jedi.inference.signature import TreeSignature
from jedi.inference.arguments import AnonymousArguments
from jedi.inference.filters import ParserTreeFilter, FunctionExecutionFilter
from jedi.inference.names import ValueName, AbstractNameDefinition, ParamName
from jedi.inference.base_value import ContextualizedNode, NO_VALUES, \
ValueSet, TreeValue, ValueWrapper
from jedi.inference.lazy_value import LazyKnownValues, LazyKnownValue, \
LazyTreeValue
from jedi.inference.context import ValueContext, TreeContextMixin
from jedi.inference.value import iterable
from jedi import parser_utils
from jedi.evaluate.parser_cache import get_yield_exprs
from jedi.evaluate.helpers import contexts_from_qualified_names
from jedi.inference.parser_cache import get_yield_exprs
from jedi.inference.helpers import values_from_qualified_names
class LambdaName(AbstractNameDefinition):
string_name = '<lambda>'
api_type = u'function'
def __init__(self, lambda_context):
self._lambda_context = lambda_context
self.parent_context = lambda_context.parent_context
def __init__(self, lambda_value):
self._lambda_value = lambda_value
self.parent_context = lambda_value.parent_context
@property
def start_pos(self):
return self._lambda_context.tree_node.start_pos
return self._lambda_value.tree_node.start_pos
def infer(self):
return ContextSet([self._lambda_context])
return ValueSet([self._lambda_value])
class FunctionAndClassBase(TreeContext):
class FunctionAndClassBase(TreeValue):
def get_qualified_names(self):
if self.parent_context.is_class():
n = self.parent_context.get_qualified_names()
@@ -55,29 +55,21 @@ class FunctionAndClassBase(TreeContext):
class FunctionMixin(object):
api_type = u'function'
def get_filters(self, search_global=False, until_position=None, origin_scope=None):
if search_global:
yield ParserTreeFilter(
self.evaluator,
context=self,
until_position=until_position,
origin_scope=origin_scope
)
else:
cls = self.py__class__()
for instance in cls.execute_evaluated():
for filter in instance.get_filters(search_global=False, origin_scope=origin_scope):
yield filter
def get_filters(self, origin_scope=None):
cls = self.py__class__()
for instance in cls.execute_with_values():
for filter in instance.get_filters(origin_scope=origin_scope):
yield filter
def py__get__(self, instance, class_context):
from jedi.evaluate.context.instance import BoundMethod
def py__get__(self, instance, class_value):
from jedi.inference.value.instance import BoundMethod
if instance is None:
# Calling the Foo.bar results in the original bar function.
return ContextSet([self])
return ContextSet([BoundMethod(instance, self)])
return ValueSet([self])
return ValueSet([BoundMethod(instance, self)])
def get_param_names(self):
function_execution = self.get_function_execution()
function_execution = self.as_context()
return [ParamName(function_execution, param.name)
for param in self.tree_node.get_params()]
@@ -85,29 +77,25 @@ class FunctionMixin(object):
def name(self):
if self.tree_node.type == 'lambdef':
return LambdaName(self)
return ContextName(self, self.tree_node.name)
return ValueName(self, self.tree_node.name)
def py__name__(self):
return self.name.string_name
def py__call__(self, arguments):
function_execution = self.get_function_execution(arguments)
function_execution = self.as_context(arguments)
return function_execution.infer()
def get_function_execution(self, arguments=None):
def _as_context(self, arguments=None):
if arguments is None:
arguments = AnonymousArguments()
return FunctionExecutionContext(self.evaluator, self.parent_context, self, arguments)
return FunctionExecutionContext(self, arguments)
def get_signatures(self):
return [TreeSignature(f) for f in self.get_signature_functions()]
class FunctionContext(use_metaclass(CachedMetaClass, FunctionMixin, FunctionAndClassBase)):
"""
Needed because of decorators. Decorators are evaluated here.
"""
class FunctionValue(use_metaclass(CachedMetaClass, FunctionMixin, FunctionAndClassBase)):
def is_function(self):
return True
@@ -115,15 +103,15 @@ class FunctionContext(use_metaclass(CachedMetaClass, FunctionMixin, FunctionAndC
def from_context(cls, context, tree_node):
def create(tree_node):
if context.is_class():
return MethodContext(
context.evaluator,
return MethodValue(
context.inference_state,
context,
parent_context=parent_context,
tree_node=tree_node
)
else:
return cls(
context.evaluator,
context.inference_state,
parent_context=parent_context,
tree_node=tree_node
)
@@ -137,14 +125,14 @@ class FunctionContext(use_metaclass(CachedMetaClass, FunctionMixin, FunctionAndC
function = create(tree_node)
if overloaded_funcs:
return OverloadedFunctionContext(
return OverloadedFunctionValue(
function,
[create(f) for f in overloaded_funcs]
)
return function
def py__class__(self):
c, = contexts_from_qualified_names(self.evaluator, u'types', u'FunctionType')
c, = values_from_qualified_names(self.inference_state, u'types', u'FunctionType')
return c
def get_default_param_context(self):
@@ -154,62 +142,50 @@ class FunctionContext(use_metaclass(CachedMetaClass, FunctionMixin, FunctionAndC
return [self]
class MethodContext(FunctionContext):
def __init__(self, evaluator, class_context, *args, **kwargs):
super(MethodContext, self).__init__(evaluator, *args, **kwargs)
class MethodValue(FunctionValue):
def __init__(self, inference_state, class_context, *args, **kwargs):
super(MethodValue, self).__init__(inference_state, *args, **kwargs)
self.class_context = class_context
def get_default_param_context(self):
return self.class_context
def get_qualified_names(self):
# Need to implement this, because the parent context of a method
# context is not the class context but the module.
# Need to implement this, because the parent value of a method
# value is not the class value but the module.
names = self.class_context.get_qualified_names()
if names is None:
return None
return names + (self.py__name__(),)
class FunctionExecutionContext(TreeContext):
"""
This class is used to evaluate functions and their returns.
This is the most complicated class, because it contains the logic to
transfer parameters. It is even more complicated, because there may be
multiple calls to functions and recursion has to be avoided. But this is
responsibility of the decorators.
"""
class FunctionExecutionContext(ValueContext, TreeContextMixin):
function_execution_filter = FunctionExecutionFilter
def __init__(self, evaluator, parent_context, function_context, var_args):
super(FunctionExecutionContext, self).__init__(
evaluator,
parent_context,
function_context.tree_node,
)
self.function_context = function_context
def __init__(self, function_value, var_args):
super(FunctionExecutionContext, self).__init__(function_value)
self.function_value = function_value
self.var_args = var_args
@evaluator_method_cache(default=NO_CONTEXTS)
@inference_state_method_cache(default=NO_VALUES)
@recursion.execution_recursion_decorator()
def get_return_values(self, check_yields=False):
funcdef = self.tree_node
if funcdef.type == 'lambdef':
return self.eval_node(funcdef.children[-1])
return self.infer_node(funcdef.children[-1])
if check_yields:
context_set = NO_CONTEXTS
returns = get_yield_exprs(self.evaluator, funcdef)
value_set = NO_VALUES
returns = get_yield_exprs(self.inference_state, funcdef)
else:
returns = funcdef.iter_return_stmts()
from jedi.evaluate.gradual.annotation import infer_return_types
context_set = infer_return_types(self)
if context_set:
from jedi.inference.gradual.annotation import infer_return_types
value_set = infer_return_types(self)
if value_set:
# If there are annotations, prefer them over anything else.
# This will make it faster.
return context_set
context_set |= docstrings.infer_return_types(self.function_context)
return value_set
value_set |= docstrings.infer_return_types(self.function_value)
for r in returns:
check = flow_analysis.reachability_check(self, funcdef, r)
@@ -217,44 +193,44 @@ class FunctionExecutionContext(TreeContext):
debug.dbg('Return unreachable: %s', r)
else:
if check_yields:
context_set |= ContextSet.from_sets(
lazy_context.infer()
for lazy_context in self._get_yield_lazy_context(r)
value_set |= ValueSet.from_sets(
lazy_value.infer()
for lazy_value in self._get_yield_lazy_value(r)
)
else:
try:
children = r.children
except AttributeError:
ctx = compiled.builtin_from_name(self.evaluator, u'None')
context_set |= ContextSet([ctx])
ctx = compiled.builtin_from_name(self.inference_state, u'None')
value_set |= ValueSet([ctx])
else:
context_set |= self.eval_node(children[1])
value_set |= self.infer_node(children[1])
if check is flow_analysis.REACHABLE:
debug.dbg('Return reachable: %s', r)
break
return context_set
return value_set
def _get_yield_lazy_context(self, yield_expr):
def _get_yield_lazy_value(self, yield_expr):
if yield_expr.type == 'keyword':
# `yield` just yields None.
ctx = compiled.builtin_from_name(self.evaluator, u'None')
yield LazyKnownContext(ctx)
ctx = compiled.builtin_from_name(self.inference_state, u'None')
yield LazyKnownValue(ctx)
return
node = yield_expr.children[1]
if node.type == 'yield_arg': # It must be a yield from.
cn = ContextualizedNode(self, node.children[1])
for lazy_context in cn.infer().iterate(cn):
yield lazy_context
for lazy_value in cn.infer().iterate(cn):
yield lazy_value
else:
yield LazyTreeContext(self, node)
yield LazyTreeValue(self, node)
@recursion.execution_recursion_decorator(default=iter([]))
def get_yield_lazy_contexts(self, is_async=False):
def get_yield_lazy_values(self, is_async=False):
# TODO: if is_async, wrap yield statements in Awaitable/async_generator_asend
for_parents = [(y, tree.search_ancestor(y, 'for_stmt', 'funcdef',
'while_stmt', 'if_stmt'))
for y in get_yield_exprs(self.evaluator, self.tree_node)]
for y in get_yield_exprs(self.inference_state, self.tree_node)]
# Calculate if the yields are placed within the same for loop.
yields_order = []
@@ -276,7 +252,7 @@ class FunctionExecutionContext(TreeContext):
else:
types = self.get_return_values(check_yields=True)
if types:
yield LazyKnownContexts(types)
yield LazyKnownValues(types)
return
last_for_stmt = for_stmt
@@ -284,42 +260,42 @@ class FunctionExecutionContext(TreeContext):
if for_stmt is None:
# No for_stmt, just normal yields.
for yield_ in yields:
for result in self._get_yield_lazy_context(yield_):
for result in self._get_yield_lazy_value(yield_):
yield result
else:
input_node = for_stmt.get_testlist()
cn = ContextualizedNode(self, input_node)
ordered = cn.infer().iterate(cn)
ordered = list(ordered)
for lazy_context in ordered:
dct = {str(for_stmt.children[1].value): lazy_context.infer()}
with helpers.predefine_names(self, for_stmt, dct):
for lazy_value in ordered:
dct = {str(for_stmt.children[1].value): lazy_value.infer()}
with self.predefine_names(for_stmt, dct):
for yield_in_same_for_stmt in yields:
for result in self._get_yield_lazy_context(yield_in_same_for_stmt):
for result in self._get_yield_lazy_value(yield_in_same_for_stmt):
yield result
def merge_yield_contexts(self, is_async=False):
return ContextSet.from_sets(
lazy_context.infer()
for lazy_context in self.get_yield_lazy_contexts()
def merge_yield_values(self, is_async=False):
return ValueSet.from_sets(
lazy_value.infer()
for lazy_value in self.get_yield_lazy_values()
)
def get_filters(self, search_global=False, until_position=None, origin_scope=None):
yield self.function_execution_filter(self.evaluator, self,
def get_filters(self, until_position=None, origin_scope=None):
yield self.function_execution_filter(self,
until_position=until_position,
origin_scope=origin_scope)
@evaluator_method_cache()
def get_executed_params_and_issues(self):
return self.var_args.get_executed_params_and_issues(self)
@inference_state_method_cache()
def get_executed_param_names_and_issues(self):
return self.var_args.get_executed_param_names_and_issues(self)
def matches_signature(self):
executed_params, issues = self.get_executed_params_and_issues()
executed_param_names, issues = self.get_executed_param_names_and_issues()
if issues:
return False
matches = all(executed_param.matches_signature()
for executed_param in executed_params)
matches = all(executed_param_name.matches_signature()
for executed_param_name in executed_param_names)
if debug.enable_notice:
signature = parser_utils.get_call_signature(self.tree_node)
if matches:
@@ -334,67 +310,67 @@ class FunctionExecutionContext(TreeContext):
"""
Created to be used by inheritance.
"""
evaluator = self.evaluator
inference_state = self.inference_state
is_coroutine = self.tree_node.parent.type in ('async_stmt', 'async_funcdef')
is_generator = bool(get_yield_exprs(evaluator, self.tree_node))
from jedi.evaluate.gradual.typing import GenericClass
is_generator = bool(get_yield_exprs(inference_state, self.tree_node))
from jedi.inference.gradual.typing import GenericClass
if is_coroutine:
if is_generator:
if evaluator.environment.version_info < (3, 6):
return NO_CONTEXTS
async_generator_classes = evaluator.typing_module \
if inference_state.environment.version_info < (3, 6):
return NO_VALUES
async_generator_classes = inference_state.typing_module \
.py__getattribute__('AsyncGenerator')
yield_contexts = self.merge_yield_contexts(is_async=True)
yield_values = self.merge_yield_values(is_async=True)
# The contravariant doesn't seem to be defined.
generics = (yield_contexts.py__class__(), NO_CONTEXTS)
return ContextSet(
generics = (yield_values.py__class__(), NO_VALUES)
return ValueSet(
# In Python 3.6 AsyncGenerator is still a class.
GenericClass(c, generics)
for c in async_generator_classes
).execute_annotation()
else:
if evaluator.environment.version_info < (3, 5):
return NO_CONTEXTS
async_classes = evaluator.typing_module.py__getattribute__('Coroutine')
return_contexts = self.get_return_values()
if inference_state.environment.version_info < (3, 5):
return NO_VALUES
async_classes = inference_state.typing_module.py__getattribute__('Coroutine')
return_values = self.get_return_values()
# Only the first generic is relevant.
generics = (return_contexts.py__class__(), NO_CONTEXTS, NO_CONTEXTS)
return ContextSet(
generics = (return_values.py__class__(), NO_VALUES, NO_VALUES)
return ValueSet(
GenericClass(c, generics) for c in async_classes
).execute_annotation()
else:
if is_generator:
return ContextSet([iterable.Generator(evaluator, self)])
return ValueSet([iterable.Generator(inference_state, self)])
else:
return self.get_return_values()
class OverloadedFunctionContext(FunctionMixin, ContextWrapper):
class OverloadedFunctionValue(FunctionMixin, ValueWrapper):
def __init__(self, function, overloaded_functions):
super(OverloadedFunctionContext, self).__init__(function)
super(OverloadedFunctionValue, self).__init__(function)
self._overloaded_functions = overloaded_functions
def py__call__(self, arguments):
debug.dbg("Execute overloaded function %s", self._wrapped_context, color='BLUE')
debug.dbg("Execute overloaded function %s", self._wrapped_value, color='BLUE')
function_executions = []
context_set = NO_CONTEXTS
value_set = NO_VALUES
matched = False
for f in self._overloaded_functions:
function_execution = f.get_function_execution(arguments)
function_execution = f.as_context(arguments)
function_executions.append(function_execution)
if function_execution.matches_signature():
matched = True
return function_execution.infer()
if matched:
return context_set
return value_set
if self.evaluator.is_analysis:
if self.inference_state.is_analysis:
# In this case we want precision.
return NO_CONTEXTS
return ContextSet.from_sets(fe.infer() for fe in function_executions)
return NO_VALUES
return ValueSet.from_sets(fe.infer() for fe in function_executions)
def get_signature_functions(self):
return self._overloaded_functions
@@ -411,7 +387,7 @@ def _find_overload_functions(context, tree_node):
for decorator in decorators:
dotted_name = decorator.children[1]
if dotted_name.type == 'name' and dotted_name.value == 'overload':
# TODO check with contexts if it's the right overload
# TODO check with values if it's the right overload
return True
return False
@@ -423,7 +399,6 @@ def _find_overload_functions(context, tree_node):
while True:
filter = ParserTreeFilter(
context.evaluator,
context,
until_position=tree_node.start_pos
)
jedi/evaluate/context/instance.py → jedi/inference/value/instance.py
+162 -146
View File
@@ -2,33 +2,32 @@ from abc import abstractproperty
from jedi import debug
from jedi import settings
from jedi.evaluate import compiled
from jedi.evaluate.compiled.context import CompiledObjectFilter
from jedi.evaluate.helpers import contexts_from_qualified_names
from jedi.evaluate.filters import AbstractFilter
from jedi.evaluate.names import ContextName, TreeNameDefinition
from jedi.evaluate.base_context import Context, NO_CONTEXTS, ContextSet, \
iterator_to_context_set, ContextWrapper
from jedi.evaluate.lazy_context import LazyKnownContext, LazyKnownContexts
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.arguments import AnonymousArguments, \
from jedi.inference import compiled
from jedi.inference.compiled.value import CompiledObjectFilter
from jedi.inference.helpers import values_from_qualified_names
from jedi.inference.filters import AbstractFilter
from jedi.inference.names import ValueName, TreeNameDefinition, ParamName
from jedi.inference.base_value import Value, NO_VALUES, ValueSet, \
iterator_to_value_set, ValueWrapper
from jedi.inference.lazy_value import LazyKnownValue, LazyKnownValues
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.arguments import AnonymousArguments, \
ValuesArguments, TreeArgumentsWrapper
from jedi.evaluate.context.function import \
FunctionContext, FunctionMixin, OverloadedFunctionContext
from jedi.evaluate.context.klass import ClassContext, apply_py__get__, \
from jedi.inference.value.function import \
FunctionValue, FunctionMixin, OverloadedFunctionValue
from jedi.inference.value.klass import ClassValue, apply_py__get__, \
ClassFilter
from jedi.evaluate.context import iterable
from jedi.inference.value import iterable
from jedi.parser_utils import get_parent_scope
class InstanceExecutedParam(object):
def __init__(self, instance, tree_param):
class InstanceExecutedParamName(ParamName):
def __init__(self, instance, execution_context, tree_param):
super(InstanceExecutedParamName, self).__init__(execution_context, tree_param.name)
self._instance = instance
self._tree_param = tree_param
self.string_name = self._tree_param.name.value
def infer(self):
return ContextSet([self._instance])
return ValueSet([self._instance])
def matches_signature(self):
return True
@@ -38,58 +37,56 @@ class AnonymousInstanceArguments(AnonymousArguments):
def __init__(self, instance):
self._instance = instance
def get_executed_params_and_issues(self, execution_context):
from jedi.evaluate.dynamic import search_params
def get_executed_param_names_and_issues(self, execution_context):
from jedi.inference.dynamic import search_param_names
tree_params = execution_context.tree_node.get_params()
if not tree_params:
return [], []
self_param = InstanceExecutedParam(self._instance, tree_params[0])
self_param = InstanceExecutedParamName(
self._instance, execution_context, tree_params[0])
if len(tree_params) == 1:
# If the only param is self, we don't need to try to find
# executions of this function, we have all the params already.
return [self_param], []
executed_params = list(search_params(
execution_context.evaluator,
executed_param_names = list(search_param_names(
execution_context.inference_state,
execution_context,
execution_context.tree_node
))
executed_params[0] = self_param
return executed_params, []
executed_param_names[0] = self_param
return executed_param_names, []
class AbstractInstanceContext(Context):
"""
This class is used to evaluate instances.
"""
class AbstractInstanceValue(Value):
api_type = u'instance'
def __init__(self, evaluator, parent_context, class_context, var_args):
super(AbstractInstanceContext, self).__init__(evaluator, parent_context)
def __init__(self, inference_state, parent_context, class_value, var_args):
super(AbstractInstanceValue, self).__init__(inference_state, parent_context)
# Generated instances are classes that are just generated by self
# (No var_args) used.
self.class_context = class_context
self.class_value = class_value
self.var_args = var_args
def is_instance(self):
return True
def get_qualified_names(self):
return self.class_context.get_qualified_names()
return self.class_value.get_qualified_names()
def get_annotated_class_object(self):
return self.class_context # This is the default.
return self.class_value # This is the default.
def py__call__(self, arguments):
names = self.get_function_slot_names(u'__call__')
if not names:
# Means the Instance is not callable.
return super(AbstractInstanceContext, self).py__call__(arguments)
return super(AbstractInstanceValue, self).py__call__(arguments)
return ContextSet.from_sets(name.infer().execute(arguments) for name in names)
return ValueSet.from_sets(name.infer().execute(arguments) for name in names)
def py__class__(self):
return self.class_context
return self.class_value
def py__bool__(self):
# Signalize that we don't know about the bool type.
@@ -105,13 +102,13 @@ class AbstractInstanceContext(Context):
return names
return []
def execute_function_slots(self, names, *evaluated_args):
return ContextSet.from_sets(
name.infer().execute_evaluated(*evaluated_args)
def execute_function_slots(self, names, *inferred_args):
return ValueSet.from_sets(
name.infer().execute_with_values(*inferred_args)
for name in names
)
def py__get__(self, obj, class_context):
def py__get__(self, obj, class_value):
"""
obj may be None.
"""
@@ -120,71 +117,69 @@ class AbstractInstanceContext(Context):
names = self.get_function_slot_names(u'__get__')
if names:
if obj is None:
obj = compiled.builtin_from_name(self.evaluator, u'None')
return self.execute_function_slots(names, obj, class_context)
obj = compiled.builtin_from_name(self.inference_state, u'None')
return self.execute_function_slots(names, obj, class_value)
else:
return ContextSet([self])
return ValueSet([self])
def get_filters(self, search_global=None, until_position=None,
origin_scope=None, include_self_names=True):
class_context = self.get_annotated_class_object()
def get_filters(self, origin_scope=None, include_self_names=True):
class_value = self.get_annotated_class_object()
if include_self_names:
for cls in class_context.py__mro__():
for cls in class_value.py__mro__():
if not isinstance(cls, compiled.CompiledObject) \
or cls.tree_node is not None:
# In this case we're excluding compiled objects that are
# not fake objects. It doesn't make sense for normal
# compiled objects to search for self variables.
yield SelfAttributeFilter(self.evaluator, self, cls, origin_scope)
yield SelfAttributeFilter(self, class_value, cls.as_context(), origin_scope)
class_filters = class_context.get_filters(
search_global=False,
class_filters = class_value.get_filters(
origin_scope=origin_scope,
is_instance=True,
)
for f in class_filters:
if isinstance(f, ClassFilter):
yield InstanceClassFilter(self.evaluator, self, f)
yield InstanceClassFilter(self, f)
elif isinstance(f, CompiledObjectFilter):
yield CompiledInstanceClassFilter(self.evaluator, self, f)
yield CompiledInstanceClassFilter(self, f)
else:
# Propably from the metaclass.
yield f
def py__getitem__(self, index_context_set, contextualized_node):
def py__getitem__(self, index_value_set, contextualized_node):
names = self.get_function_slot_names(u'__getitem__')
if not names:
return super(AbstractInstanceContext, self).py__getitem__(
index_context_set,
return super(AbstractInstanceValue, self).py__getitem__(
index_value_set,
contextualized_node,
)
args = ValuesArguments([index_context_set])
return ContextSet.from_sets(name.infer().execute(args) for name in names)
args = ValuesArguments([index_value_set])
return ValueSet.from_sets(name.infer().execute(args) for name in names)
def py__iter__(self, contextualized_node=None):
iter_slot_names = self.get_function_slot_names(u'__iter__')
if not iter_slot_names:
return super(AbstractInstanceContext, self).py__iter__(contextualized_node)
return super(AbstractInstanceValue, self).py__iter__(contextualized_node)
def iterate():
for generator in self.execute_function_slots(iter_slot_names):
if generator.is_instance() and not generator.is_compiled():
# `__next__` logic.
if self.evaluator.environment.version_info.major == 2:
if self.inference_state.environment.version_info.major == 2:
name = u'next'
else:
name = u'__next__'
next_slot_names = generator.get_function_slot_names(name)
if next_slot_names:
yield LazyKnownContexts(
yield LazyKnownValues(
generator.execute_function_slots(next_slot_names)
)
else:
debug.warning('Instance has no __next__ function in %s.', generator)
else:
for lazy_context in generator.py__iter__():
yield lazy_context
for lazy_value in generator.py__iter__():
yield lazy_value
return iterate()
@abstractproperty
@@ -195,14 +190,14 @@ class AbstractInstanceContext(Context):
for name in self.get_function_slot_names(u'__init__'):
# TODO is this correct? I think we need to check for functions.
if isinstance(name, LazyInstanceClassName):
function = FunctionContext.from_context(
function = FunctionValue.from_context(
self.parent_context,
name.tree_name.parent
)
bound_method = BoundMethod(self, function)
yield bound_method.get_function_execution(self.var_args)
yield bound_method.as_context(self.var_args)
@evaluator_method_cache()
@inference_state_method_cache()
def create_instance_context(self, class_context, node):
if node.parent.type in ('funcdef', 'classdef'):
node = node.parent
@@ -212,18 +207,18 @@ class AbstractInstanceContext(Context):
else:
parent_context = self.create_instance_context(class_context, scope)
if scope.type == 'funcdef':
func = FunctionContext.from_context(
func = FunctionValue.from_context(
parent_context,
scope,
)
bound_method = BoundMethod(self, func)
if scope.name.value == '__init__' and parent_context == class_context:
return bound_method.get_function_execution(self.var_args)
return bound_method.as_context(self.var_args)
else:
return bound_method.get_function_execution()
return bound_method.as_context()
elif scope.type == 'classdef':
class_context = ClassContext(self.evaluator, parent_context, scope)
return class_context
class_context = ClassValue(self.inference_state, parent_context, scope)
return class_context.as_context()
elif scope.type in ('comp_for', 'sync_comp_for'):
# Comprehensions currently don't have a special scope in Jedi.
return self.create_instance_context(class_context, scope)
@@ -232,127 +227,146 @@ class AbstractInstanceContext(Context):
return class_context
def get_signatures(self):
call_funcs = self.py__getattribute__('__call__').py__get__(self, self.class_context)
call_funcs = self.py__getattribute__('__call__').py__get__(self, self.class_value)
return [s.bind(self) for s in call_funcs.get_signatures()]
def __repr__(self):
return "<%s of %s(%s)>" % (self.__class__.__name__, self.class_context,
return "<%s of %s(%s)>" % (self.__class__.__name__, self.class_value,
self.var_args)
class CompiledInstance(AbstractInstanceContext):
def __init__(self, evaluator, parent_context, class_context, var_args):
class CompiledInstance(AbstractInstanceValue):
def __init__(self, inference_state, parent_context, class_value, var_args):
self._original_var_args = var_args
super(CompiledInstance, self).__init__(evaluator, parent_context, class_context, var_args)
super(CompiledInstance, self).__init__(inference_state, parent_context, class_value, var_args)
@property
def name(self):
return compiled.CompiledContextName(self, self.class_context.name.string_name)
return compiled.CompiledValueName(self, self.class_value.name.string_name)
def get_first_non_keyword_argument_contexts(self):
key, lazy_context = next(self._original_var_args.unpack(), ('', None))
def get_first_non_keyword_argument_values(self):
key, lazy_value = next(self._original_var_args.unpack(), ('', None))
if key is not None:
return NO_CONTEXTS
return NO_VALUES
return lazy_context.infer()
return lazy_value.infer()
def is_stub(self):
return False
class TreeInstance(AbstractInstanceContext):
def __init__(self, evaluator, parent_context, class_context, var_args):
class TreeInstance(AbstractInstanceValue):
def __init__(self, inference_state, parent_context, class_value, var_args):
# I don't think that dynamic append lookups should happen here. That
# sounds more like something that should go to py__iter__.
if class_context.py__name__() in ['list', 'set'] \
and parent_context.get_root_context() == evaluator.builtins_module:
if class_value.py__name__() in ['list', 'set'] \
and parent_context.get_root_context().is_builtins_module():
# compare the module path with the builtin name.
if settings.dynamic_array_additions:
var_args = iterable.get_dynamic_array_instance(self, var_args)
super(TreeInstance, self).__init__(evaluator, parent_context,
class_context, var_args)
self.tree_node = class_context.tree_node
super(TreeInstance, self).__init__(inference_state, parent_context,
class_value, var_args)
self.tree_node = class_value.tree_node
@property
def name(self):
return ContextName(self, self.class_context.name.tree_name)
return ValueName(self, self.class_value.name.tree_name)
# This can recurse, if the initialization of the class includes a reference
# to itself.
@evaluator_method_cache(default=None)
@inference_state_method_cache(default=None)
def _get_annotated_class_object(self):
from jedi.evaluate.gradual.annotation import py__annotations__, \
from jedi.inference.gradual.annotation import py__annotations__, \
infer_type_vars_for_execution
for func in self._get_annotation_init_functions():
# Just take the first result, it should always be one, because we
# control the typeshed code.
bound = BoundMethod(self, func)
execution = bound.get_function_execution(self.var_args)
execution = bound.as_context(self.var_args)
if not execution.matches_signature():
# First check if the signature even matches, if not we don't
# need to infer anything.
continue
all_annotations = py__annotations__(execution.tree_node)
defined, = self.class_context.define_generics(
defined, = self.class_value.define_generics(
infer_type_vars_for_execution(execution, all_annotations),
)
debug.dbg('Inferred instance context as %s', defined, color='BLUE')
debug.dbg('Inferred instance value as %s', defined, color='BLUE')
return defined
return None
def get_annotated_class_object(self):
return self._get_annotated_class_object() or self.class_context
return self._get_annotated_class_object() or self.class_value
def _get_annotation_init_functions(self):
filter = next(self.class_context.get_filters())
filter = next(self.class_value.get_filters())
for init_name in filter.get('__init__'):
for init in init_name.infer():
if init.is_function():
for signature in init.get_signatures():
yield signature.context
yield signature.value
def py__getattribute__alternatives(self, string_name):
'''
Since nothing was inferred, now check the __getattr__ and
__getattribute__ methods. Stubs don't need to be checked, because
they don't contain any logic.
'''
if self.is_stub():
return NO_VALUES
name = compiled.create_simple_object(self.inference_state, string_name)
# This is a little bit special. `__getattribute__` is in Python
# executed before `__getattr__`. But: I know no use case, where
# this could be practical and where Jedi would return wrong types.
# If you ever find something, let me know!
# We are inversing this, because a hand-crafted `__getattribute__`
# could still call another hand-crafted `__getattr__`, but not the
# other way around.
names = (self.get_function_slot_names(u'__getattr__') or
self.get_function_slot_names(u'__getattribute__'))
return self.execute_function_slots(names, name)
class AnonymousInstance(TreeInstance):
def __init__(self, evaluator, parent_context, class_context):
def __init__(self, inference_state, parent_context, class_value):
super(AnonymousInstance, self).__init__(
evaluator,
inference_state,
parent_context,
class_context,
class_value,
var_args=AnonymousInstanceArguments(self),
)
def get_annotated_class_object(self):
return self.class_context # This is the default.
return self.class_value # This is the default.
class CompiledInstanceName(compiled.CompiledName):
def __init__(self, evaluator, instance, klass, name):
def __init__(self, inference_state, instance, klass, name):
super(CompiledInstanceName, self).__init__(
evaluator,
inference_state,
klass.parent_context,
name.string_name
)
self._instance = instance
self._class_member_name = name
@iterator_to_context_set
@iterator_to_value_set
def infer(self):
for result_context in self._class_member_name.infer():
if result_context.api_type == 'function':
yield CompiledBoundMethod(result_context)
for result_value in self._class_member_name.infer():
if result_value.api_type == 'function':
yield CompiledBoundMethod(result_value)
else:
yield result_context
yield result_value
class CompiledInstanceClassFilter(AbstractFilter):
name_class = CompiledInstanceName
def __init__(self, evaluator, instance, f):
self._evaluator = evaluator
def __init__(self, instance, f):
self._instance = instance
self._class_filter = f
@@ -365,12 +379,12 @@ class CompiledInstanceClassFilter(AbstractFilter):
def _convert(self, names):
klass = self._class_filter.compiled_object
return [
CompiledInstanceName(self._evaluator, self._instance, klass, n)
CompiledInstanceName(self._instance.inference_state, self._instance, klass, n)
for n in names
]
class BoundMethod(FunctionMixin, ContextWrapper):
class BoundMethod(FunctionMixin, ValueWrapper):
def __init__(self, instance, function):
super(BoundMethod, self).__init__(function)
self.instance = instance
@@ -379,7 +393,7 @@ class BoundMethod(FunctionMixin, ContextWrapper):
return True
def py__class__(self):
c, = contexts_from_qualified_names(self.evaluator, u'types', u'MethodType')
c, = values_from_qualified_names(self.inference_state, u'types', u'MethodType')
return c
def _get_arguments(self, arguments):
@@ -388,36 +402,36 @@ class BoundMethod(FunctionMixin, ContextWrapper):
return InstanceArguments(self.instance, arguments)
def get_function_execution(self, arguments=None):
def as_context(self, arguments=None):
arguments = self._get_arguments(arguments)
return super(BoundMethod, self).get_function_execution(arguments)
return super(BoundMethod, self).as_context(arguments)
def py__call__(self, arguments):
if isinstance(self._wrapped_context, OverloadedFunctionContext):
return self._wrapped_context.py__call__(self._get_arguments(arguments))
if isinstance(self._wrapped_value, OverloadedFunctionValue):
return self._wrapped_value.py__call__(self._get_arguments(arguments))
function_execution = self.get_function_execution(arguments)
function_execution = self.as_context(arguments)
return function_execution.infer()
def get_signature_functions(self):
return [
BoundMethod(self.instance, f)
for f in self._wrapped_context.get_signature_functions()
for f in self._wrapped_value.get_signature_functions()
]
def get_signatures(self):
return [sig.bind(self) for sig in super(BoundMethod, self).get_signatures()]
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self._wrapped_context)
return '<%s: %s>' % (self.__class__.__name__, self._wrapped_value)
class CompiledBoundMethod(ContextWrapper):
class CompiledBoundMethod(ValueWrapper):
def is_bound_method(self):
return True
def get_signatures(self):
return [sig.bind(self) for sig in self._wrapped_context.get_signatures()]
return [sig.bind(self) for sig in self._wrapped_value.get_signatures()]
class SelfName(TreeNameDefinition):
@@ -431,19 +445,21 @@ class SelfName(TreeNameDefinition):
@property
def parent_context(self):
return self._instance.create_instance_context(self.class_context, self.tree_name)
return self._instance.create_instance_context(
self.class_context,
self.tree_name
)
class LazyInstanceClassName(object):
def __init__(self, instance, class_context, class_member_name):
def __init__(self, instance, class_member_name):
self._instance = instance
self.class_context = class_context
self._class_member_name = class_member_name
@iterator_to_context_set
@iterator_to_value_set
def infer(self):
for result_context in self._class_member_name.infer():
for c in apply_py__get__(result_context, self._instance, self.class_context):
for result_value in self._class_member_name.infer():
for c in apply_py__get__(result_value, self._instance, self._instance.py__class__()):
yield c
def __getattr__(self, name):
@@ -456,10 +472,10 @@ class LazyInstanceClassName(object):
class InstanceClassFilter(AbstractFilter):
"""
This filter is special in that it uses the class filter and wraps the
resulting names in LazyINstanceClassName. The idea is that the class name
resulting names in LazyInstanceClassName. The idea is that the class name
filtering can be very flexible and always be reflected in instances.
"""
def __init__(self, evaluator, instance, class_filter):
def __init__(self, instance, class_filter):
self._instance = instance
self._class_filter = class_filter
@@ -470,7 +486,10 @@ class InstanceClassFilter(AbstractFilter):
return self._convert(self._class_filter.values(from_instance=True))
def _convert(self, names):
return [LazyInstanceClassName(self._instance, self._class_filter.context, n) for n in names]
return [
LazyInstanceClassName(self._instance, n)
for n in names
]
def __repr__(self):
return '<%s for %s>' % (self.__class__.__name__, self._class_filter.context)
@@ -480,17 +499,14 @@ class SelfAttributeFilter(ClassFilter):
"""
This class basically filters all the use cases where `self.*` was assigned.
"""
name_class = SelfName
def __init__(self, evaluator, context, class_context, origin_scope):
def __init__(self, instance, instance_class, node_context, origin_scope):
super(SelfAttributeFilter, self).__init__(
evaluator=evaluator,
context=context,
node_context=class_context,
class_value=instance_class,
node_context=node_context,
origin_scope=origin_scope,
is_instance=True,
)
self._class_context = class_context
self._instance = instance
def _filter(self, names):
names = self._filter_self_names(names)
@@ -508,7 +524,7 @@ class SelfAttributeFilter(ClassFilter):
yield name
def _convert_names(self, names):
return [self.name_class(self.context, self._class_context, name) for name in names]
return [SelfName(self._instance, self._node_context, name) for name in names]
def _check_flows(self, names):
return names
@@ -520,12 +536,12 @@ class InstanceArguments(TreeArgumentsWrapper):
self.instance = instance
def unpack(self, func=None):
yield None, LazyKnownContext(self.instance)
yield None, LazyKnownValue(self.instance)
for values in self._wrapped_arguments.unpack(func):
yield values
def get_executed_params_and_issues(self, execution_context):
def get_executed_param_names_and_issues(self, execution_context):
if isinstance(self._wrapped_arguments, AnonymousInstanceArguments):
return self._wrapped_arguments.get_executed_params_and_issues(execution_context)
return self._wrapped_arguments.get_executed_param_names_and_issues(execution_context)
return super(InstanceArguments, self).get_executed_params_and_issues(execution_context)
return super(InstanceArguments, self).get_executed_param_names_and_issues(execution_context)
jedi/evaluate/context/iterable.py → jedi/inference/value/iterable.py
+182 -194
View File
@@ -25,48 +25,48 @@ import sys
from jedi import debug
from jedi import settings
from jedi._compatibility import force_unicode, is_py3
from jedi.evaluate import compiled
from jedi.evaluate import analysis
from jedi.evaluate import recursion
from jedi.evaluate.lazy_context import LazyKnownContext, LazyKnownContexts, \
LazyTreeContext
from jedi.evaluate.helpers import get_int_or_none, is_string, \
predefine_names, evaluate_call_of_leaf, reraise_getitem_errors, \
SimpleGetItemNotFound
from jedi.evaluate.utils import safe_property, to_list
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.filters import ParserTreeFilter, LazyAttributeOverwrite, \
publish_method
from jedi.evaluate.base_context import ContextSet, Context, NO_CONTEXTS, \
TreeContext, ContextualizedNode, iterate_contexts, HelperContextMixin, _sentinel
from jedi.inference import compiled
from jedi.inference import analysis
from jedi.inference import recursion
from jedi.inference.lazy_value import LazyKnownValue, LazyKnownValues, \
LazyTreeValue
from jedi.inference.helpers import get_int_or_none, is_string, \
infer_call_of_leaf, reraise_getitem_errors, SimpleGetItemNotFound
from jedi.inference.utils import safe_property, to_list
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.filters import LazyAttributeOverwrite, publish_method
from jedi.inference.base_value import ValueSet, Value, NO_VALUES, \
ContextualizedNode, iterate_values, HelperValueMixin, sentinel, \
LazyValueWrapper
from jedi.parser_utils import get_sync_comp_fors
from jedi.inference.context import CompForContext
class IterableMixin(object):
def py__stop_iteration_returns(self):
return ContextSet([compiled.builtin_from_name(self.evaluator, u'None')])
return ValueSet([compiled.builtin_from_name(self.inference_state, u'None')])
# At the moment, safe values are simple values like "foo", 1 and not
# lists/dicts. Therefore as a small speed optimization we can just do the
# default instead of resolving the lazy wrapped contexts, that are just
# default instead of resolving the lazy wrapped values, that are just
# doing this in the end as well.
# This mostly speeds up patterns like `sys.version_info >= (3, 0)` in
# typeshed.
if sys.version_info[0] == 2:
# Python 2...........
def get_safe_value(self, default=_sentinel):
if default is _sentinel:
raise ValueError("There exists no safe value for context %s" % self)
def get_safe_value(self, default=sentinel):
if default is sentinel:
raise ValueError("There exists no safe value for value %s" % self)
return default
else:
get_safe_value = Context.get_safe_value
get_safe_value = Value.get_safe_value
class GeneratorBase(LazyAttributeOverwrite, IterableMixin):
array_type = None
def _get_wrapped_context(self):
generator, = self.evaluator.typing_module \
def _get_wrapped_value(self):
generator, = self.inference_state.typing_module \
.py__getattribute__('Generator') \
.execute_annotation()
return generator
@@ -79,30 +79,30 @@ class GeneratorBase(LazyAttributeOverwrite, IterableMixin):
@publish_method('__iter__')
def py__iter__(self, contextualized_node=None):
return ContextSet([self])
return ValueSet([self])
@publish_method('send')
@publish_method('next', python_version_match=2)
@publish_method('__next__', python_version_match=3)
def py__next__(self):
return ContextSet.from_sets(lazy_context.infer() for lazy_context in self.py__iter__())
return ValueSet.from_sets(lazy_value.infer() for lazy_value in self.py__iter__())
def py__stop_iteration_returns(self):
return ContextSet([compiled.builtin_from_name(self.evaluator, u'None')])
return ValueSet([compiled.builtin_from_name(self.inference_state, u'None')])
@property
def name(self):
return compiled.CompiledContextName(self, 'Generator')
return compiled.CompiledValueName(self, 'Generator')
class Generator(GeneratorBase):
"""Handling of `yield` functions."""
def __init__(self, evaluator, func_execution_context):
super(Generator, self).__init__(evaluator)
def __init__(self, inference_state, func_execution_context):
super(Generator, self).__init__(inference_state)
self._func_execution_context = func_execution_context
def py__iter__(self, contextualized_node=None):
return self._func_execution_context.get_yield_lazy_contexts()
return self._func_execution_context.get_yield_lazy_values()
def py__stop_iteration_returns(self):
return self._func_execution_context.get_return_values()
@@ -111,16 +111,7 @@ class Generator(GeneratorBase):
return "<%s of %s>" % (type(self).__name__, self._func_execution_context)
class CompForContext(TreeContext):
@classmethod
def from_comp_for(cls, parent_context, comp_for):
return cls(parent_context.evaluator, parent_context, comp_for)
def get_filters(self, search_global=False, until_position=None, origin_scope=None):
yield ParserTreeFilter(self.evaluator, self)
def comprehension_from_atom(evaluator, context, atom):
def comprehension_from_atom(inference_state, value, atom):
bracket = atom.children[0]
test_list_comp = atom.children[1]
@@ -131,8 +122,8 @@ def comprehension_from_atom(evaluator, context, atom):
sync_comp_for = sync_comp_for.children[1]
return DictComprehension(
evaluator,
context,
inference_state,
value,
sync_comp_for_node=sync_comp_for,
key_node=test_list_comp.children[0],
value_node=test_list_comp.children[2],
@@ -149,17 +140,17 @@ def comprehension_from_atom(evaluator, context, atom):
sync_comp_for = sync_comp_for.children[1]
return cls(
evaluator,
defining_context=context,
inference_state,
defining_context=value,
sync_comp_for_node=sync_comp_for,
entry_node=test_list_comp.children[0],
)
class ComprehensionMixin(object):
@evaluator_method_cache()
@inference_state_method_cache()
def _get_comp_for_context(self, parent_context, comp_for):
return CompForContext.from_comp_for(parent_context, comp_for)
return CompForContext(parent_context, comp_for)
def _nested(self, comp_fors, parent_context=None):
comp_for = comp_fors[0]
@@ -168,31 +159,31 @@ class ComprehensionMixin(object):
input_node = comp_for.children[3]
parent_context = parent_context or self._defining_context
input_types = parent_context.eval_node(input_node)
input_types = parent_context.infer_node(input_node)
# TODO: simulate await if self.is_async
cn = ContextualizedNode(parent_context, input_node)
iterated = input_types.iterate(cn, is_async=is_async)
exprlist = comp_for.children[1]
for i, lazy_context in enumerate(iterated):
types = lazy_context.infer()
for i, lazy_value in enumerate(iterated):
types = lazy_value.infer()
dct = unpack_tuple_to_dict(parent_context, types, exprlist)
context_ = self._get_comp_for_context(
context = self._get_comp_for_context(
parent_context,
comp_for,
)
with predefine_names(context_, comp_for, dct):
with context.predefine_names(comp_for, dct):
try:
for result in self._nested(comp_fors[1:], context_):
for result in self._nested(comp_fors[1:], context):
yield result
except IndexError:
iterated = context_.eval_node(self._entry_node)
iterated = context.infer_node(self._entry_node)
if self.array_type == 'dict':
yield iterated, context_.eval_node(self._value_node)
yield iterated, context.infer_node(self._value_node)
else:
yield iterated
@evaluator_method_cache(default=[])
@inference_state_method_cache(default=[])
@to_list
def _iterate(self):
comp_fors = tuple(get_sync_comp_fors(self._sync_comp_for_node))
@@ -201,7 +192,7 @@ class ComprehensionMixin(object):
def py__iter__(self, contextualized_node=None):
for set_ in self._iterate():
yield LazyKnownContexts(set_)
yield LazyKnownValues(set_)
def __repr__(self):
return "<%s of %s>" % (type(self).__name__, self._sync_comp_for_node)
@@ -209,7 +200,7 @@ class ComprehensionMixin(object):
class _DictMixin(object):
def _get_generics(self):
return tuple(c_set.py__class__() for c_set in self.get_mapping_item_contexts())
return tuple(c_set.py__class__() for c_set in self.get_mapping_item_values())
class Sequence(LazyAttributeOverwrite, IterableMixin):
@@ -217,14 +208,14 @@ class Sequence(LazyAttributeOverwrite, IterableMixin):
@property
def name(self):
return compiled.CompiledContextName(self, self.array_type)
return compiled.CompiledValueName(self, self.array_type)
def _get_generics(self):
return (self.merge_types_of_iterate().py__class__(),)
def _get_wrapped_context(self):
from jedi.evaluate.gradual.typing import GenericClass
klass = compiled.builtin_from_name(self.evaluator, self.array_type)
def _get_wrapped_value(self):
from jedi.inference.gradual.typing import GenericClass
klass = compiled.builtin_from_name(self.inference_state, self.array_type)
c, = GenericClass(klass, self._get_generics()).execute_annotation()
return c
@@ -232,22 +223,22 @@ class Sequence(LazyAttributeOverwrite, IterableMixin):
return None # We don't know the length, because of appends.
def py__class__(self):
return compiled.builtin_from_name(self.evaluator, self.array_type)
return compiled.builtin_from_name(self.inference_state, self.array_type)
@safe_property
def parent(self):
return self.evaluator.builtins_module
return self.inference_state.builtins_module
def py__getitem__(self, index_context_set, contextualized_node):
def py__getitem__(self, index_value_set, contextualized_node):
if self.array_type == 'dict':
return self._dict_values()
return iterate_contexts(ContextSet([self]))
return iterate_values(ValueSet([self]))
class _BaseComprehension(ComprehensionMixin):
def __init__(self, evaluator, defining_context, sync_comp_for_node, entry_node):
def __init__(self, inference_state, defining_context, sync_comp_for_node, entry_node):
assert sync_comp_for_node.type == 'sync_comp_for'
super(_BaseComprehension, self).__init__(evaluator)
super(_BaseComprehension, self).__init__(inference_state)
self._defining_context = defining_context
self._sync_comp_for_node = sync_comp_for_node
self._entry_node = entry_node
@@ -258,12 +249,12 @@ class ListComprehension(_BaseComprehension, Sequence):
def py__simple_getitem__(self, index):
if isinstance(index, slice):
return ContextSet([self])
return ValueSet([self])
all_types = list(self.py__iter__())
with reraise_getitem_errors(IndexError, TypeError):
lazy_context = all_types[index]
return lazy_context.infer()
lazy_value = all_types[index]
return lazy_value.infer()
class SetComprehension(_BaseComprehension, Sequence):
@@ -277,9 +268,9 @@ class GeneratorComprehension(_BaseComprehension, GeneratorBase):
class DictComprehension(ComprehensionMixin, Sequence):
array_type = u'dict'
def __init__(self, evaluator, defining_context, sync_comp_for_node, key_node, value_node):
def __init__(self, inference_state, defining_context, sync_comp_for_node, key_node, value_node):
assert sync_comp_for_node.type == 'sync_comp_for'
super(DictComprehension, self).__init__(evaluator)
super(DictComprehension, self).__init__(inference_state)
self._defining_context = defining_context
self._sync_comp_for_node = sync_comp_for_node
self._entry_node = key_node
@@ -287,46 +278,45 @@ class DictComprehension(ComprehensionMixin, Sequence):
def py__iter__(self, contextualized_node=None):
for keys, values in self._iterate():
yield LazyKnownContexts(keys)
yield LazyKnownValues(keys)
def py__simple_getitem__(self, index):
for keys, values in self._iterate():
for k in keys:
if isinstance(k, compiled.CompiledObject):
# Be careful in the future if refactoring, index could be a
# slice.
if k.get_safe_value(default=object()) == index:
return values
# Be careful in the future if refactoring, index could be a
# slice object.
if k.get_safe_value(default=object()) == index:
return values
raise SimpleGetItemNotFound()
def _dict_keys(self):
return ContextSet.from_sets(keys for keys, values in self._iterate())
return ValueSet.from_sets(keys for keys, values in self._iterate())
def _dict_values(self):
return ContextSet.from_sets(values for keys, values in self._iterate())
return ValueSet.from_sets(values for keys, values in self._iterate())
@publish_method('values')
def _imitate_values(self):
lazy_context = LazyKnownContexts(self._dict_values())
return ContextSet([FakeSequence(self.evaluator, u'list', [lazy_context])])
lazy_value = LazyKnownValues(self._dict_values())
return ValueSet([FakeSequence(self.inference_state, u'list', [lazy_value])])
@publish_method('items')
def _imitate_items(self):
lazy_contexts = [
LazyKnownContext(
lazy_values = [
LazyKnownValue(
FakeSequence(
self.evaluator,
self.inference_state,
u'tuple',
[LazyKnownContexts(key),
LazyKnownContexts(value)]
[LazyKnownValues(key),
LazyKnownValues(value)]
)
)
for key, value in self._iterate()
]
return ContextSet([FakeSequence(self.evaluator, u'list', lazy_contexts)])
return ValueSet([FakeSequence(self.inference_state, u'list', lazy_values)])
def get_mapping_item_contexts(self):
def get_mapping_item_values(self):
return self._dict_keys(), self._dict_values()
def exact_key_items(self):
@@ -335,44 +325,44 @@ class DictComprehension(ComprehensionMixin, Sequence):
return []
class SequenceLiteralContext(Sequence):
class SequenceLiteralValue(Sequence):
_TUPLE_LIKE = 'testlist_star_expr', 'testlist', 'subscriptlist'
mapping = {'(': u'tuple',
'[': u'list',
'{': u'set'}
def __init__(self, evaluator, defining_context, atom):
super(SequenceLiteralContext, self).__init__(evaluator)
def __init__(self, inference_state, defining_context, atom):
super(SequenceLiteralValue, self).__init__(inference_state)
self.atom = atom
self._defining_context = defining_context
if self.atom.type in self._TUPLE_LIKE:
self.array_type = u'tuple'
else:
self.array_type = SequenceLiteralContext.mapping[atom.children[0]]
self.array_type = SequenceLiteralValue.mapping[atom.children[0]]
"""The builtin name of the array (list, set, tuple or dict)."""
def py__simple_getitem__(self, index):
"""Here the index is an int/str. Raises IndexError/KeyError."""
if self.array_type == u'dict':
compiled_obj_index = compiled.create_simple_object(self.evaluator, index)
compiled_obj_index = compiled.create_simple_object(self.inference_state, index)
for key, value in self.get_tree_entries():
for k in self._defining_context.eval_node(key):
for k in self._defining_context.infer_node(key):
try:
method = k.execute_operation
except AttributeError:
pass
else:
if method(compiled_obj_index, u'==').get_safe_value():
return self._defining_context.eval_node(value)
return self._defining_context.infer_node(value)
raise SimpleGetItemNotFound('No key found in dictionary %s.' % self)
if isinstance(index, slice):
return ContextSet([self])
return ValueSet([self])
else:
with reraise_getitem_errors(TypeError, KeyError, IndexError):
node = self.get_tree_entries()[index]
return self._defining_context.eval_node(node)
return self._defining_context.infer_node(node)
def py__iter__(self, contextualized_node=None):
"""
@@ -381,21 +371,21 @@ class SequenceLiteralContext(Sequence):
"""
if self.array_type == u'dict':
# Get keys.
types = NO_CONTEXTS
types = NO_VALUES
for k, _ in self.get_tree_entries():
types |= self._defining_context.eval_node(k)
types |= self._defining_context.infer_node(k)
# We don't know which dict index comes first, therefore always
# yield all the types.
for _ in types:
yield LazyKnownContexts(types)
yield LazyKnownValues(types)
else:
for node in self.get_tree_entries():
if node == ':' or node.type == 'subscript':
# TODO this should probably use at least part of the code
# of eval_subscript_list.
yield LazyKnownContext(Slice(self._defining_context, None, None, None))
# of infer_subscript_list.
yield LazyKnownValue(Slice(self._defining_context, None, None, None))
else:
yield LazyTreeContext(self._defining_context, node)
yield LazyTreeValue(self._defining_context, node)
for addition in check_array_additions(self._defining_context, self):
yield addition
@@ -404,8 +394,8 @@ class SequenceLiteralContext(Sequence):
return len(self.get_tree_entries())
def _dict_values(self):
return ContextSet.from_sets(
self._defining_context.eval_node(v)
return ValueSet.from_sets(
self._defining_context.infer_node(v)
for k, v in self.get_tree_entries()
)
@@ -457,97 +447,97 @@ class SequenceLiteralContext(Sequence):
def exact_key_items(self):
"""
Returns a generator of tuples like dict.items(), where the key is
resolved (as a string) and the values are still lazy contexts.
resolved (as a string) and the values are still lazy values.
"""
for key_node, value in self.get_tree_entries():
for key in self._defining_context.eval_node(key_node):
for key in self._defining_context.infer_node(key_node):
if is_string(key):
yield key.get_safe_value(), LazyTreeContext(self._defining_context, value)
yield key.get_safe_value(), LazyTreeValue(self._defining_context, value)
def __repr__(self):
return "<%s of %s>" % (self.__class__.__name__, self.atom)
class DictLiteralContext(_DictMixin, SequenceLiteralContext):
class DictLiteralValue(_DictMixin, SequenceLiteralValue):
array_type = u'dict'
def __init__(self, evaluator, defining_context, atom):
super(SequenceLiteralContext, self).__init__(evaluator)
def __init__(self, inference_state, defining_context, atom):
super(SequenceLiteralValue, self).__init__(inference_state)
self._defining_context = defining_context
self.atom = atom
@publish_method('values')
def _imitate_values(self):
lazy_context = LazyKnownContexts(self._dict_values())
return ContextSet([FakeSequence(self.evaluator, u'list', [lazy_context])])
lazy_value = LazyKnownValues(self._dict_values())
return ValueSet([FakeSequence(self.inference_state, u'list', [lazy_value])])
@publish_method('items')
def _imitate_items(self):
lazy_contexts = [
LazyKnownContext(FakeSequence(
self.evaluator, u'tuple',
(LazyTreeContext(self._defining_context, key_node),
LazyTreeContext(self._defining_context, value_node))
lazy_values = [
LazyKnownValue(FakeSequence(
self.inference_state, u'tuple',
(LazyTreeValue(self._defining_context, key_node),
LazyTreeValue(self._defining_context, value_node))
)) for key_node, value_node in self.get_tree_entries()
]
return ContextSet([FakeSequence(self.evaluator, u'list', lazy_contexts)])
return ValueSet([FakeSequence(self.inference_state, u'list', lazy_values)])
def _dict_keys(self):
return ContextSet.from_sets(
self._defining_context.eval_node(k)
return ValueSet.from_sets(
self._defining_context.infer_node(k)
for k, v in self.get_tree_entries()
)
def get_mapping_item_contexts(self):
def get_mapping_item_values(self):
return self._dict_keys(), self._dict_values()
class _FakeArray(SequenceLiteralContext):
def __init__(self, evaluator, container, type):
super(SequenceLiteralContext, self).__init__(evaluator)
class _FakeArray(SequenceLiteralValue):
def __init__(self, inference_state, container, type):
super(SequenceLiteralValue, self).__init__(inference_state)
self.array_type = type
self.atom = container
# TODO is this class really needed?
class FakeSequence(_FakeArray):
def __init__(self, evaluator, array_type, lazy_context_list):
def __init__(self, inference_state, array_type, lazy_value_list):
"""
type should be one of "tuple", "list"
"""
super(FakeSequence, self).__init__(evaluator, None, array_type)
self._lazy_context_list = lazy_context_list
super(FakeSequence, self).__init__(inference_state, None, array_type)
self._lazy_value_list = lazy_value_list
def py__simple_getitem__(self, index):
if isinstance(index, slice):
return ContextSet([self])
return ValueSet([self])
with reraise_getitem_errors(IndexError, TypeError):
lazy_context = self._lazy_context_list[index]
return lazy_context.infer()
lazy_value = self._lazy_value_list[index]
return lazy_value.infer()
def py__iter__(self, contextualized_node=None):
return self._lazy_context_list
return self._lazy_value_list
def py__bool__(self):
return bool(len(self._lazy_context_list))
return bool(len(self._lazy_value_list))
def __repr__(self):
return "<%s of %s>" % (type(self).__name__, self._lazy_context_list)
return "<%s of %s>" % (type(self).__name__, self._lazy_value_list)
class FakeDict(_DictMixin, _FakeArray):
def __init__(self, evaluator, dct):
super(FakeDict, self).__init__(evaluator, dct, u'dict')
def __init__(self, inference_state, dct):
super(FakeDict, self).__init__(inference_state, dct, u'dict')
self._dct = dct
def py__iter__(self, contextualized_node=None):
for key in self._dct:
yield LazyKnownContext(compiled.create_simple_object(self.evaluator, key))
yield LazyKnownValue(compiled.create_simple_object(self.inference_state, key))
def py__simple_getitem__(self, index):
if is_py3 and self.evaluator.environment.version_info.major == 2:
if is_py3 and self.inference_state.environment.version_info.major == 2:
# In Python 2 bytes and unicode compare.
if isinstance(index, bytes):
index_unicode = force_unicode(index)
@@ -563,23 +553,23 @@ class FakeDict(_DictMixin, _FakeArray):
pass
with reraise_getitem_errors(KeyError, TypeError):
lazy_context = self._dct[index]
return lazy_context.infer()
lazy_value = self._dct[index]
return lazy_value.infer()
@publish_method('values')
def _values(self):
return ContextSet([FakeSequence(
self.evaluator, u'tuple',
[LazyKnownContexts(self._dict_values())]
return ValueSet([FakeSequence(
self.inference_state, u'tuple',
[LazyKnownValues(self._dict_values())]
)])
def _dict_values(self):
return ContextSet.from_sets(lazy_context.infer() for lazy_context in self._dct.values())
return ValueSet.from_sets(lazy_value.infer() for lazy_value in self._dct.values())
def _dict_keys(self):
return ContextSet.from_sets(lazy_context.infer() for lazy_context in self.py__iter__())
return ValueSet.from_sets(lazy_value.infer() for lazy_value in self.py__iter__())
def get_mapping_item_contexts(self):
def get_mapping_item_values(self):
return self._dict_keys(), self._dict_values()
def exact_key_items(self):
@@ -587,17 +577,17 @@ class FakeDict(_DictMixin, _FakeArray):
class MergedArray(_FakeArray):
def __init__(self, evaluator, arrays):
super(MergedArray, self).__init__(evaluator, arrays, arrays[-1].array_type)
def __init__(self, inference_state, arrays):
super(MergedArray, self).__init__(inference_state, arrays, arrays[-1].array_type)
self._arrays = arrays
def py__iter__(self, contextualized_node=None):
for array in self._arrays:
for lazy_context in array.py__iter__():
yield lazy_context
for lazy_value in array.py__iter__():
yield lazy_value
def py__simple_getitem__(self, index):
return ContextSet.from_sets(lazy_context.infer() for lazy_context in self.py__iter__())
return ValueSet.from_sets(lazy_value.infer() for lazy_value in self.py__iter__())
def get_tree_entries(self):
for array in self._arrays:
@@ -608,33 +598,33 @@ class MergedArray(_FakeArray):
return sum(len(a) for a in self._arrays)
def unpack_tuple_to_dict(context, types, exprlist):
def unpack_tuple_to_dict(value, types, exprlist):
"""
Unpacking tuple assignments in for statements and expr_stmts.
"""
if exprlist.type == 'name':
return {exprlist.value: types}
elif exprlist.type == 'atom' and exprlist.children[0] in ('(', '['):
return unpack_tuple_to_dict(context, types, exprlist.children[1])
return unpack_tuple_to_dict(value, types, exprlist.children[1])
elif exprlist.type in ('testlist', 'testlist_comp', 'exprlist',
'testlist_star_expr'):
dct = {}
parts = iter(exprlist.children[::2])
n = 0
for lazy_context in types.iterate(exprlist):
for lazy_value in types.iterate(exprlist):
n += 1
try:
part = next(parts)
except StopIteration:
# TODO this context is probably not right.
analysis.add(context, 'value-error-too-many-values', part,
# TODO this value is probably not right.
analysis.add(value, 'value-error-too-many-values', part,
message="ValueError: too many values to unpack (expected %s)" % n)
else:
dct.update(unpack_tuple_to_dict(context, lazy_context.infer(), part))
dct.update(unpack_tuple_to_dict(value, lazy_value.infer(), part))
has_parts = next(parts, None)
if types and has_parts is not None:
# TODO this context is probably not right.
analysis.add(context, 'value-error-too-few-values', has_parts,
# TODO this value is probably not right.
analysis.add(value, 'value-error-too-few-values', has_parts,
message="ValueError: need more than %s values to unpack" % n)
return dct
elif exprlist.type == 'power' or exprlist.type == 'atom_expr':
@@ -652,12 +642,12 @@ def check_array_additions(context, sequence):
""" Just a mapper function for the internal _check_array_additions """
if sequence.array_type not in ('list', 'set'):
# TODO also check for dict updates
return NO_CONTEXTS
return NO_VALUES
return _check_array_additions(context, sequence)
@evaluator_method_cache(default=NO_CONTEXTS)
@inference_state_method_cache(default=NO_VALUES)
@debug.increase_indent
def _check_array_additions(context, sequence):
"""
@@ -666,25 +656,25 @@ def _check_array_additions(context, sequence):
>>> a = [""]
>>> a.append(1)
"""
from jedi.evaluate import arguments
from jedi.inference import arguments
debug.dbg('Dynamic array search for %s' % sequence, color='MAGENTA')
module_context = context.get_root_context()
if not settings.dynamic_array_additions or isinstance(module_context, compiled.CompiledObject):
if not settings.dynamic_array_additions or module_context.is_compiled():
debug.dbg('Dynamic array search aborted.', color='MAGENTA')
return NO_CONTEXTS
return NO_VALUES
def find_additions(context, arglist, add_name):
params = list(arguments.TreeArguments(context.evaluator, context, arglist).unpack())
params = list(arguments.TreeArguments(context.inference_state, context, arglist).unpack())
result = set()
if add_name in ['insert']:
params = params[1:]
if add_name in ['append', 'add', 'insert']:
for key, lazy_context in params:
result.add(lazy_context)
for key, lazy_value in params:
result.add(lazy_value)
elif add_name in ['extend', 'update']:
for key, lazy_context in params:
result |= set(lazy_context.infer().iterate())
for key, lazy_value in params:
result |= set(lazy_value.infer().iterate())
return result
temp_param_add, settings.dynamic_params_for_other_modules = \
@@ -701,8 +691,8 @@ def _check_array_additions(context, sequence):
continue
else:
for name in possible_names:
context_node = context.tree_node
if not (context_node.start_pos < name.start_pos < context_node.end_pos):
value_node = context.tree_node
if not (value_node.start_pos < name.start_pos < value_node.end_pos):
continue
trailer = name.parent
power = trailer.parent
@@ -719,9 +709,9 @@ def _check_array_additions(context, sequence):
random_context = context.create_context(name)
with recursion.execution_allowed(context.evaluator, power) as allowed:
with recursion.execution_allowed(context.inference_state, power) as allowed:
if allowed:
found = evaluate_call_of_leaf(
found = infer_call_of_leaf(
random_context,
name,
cut_own_trailer=True
@@ -743,11 +733,11 @@ def _check_array_additions(context, sequence):
def get_dynamic_array_instance(instance, arguments):
"""Used for set() and list() instances."""
ai = _ArrayInstance(instance, arguments)
from jedi.evaluate import arguments
return arguments.ValuesArguments([ContextSet([ai])])
from jedi.inference import arguments
return arguments.ValuesArguments([ValueSet([ai])])
class _ArrayInstance(HelperContextMixin):
class _ArrayInstance(HelperValueMixin):
"""
Used for the usage of set() and list().
This is definitely a hack, but a good one :-)
@@ -758,20 +748,20 @@ class _ArrayInstance(HelperContextMixin):
self.var_args = var_args
def py__class__(self):
tuple_, = self.instance.evaluator.builtins_module.py__getattribute__('tuple')
tuple_, = self.instance.inference_state.builtins_module.py__getattribute__('tuple')
return tuple_
def py__iter__(self, contextualized_node=None):
var_args = self.var_args
try:
_, lazy_context = next(var_args.unpack())
_, lazy_value = next(var_args.unpack())
except StopIteration:
pass
else:
for lazy in lazy_context.infer().iterate():
for lazy in lazy_value.infer().iterate():
yield lazy
from jedi.evaluate import arguments
from jedi.inference import arguments
if isinstance(var_args, arguments.TreeArguments):
additions = _check_array_additions(var_args.context, self.instance)
for addition in additions:
@@ -781,23 +771,21 @@ class _ArrayInstance(HelperContextMixin):
return self.py__iter__(contextualized_node)
class Slice(object):
def __init__(self, context, start, stop, step):
self._context = context
self._slice_object = None
class Slice(LazyValueWrapper):
def __init__(self, python_context, start, stop, step):
self.inference_state = python_context.inference_state
self._context = python_context
# All of them are either a Precedence or None.
self._start = start
self._stop = stop
self._step = step
def __getattr__(self, name):
if self._slice_object is None:
context = compiled.builtin_from_name(self._context.evaluator, 'slice')
self._slice_object, = context.execute_evaluated()
return getattr(self._slice_object, name)
def _get_wrapped_value(self):
value = compiled.builtin_from_name(self._context.inference_state, 'slice')
slice_value, = value.execute_with_values()
return slice_value
@property
def obj(self):
def get_safe_value(self, default=sentinel):
"""
Imitate CompiledObject.obj behavior and return a ``builtin.slice()``
object.
@@ -806,14 +794,14 @@ class Slice(object):
if element is None:
return None
result = self._context.eval_node(element)
result = self._context.infer_node(element)
if len(result) != 1:
# For simplicity, we want slices to be clear defined with just
# one type. Otherwise we will return an empty slice object.
raise IndexError
context, = result
return get_int_or_none(context)
value, = result
return get_int_or_none(value)
try:
return slice(get(self._start), get(self._stop), get(self._step))
jedi/evaluate/context/klass.py → jedi/inference/value/klass.py
+96 -103
View File
@@ -25,77 +25,83 @@ py__iter__() Returns a generator of a set of types.
py__class__() Returns the class of an instance.
py__simple_getitem__(index: int/str) Returns a a set of types of the index.
Can raise an IndexError/KeyError.
py__getitem__(indexes: ContextSet) Returns a a set of types of the index.
py__getitem__(indexes: ValueSet) Returns a a set of types of the index.
py__file__() Only on modules. Returns None if does
not exist.
py__package__() -> List[str] Only on modules. For the import system.
py__path__() Only on modules. For the import system.
py__get__(call_object) Only on instances. Simulates
descriptors.
py__doc__() Returns the docstring for a context.
py__doc__() Returns the docstring for a value.
====================================== ========================================
"""
from jedi import debug
from jedi._compatibility import use_metaclass
from jedi.parser_utils import get_cached_parent_scope
from jedi.evaluate.cache import evaluator_method_cache, CachedMetaClass, \
evaluator_method_generator_cache
from jedi.evaluate import compiled
from jedi.evaluate.lazy_context import LazyKnownContexts
from jedi.evaluate.filters import ParserTreeFilter
from jedi.evaluate.names import TreeNameDefinition, ContextName
from jedi.evaluate.arguments import unpack_arglist, ValuesArguments
from jedi.evaluate.base_context import ContextSet, iterator_to_context_set, \
NO_CONTEXTS
from jedi.evaluate.context.function import FunctionAndClassBase
from jedi.inference.cache import inference_state_method_cache, CachedMetaClass, \
inference_state_method_generator_cache
from jedi.inference import compiled
from jedi.inference.lazy_value import LazyKnownValues
from jedi.inference.filters import ParserTreeFilter
from jedi.inference.names import TreeNameDefinition, ValueName
from jedi.inference.arguments import unpack_arglist, ValuesArguments
from jedi.inference.base_value import ValueSet, iterator_to_value_set, \
NO_VALUES
from jedi.inference.context import ClassContext
from jedi.inference.value.function import FunctionAndClassBase
from jedi.plugins import plugin_manager
def apply_py__get__(context, instance, class_context):
def apply_py__get__(value, instance, class_value):
try:
method = context.py__get__
method = value.py__get__
except AttributeError:
yield context
yield value
else:
for descriptor_context in method(instance, class_context):
yield descriptor_context
for descriptor_value in method(instance, class_value):
yield descriptor_value
class ClassName(TreeNameDefinition):
def __init__(self, parent_context, tree_name, name_context, apply_decorators):
super(ClassName, self).__init__(parent_context, tree_name)
def __init__(self, class_value, tree_name, name_context, apply_decorators):
super(ClassName, self).__init__(class_value.as_context(), tree_name)
self._name_context = name_context
self._apply_decorators = apply_decorators
self._class_value = class_value
@iterator_to_context_set
@iterator_to_value_set
def infer(self):
# We're using a different context to infer, so we cannot call super().
from jedi.evaluate.syntax_tree import tree_name_to_contexts
inferred = tree_name_to_contexts(
self.parent_context.evaluator, self._name_context, self.tree_name)
# We're using a different value to infer, so we cannot call super().
from jedi.inference.syntax_tree import tree_name_to_values
inferred = tree_name_to_values(
self.parent_context.inference_state, self._name_context, self.tree_name)
for result_context in inferred:
for result_value in inferred:
if self._apply_decorators:
for c in apply_py__get__(result_context,
for c in apply_py__get__(result_value,
instance=None,
class_context=self.parent_context):
class_value=self._class_value):
yield c
else:
yield result_context
yield result_value
class ClassFilter(ParserTreeFilter):
name_class = ClassName
def __init__(self, *args, **kwargs):
self._is_instance = kwargs.pop('is_instance') # Python 2 :/
super(ClassFilter, self).__init__(*args, **kwargs)
def __init__(self, class_value, node_context=None, until_position=None,
origin_scope=None, is_instance=False):
super(ClassFilter, self).__init__(
class_value.as_context(), node_context,
until_position=until_position,
origin_scope=origin_scope,
)
self._class_value = class_value
self._is_instance = is_instance
def _convert_names(self, names):
return [
self.name_class(
parent_context=self.context,
ClassName(
class_value=self._class_value,
tree_name=name,
name_context=self._node_context,
apply_decorators=not self._is_instance,
@@ -105,7 +111,7 @@ class ClassFilter(ParserTreeFilter):
def _equals_origin_scope(self):
node = self._origin_scope
while node is not None:
if node == self._parser_scope or node == self.context:
if node == self._parser_scope or node == self.parent_context:
return True
node = get_cached_parent_scope(self._used_names, node)
return False
@@ -138,28 +144,28 @@ class ClassMixin(object):
return True
def py__call__(self, arguments=None):
from jedi.evaluate.context import TreeInstance
from jedi.inference.value import TreeInstance
if arguments is None:
arguments = ValuesArguments([])
return ContextSet([TreeInstance(self.evaluator, self.parent_context, self, arguments)])
return ValueSet([TreeInstance(self.inference_state, self.parent_context, self, arguments)])
def py__class__(self):
return compiled.builtin_from_name(self.evaluator, u'type')
return compiled.builtin_from_name(self.inference_state, u'type')
@property
def name(self):
return ContextName(self, self.tree_node.name)
return ValueName(self, self.tree_node.name)
def py__name__(self):
return self.name.string_name
def get_param_names(self):
for context_ in self.py__getattribute__(u'__init__'):
if context_.is_function():
return list(context_.get_param_names())[1:]
for value_ in self.py__getattribute__(u'__init__'):
if value_.is_function():
return list(value_.get_param_names())[1:]
return []
@evaluator_method_generator_cache()
@inference_state_method_generator_cache()
def py__mro__(self):
mro = [self]
yield self
@@ -192,30 +198,26 @@ class ClassMixin(object):
mro.append(cls_new)
yield cls_new
def get_filters(self, search_global=False, until_position=None,
origin_scope=None, is_instance=False):
def get_filters(self, origin_scope=None, is_instance=False):
metaclasses = self.get_metaclasses()
if metaclasses:
for f in self.get_metaclass_filters(metaclasses):
yield f
if search_global:
yield self.get_global_filter(until_position, origin_scope)
else:
for cls in self.py__mro__():
if isinstance(cls, compiled.CompiledObject):
for filter in cls.get_filters(is_instance=is_instance):
yield filter
else:
yield ClassFilter(
self.evaluator, self, node_context=cls,
origin_scope=origin_scope,
is_instance=is_instance
)
for cls in self.py__mro__():
if isinstance(cls, compiled.CompiledObject):
for filter in cls.get_filters(is_instance=is_instance):
yield filter
else:
yield ClassFilter(
self, node_context=cls.as_context(),
origin_scope=origin_scope,
is_instance=is_instance
)
if not is_instance:
from jedi.evaluate.compiled import builtin_from_name
type_ = builtin_from_name(self.evaluator, u'type')
assert isinstance(type_, ClassContext)
from jedi.inference.compiled import builtin_from_name
type_ = builtin_from_name(self.inference_state, u'type')
assert isinstance(type_, ClassValue)
if type_ != self:
for instance in type_.py__call__():
instance_filters = instance.get_filters()
@@ -228,23 +230,14 @@ class ClassMixin(object):
init_funcs = self.py__call__().py__getattribute__('__init__')
return [sig.bind(self) for sig in init_funcs.get_signatures()]
def get_global_filter(self, until_position=None, origin_scope=None):
return ParserTreeFilter(
self.evaluator,
context=self,
until_position=until_position,
origin_scope=origin_scope
)
def _as_context(self):
return ClassContext(self)
class ClassContext(use_metaclass(CachedMetaClass, ClassMixin, FunctionAndClassBase)):
"""
This class is not only important to extend `tree.Class`, it is also a
important for descriptors (if the descriptor methods are evaluated or not).
"""
class ClassValue(use_metaclass(CachedMetaClass, ClassMixin, FunctionAndClassBase)):
api_type = u'class'
@evaluator_method_cache()
@inference_state_method_cache()
def list_type_vars(self):
found = []
arglist = self.tree_node.get_super_arglist()
@@ -255,7 +248,7 @@ class ClassContext(use_metaclass(CachedMetaClass, ClassMixin, FunctionAndClassBa
if stars:
continue # These are not relevant for this search.
from jedi.evaluate.gradual.annotation import find_unknown_type_vars
from jedi.inference.gradual.annotation import find_unknown_type_vars
for type_var in find_unknown_type_vars(self.parent_context, node):
if type_var not in found:
# The order matters and it's therefore a list.
@@ -265,11 +258,11 @@ class ClassContext(use_metaclass(CachedMetaClass, ClassMixin, FunctionAndClassBa
def _get_bases_arguments(self):
arglist = self.tree_node.get_super_arglist()
if arglist:
from jedi.evaluate import arguments
return arguments.TreeArguments(self.evaluator, self.parent_context, arglist)
from jedi.inference import arguments
return arguments.TreeArguments(self.inference_state, self.parent_context, arglist)
return None
@evaluator_method_cache(default=())
@inference_state_method_cache(default=())
def py__bases__(self):
args = self._get_bases_arguments()
if args is not None:
@@ -278,27 +271,27 @@ class ClassContext(use_metaclass(CachedMetaClass, ClassMixin, FunctionAndClassBa
return lst
if self.py__name__() == 'object' \
and self.parent_context == self.evaluator.builtins_module:
and self.parent_context.is_builtins_module():
return []
return [LazyKnownContexts(
self.evaluator.builtins_module.py__getattribute__('object')
return [LazyKnownValues(
self.inference_state.builtins_module.py__getattribute__('object')
)]
def py__getitem__(self, index_context_set, contextualized_node):
from jedi.evaluate.gradual.typing import LazyGenericClass
if not index_context_set:
return ContextSet([self])
return ContextSet(
def py__getitem__(self, index_value_set, contextualized_node):
from jedi.inference.gradual.typing import LazyGenericClass
if not index_value_set:
return ValueSet([self])
return ValueSet(
LazyGenericClass(
self,
index_context,
context_of_index=contextualized_node.context,
index_value,
value_of_index=contextualized_node.context,
)
for index_context in index_context_set
for index_value in index_value_set
)
def define_generics(self, type_var_dict):
from jedi.evaluate.gradual.typing import GenericClass
from jedi.inference.gradual.typing import GenericClass
def remap_type_vars():
"""
@@ -311,34 +304,34 @@ class ClassContext(use_metaclass(CachedMetaClass, ClassMixin, FunctionAndClassBa
a different type var name.
"""
for type_var in self.list_type_vars():
yield type_var_dict.get(type_var.py__name__(), NO_CONTEXTS)
yield type_var_dict.get(type_var.py__name__(), NO_VALUES)
if type_var_dict:
return ContextSet([GenericClass(
return ValueSet([GenericClass(
self,
generics=tuple(remap_type_vars())
)])
return ContextSet({self})
return ValueSet({self})
@plugin_manager.decorate()
def get_metaclass_filters(self, metaclass):
debug.dbg('Unprocessed metaclass %s', metaclass)
return []
@evaluator_method_cache(default=NO_CONTEXTS)
@inference_state_method_cache(default=NO_VALUES)
def get_metaclasses(self):
args = self._get_bases_arguments()
if args is not None:
m = [value for key, value in args.unpack() if key == 'metaclass']
metaclasses = ContextSet.from_sets(lazy_context.infer() for lazy_context in m)
metaclasses = ContextSet(m for m in metaclasses if m.is_class())
metaclasses = ValueSet.from_sets(lazy_value.infer() for lazy_value in m)
metaclasses = ValueSet(m for m in metaclasses if m.is_class())
if metaclasses:
return metaclasses
for lazy_base in self.py__bases__():
for context in lazy_base.infer():
if context.is_class():
contexts = context.get_metaclasses()
if contexts:
return contexts
return NO_CONTEXTS
for value in lazy_base.infer():
if value.is_class():
values = value.get_metaclasses()
if values:
return values
return NO_VALUES
jedi/evaluate/context/module.py → jedi/inference/value/module.py
+57 -67
View File
@@ -2,15 +2,16 @@ import re
import os
from jedi import debug
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.names import ContextNameMixin, AbstractNameDefinition
from jedi.evaluate.filters import GlobalNameFilter, ParserTreeFilter, DictFilter, MergedFilter
from jedi.evaluate import compiled
from jedi.evaluate.base_context import TreeContext
from jedi.evaluate.names import SubModuleName
from jedi.evaluate.helpers import contexts_from_qualified_names
from jedi.evaluate.compiled import create_simple_object
from jedi.evaluate.base_context import ContextSet
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.names import ValueNameMixin, AbstractNameDefinition
from jedi.inference.filters import GlobalNameFilter, ParserTreeFilter, DictFilter, MergedFilter
from jedi.inference import compiled
from jedi.inference.base_value import TreeValue
from jedi.inference.names import SubModuleName
from jedi.inference.helpers import values_from_qualified_names
from jedi.inference.compiled import create_simple_object
from jedi.inference.base_value import ValueSet
from jedi.inference.context import ModuleContext
class _ModuleAttributeName(AbstractNameDefinition):
@@ -27,20 +28,20 @@ class _ModuleAttributeName(AbstractNameDefinition):
def infer(self):
if self._string_value is not None:
s = self._string_value
if self.parent_context.evaluator.environment.version_info.major == 2 \
if self.parent_context.inference_state.environment.version_info.major == 2 \
and not isinstance(s, bytes):
s = s.encode('utf-8')
return ContextSet([
create_simple_object(self.parent_context.evaluator, s)
return ValueSet([
create_simple_object(self.parent_context.inference_state, s)
])
return compiled.get_string_context_set(self.parent_context.evaluator)
return compiled.get_string_value_set(self.parent_context.inference_state)
class ModuleName(ContextNameMixin, AbstractNameDefinition):
class ModuleName(ValueNameMixin, AbstractNameDefinition):
start_pos = 1, 0
def __init__(self, context, name):
self._context = context
def __init__(self, value, name):
self._value = value
self._name = name
@property
@@ -48,9 +49,9 @@ class ModuleName(ContextNameMixin, AbstractNameDefinition):
return self._name
def iter_module_names(evaluator, paths):
def iter_module_names(inference_state, paths):
# Python modules/packages
for n in evaluator.compiled_subprocess.list_module_names(paths):
for n in inference_state.compiled_subprocess.list_module_names(paths):
yield n
for path in paths:
@@ -75,19 +76,15 @@ def iter_module_names(evaluator, paths):
class SubModuleDictMixin(object):
@evaluator_method_cache()
@inference_state_method_cache()
def sub_modules_dict(self):
"""
Lists modules in the directory of this module (if this module is a
package).
"""
names = {}
try:
method = self.py__path__
except AttributeError:
pass
else:
mods = iter_module_names(self.evaluator, method())
if self.is_package:
mods = iter_module_names(self.inference_state, self.py__path__())
for name in mods:
# It's obviously a relative import to the current module.
names[name] = SubModuleName(self, name)
@@ -98,12 +95,10 @@ class SubModuleDictMixin(object):
class ModuleMixin(SubModuleDictMixin):
def get_filters(self, search_global=False, until_position=None, origin_scope=None):
def get_filters(self, origin_scope=None):
yield MergedFilter(
ParserTreeFilter(
self.evaluator,
context=self,
until_position=until_position,
parent_context=self.as_context(),
origin_scope=origin_scope
),
GlobalNameFilter(self, self.tree_node),
@@ -114,7 +109,7 @@ class ModuleMixin(SubModuleDictMixin):
yield star_filter
def py__class__(self):
c, = contexts_from_qualified_names(self.evaluator, u'types', u'ModuleType')
c, = values_from_qualified_names(self.inference_state, u'types', u'ModuleType')
return c
def is_module(self):
@@ -124,7 +119,7 @@ class ModuleMixin(SubModuleDictMixin):
return False
@property
@evaluator_method_cache()
@inference_state_method_cache()
def name(self):
return ModuleName(self, self._string_name)
@@ -132,7 +127,7 @@ class ModuleMixin(SubModuleDictMixin):
def _string_name(self):
""" This is used for the goto functions. """
# TODO It's ugly that we even use this, the name is usually well known
# ahead so just pass it when create a ModuleContext.
# ahead so just pass it when create a ModuleValue.
if self._path is None:
return '' # no path -> empty name
else:
@@ -141,7 +136,7 @@ class ModuleMixin(SubModuleDictMixin):
# Remove PEP 3149 names
return re.sub(r'\.[a-z]+-\d{2}[mud]{0,3}$', '', r.group(1))
@evaluator_method_cache()
@inference_state_method_cache()
def _module_attributes_dict(self):
names = ['__package__', '__doc__', '__name__']
# All the additional module attributes are strings.
@@ -151,29 +146,30 @@ class ModuleMixin(SubModuleDictMixin):
dct['__file__'] = _ModuleAttributeName(self, '__file__', file)
return dct
def iter_star_filters(self, search_global=False):
def iter_star_filters(self):
for star_module in self.star_imports():
yield next(star_module.get_filters(search_global))
yield next(star_module.get_filters())
# I'm not sure if the star import cache is really that effective anymore
# with all the other really fast import caches. Recheck. Also we would need
# to push the star imports into Evaluator.module_cache, if we reenable this.
@evaluator_method_cache([])
# to push the star imports into InferenceState.module_cache, if we reenable this.
@inference_state_method_cache([])
def star_imports(self):
from jedi.evaluate.imports import Importer
from jedi.inference.imports import Importer
modules = []
module_context = self.as_context()
for i in self.tree_node.iter_imports():
if i.is_star_import():
new = Importer(
self.evaluator,
self.inference_state,
import_path=i.get_paths()[-1],
module_context=self,
module_context=module_context,
level=i.level
).follow()
for module in new:
if isinstance(module, ModuleContext):
if isinstance(module, ModuleValue):
modules += module.star_imports()
modules += new
return modules
@@ -182,18 +178,18 @@ class ModuleMixin(SubModuleDictMixin):
"""
A module doesn't have a qualified name, but it's important to note that
it's reachable and not `None`. With this information we can add
qualified names on top for all context children.
qualified names on top for all value children.
"""
return ()
class ModuleContext(ModuleMixin, TreeContext):
class ModuleValue(ModuleMixin, TreeValue):
api_type = u'module'
parent_context = None
def __init__(self, evaluator, module_node, file_io, string_names, code_lines, is_package=False):
super(ModuleContext, self).__init__(
evaluator,
def __init__(self, inference_state, module_node, file_io, string_names,
code_lines, is_package=False):
super(ModuleValue, self).__init__(
inference_state,
parent_context=None,
tree_node=module_node
)
@@ -210,9 +206,9 @@ class ModuleContext(ModuleMixin, TreeContext):
if self._path is not None and self._path.endswith('.pyi'):
# Currently this is the way how we identify stubs when e.g. goto is
# used in them. This could be changed if stubs would be identified
# sooner and used as StubModuleContext.
# sooner and used as StubModuleValue.
return True
return super(ModuleContext, self).is_stub()
return super(ModuleValue, self).is_stub()
def py__name__(self):
if self.string_names is None:
@@ -233,7 +229,15 @@ class ModuleContext(ModuleMixin, TreeContext):
return self.string_names
return self.string_names[:-1]
def _py__path__(self):
def py__path__(self):
"""
In case of a package, this returns Python's __path__ attribute, which
is a list of paths (strings).
Returns None if the module is not a package.
"""
if not self.is_package:
return None
# A namespace package is typically auto generated and ~10 lines long.
first_few_lines = ''.join(self.code_lines[:50])
# these are strings that need to be used for namespace packages,
@@ -243,7 +247,7 @@ class ModuleContext(ModuleMixin, TreeContext):
# It is a namespace, now try to find the rest of the
# modules on sys_path or whatever the search_path is.
paths = set()
for s in self.evaluator.get_sys_path():
for s in self.inference_state.get_sys_path():
other = os.path.join(s, self.name.string_name)
if os.path.isdir(other):
paths.add(other)
@@ -258,22 +262,8 @@ class ModuleContext(ModuleMixin, TreeContext):
assert file is not None # Shouldn't be a package in the first place.
return [os.path.dirname(file)]
@property
def py__path__(self):
"""
Not seen here, since it's a property. The callback actually uses a
variable, so use it like::
foo.py__path__(sys_path)
In case of a package, this returns Python's __path__ attribute, which
is a list of paths (strings).
Raises an AttributeError if the module is not a package.
"""
if self.is_package:
return self._py__path__
else:
raise AttributeError('Only packages have __path__ attributes.')
def _as_context(self):
return ModuleContext(self)
def __repr__(self):
return "<%s: %s@%s-%s is_stub=%s>" % (
jedi/evaluate/context/namespace.py → jedi/inference/value/namespace.py
+22 -14
View File
@@ -1,21 +1,22 @@
from jedi.evaluate.cache import evaluator_method_cache
from jedi.evaluate.filters import DictFilter
from jedi.evaluate.names import ContextNameMixin, AbstractNameDefinition
from jedi.evaluate.base_context import Context
from jedi.evaluate.context.module import SubModuleDictMixin
from jedi.inference.cache import inference_state_method_cache
from jedi.inference.filters import DictFilter
from jedi.inference.names import ValueNameMixin, AbstractNameDefinition
from jedi.inference.base_value import Value
from jedi.inference.value.module import SubModuleDictMixin
from jedi.inference.context import NamespaceContext
class ImplicitNSName(ContextNameMixin, AbstractNameDefinition):
class ImplicitNSName(ValueNameMixin, AbstractNameDefinition):
"""
Accessing names for implicit namespace packages should infer to nothing.
This object will prevent Jedi from raising exceptions
"""
def __init__(self, implicit_ns_context, string_name):
self._context = implicit_ns_context
def __init__(self, implicit_ns_value, string_name):
self._value = implicit_ns_value
self.string_name = string_name
class ImplicitNamespaceContext(Context, SubModuleDictMixin):
class ImplicitNamespaceValue(Value, SubModuleDictMixin):
"""
Provides support for implicit namespace packages
"""
@@ -25,17 +26,17 @@ class ImplicitNamespaceContext(Context, SubModuleDictMixin):
api_type = u'module'
parent_context = None
def __init__(self, evaluator, fullname, paths):
super(ImplicitNamespaceContext, self).__init__(evaluator, parent_context=None)
self.evaluator = evaluator
def __init__(self, inference_state, fullname, paths):
super(ImplicitNamespaceValue, self).__init__(inference_state, parent_context=None)
self.inference_state = inference_state
self._fullname = fullname
self._paths = paths
def get_filters(self, search_global=False, until_position=None, origin_scope=None):
def get_filters(self, origin_scope=None):
yield DictFilter(self.sub_modules_dict())
@property
@evaluator_method_cache()
@inference_state_method_cache()
def name(self):
string_name = self.py__package__()[-1]
return ImplicitNSName(self, string_name)
@@ -60,5 +61,12 @@ class ImplicitNamespaceContext(Context, SubModuleDictMixin):
def is_stub(self):
return False
@property
def is_package(self):
return True
def as_context(self):
return NamespaceContext(self)
def __repr__(self):
return '<%s: %s>' % (self.__class__.__name__, self._fullname)
jedi/parser_utils.py
+2 -17
View File
@@ -30,7 +30,7 @@ def get_executable_nodes(node, last_added=False):
if last_added is False and node.parent.type != 'param' and next_leaf != '=':
result.append(node)
elif typ == 'expr_stmt':
# I think evaluating the statement (and possibly returned arrays),
# I think inferring the statement (and possibly returned arrays),
# should be enough for static analysis.
result.append(node)
for child in node.children:
@@ -94,21 +94,6 @@ def get_flow_branch_keyword(flow_node, node):
return 0
def get_statement_of_position(node, pos):
for c in node.children:
if c.start_pos <= pos <= c.end_pos:
if c.type not in ('decorated', 'simple_stmt', 'suite',
'async_stmt', 'async_funcdef') \
and not isinstance(c, (tree.Flow, tree.ClassOrFunc)):
return c
else:
try:
return get_statement_of_position(c, pos)
except AttributeError:
pass # Must be a non-scope
return None
def clean_scope_docstring(scope_node):
""" Returns a cleaned version of the docstring token. """
node = scope_node.get_doc_node()
@@ -294,4 +279,4 @@ def cut_value_at_position(leaf, position):
def get_string_quote(leaf):
return re.match('\w*("""|\'{3}|"|\')', leaf.value).group(1)
return re.match(r'\w*("""|\'{3}|"|\')', leaf.value).group(1)
jedi/plugins/flask.py
+8 -8
View File
@@ -3,19 +3,19 @@ def import_module(callback):
Handle "magic" Flask extension imports:
``flask.ext.foo`` is really ``flask_foo`` or ``flaskext.foo``.
"""
def wrapper(evaluator, import_names, module_context, *args, **kwargs):
def wrapper(inference_state, import_names, module_context, *args, **kwargs):
if len(import_names) == 3 and import_names[:2] == ('flask', 'ext'):
# New style.
ipath = (u'flask_' + import_names[2]),
context_set = callback(evaluator, ipath, None, *args, **kwargs)
if context_set:
return context_set
context_set = callback(evaluator, (u'flaskext',), None, *args, **kwargs)
value_set = callback(inference_state, ipath, None, *args, **kwargs)
if value_set:
return value_set
value_set = callback(inference_state, (u'flaskext',), None, *args, **kwargs)
return callback(
evaluator,
inference_state,
(u'flaskext', import_names[2]),
next(iter(context_set)),
next(iter(value_set)),
*args, **kwargs
)
return callback(evaluator, import_names, module_context, *args, **kwargs)
return callback(inference_state, import_names, module_context, *args, **kwargs)
return wrapper
jedi/plugins/stdlib.py
+185 -186
View File
@@ -14,27 +14,26 @@ import os
from jedi._compatibility import force_unicode, Parameter
from jedi import debug
from jedi.evaluate.utils import safe_property
from jedi.evaluate.helpers import get_str_or_none
from jedi.evaluate.arguments import ValuesArguments, \
from jedi.inference.utils import safe_property
from jedi.inference.helpers import get_str_or_none
from jedi.inference.arguments import ValuesArguments, \
repack_with_argument_clinic, AbstractArguments, TreeArgumentsWrapper
from jedi.evaluate import analysis
from jedi.evaluate import compiled
from jedi.evaluate.context.instance import BoundMethod, InstanceArguments
from jedi.evaluate.base_context import ContextualizedNode, \
NO_CONTEXTS, ContextSet, ContextWrapper, LazyContextWrapper
from jedi.evaluate.context import ClassContext, ModuleContext, \
from jedi.inference import analysis
from jedi.inference import compiled
from jedi.inference.value.instance import BoundMethod, InstanceArguments
from jedi.inference.base_value import ContextualizedNode, \
NO_VALUES, ValueSet, ValueWrapper, LazyValueWrapper
from jedi.inference.value import ClassValue, ModuleValue, \
FunctionExecutionContext
from jedi.evaluate.context.klass import ClassMixin
from jedi.evaluate.context.function import FunctionMixin
from jedi.evaluate.context import iterable
from jedi.evaluate.lazy_context import LazyTreeContext, LazyKnownContext, \
LazyKnownContexts
from jedi.evaluate.names import ContextName, BaseTreeParamName
from jedi.evaluate.syntax_tree import is_string
from jedi.evaluate.filters import AttributeOverwrite, publish_method, \
from jedi.inference.value.klass import ClassMixin
from jedi.inference.value.function import FunctionMixin
from jedi.inference.value import iterable
from jedi.inference.lazy_value import LazyTreeValue, LazyKnownValue, \
LazyKnownValues
from jedi.inference.names import ValueName, BaseTreeParamName
from jedi.inference.filters import AttributeOverwrite, publish_method, \
ParserTreeFilter, DictFilter
from jedi.evaluate.signature import AbstractSignature, SignatureWrapper
from jedi.inference.signature import AbstractSignature, SignatureWrapper
# Copied from Python 3.6's stdlib.
@@ -105,34 +104,35 @@ _NAMEDTUPLE_FIELD_TEMPLATE = '''\
def execute(callback):
def wrapper(context, arguments):
def wrapper(value, arguments):
def call():
return callback(context, arguments=arguments)
return callback(value, arguments=arguments)
try:
obj_name = context.name.string_name
obj_name = value.name.string_name
except AttributeError:
pass
else:
if context.parent_context == context.evaluator.builtins_module:
p = value.parent_context
if p is not None and p.is_builtins_module():
module_name = 'builtins'
elif context.parent_context is not None and context.parent_context.is_module():
module_name = context.parent_context.py__name__()
elif p is not None and p.is_module():
module_name = p.py__name__()
else:
return call()
if isinstance(context, BoundMethod):
if isinstance(value, BoundMethod):
if module_name == 'builtins':
if context.py__name__() == '__get__':
if context.class_context.py__name__() == 'property':
if value.py__name__() == '__get__':
if value.class_context.py__name__() == 'property':
return builtins_property(
context,
value,
arguments=arguments,
callback=call,
)
elif context.py__name__() in ('deleter', 'getter', 'setter'):
if context.class_context.py__name__() == 'property':
return ContextSet([context.instance])
elif value.py__name__() in ('deleter', 'getter', 'setter'):
if value.class_context.py__name__() == 'property':
return ValueSet([value.instance])
return call()
@@ -142,23 +142,23 @@ def execute(callback):
except KeyError:
pass
else:
return func(context, arguments=arguments, callback=call)
return func(value, arguments=arguments, callback=call)
return call()
return wrapper
def _follow_param(evaluator, arguments, index):
def _follow_param(inference_state, arguments, index):
try:
key, lazy_context = list(arguments.unpack())[index]
key, lazy_value = list(arguments.unpack())[index]
except IndexError:
return NO_CONTEXTS
return NO_VALUES
else:
return lazy_context.infer()
return lazy_value.infer()
def argument_clinic(string, want_obj=False, want_context=False,
want_arguments=False, want_evaluator=False,
want_arguments=False, want_inference_state=False,
want_callback=False):
"""
Works like Argument Clinic (PEP 436), to validate function params.
@@ -172,13 +172,13 @@ def argument_clinic(string, want_obj=False, want_context=False,
callback = kwargs.pop('callback')
assert not kwargs # Python 2...
debug.dbg('builtin start %s' % obj, color='MAGENTA')
result = NO_CONTEXTS
result = NO_VALUES
if want_context:
kwargs['context'] = arguments.context
if want_obj:
kwargs['obj'] = obj
if want_evaluator:
kwargs['evaluator'] = obj.evaluator
if want_inference_state:
kwargs['inference_state'] = obj.inference_state
if want_arguments:
kwargs['arguments'] = arguments
if want_callback:
@@ -194,30 +194,30 @@ def argument_clinic(string, want_obj=False, want_context=False,
@argument_clinic('obj, type, /', want_obj=True, want_arguments=True)
def builtins_property(objects, types, obj, arguments):
property_args = obj.instance.var_args.unpack()
key, lazy_context = next(property_args, (None, None))
if key is not None or lazy_context is None:
key, lazy_value = next(property_args, (None, None))
if key is not None or lazy_value is None:
debug.warning('property expected a first param, not %s', arguments)
return NO_CONTEXTS
return NO_VALUES
return lazy_context.infer().py__call__(arguments=ValuesArguments([objects]))
return lazy_value.infer().py__call__(arguments=ValuesArguments([objects]))
@argument_clinic('iterator[, default], /', want_evaluator=True)
def builtins_next(iterators, defaults, evaluator):
if evaluator.environment.version_info.major == 2:
@argument_clinic('iterator[, default], /', want_inference_state=True)
def builtins_next(iterators, defaults, inference_state):
if inference_state.environment.version_info.major == 2:
name = 'next'
else:
name = '__next__'
# TODO theoretically we have to check here if something is an iterator.
# That is probably done by checking if it's not a class.
return defaults | iterators.py__getattribute__(name).execute_evaluated()
return defaults | iterators.py__getattribute__(name).execute_with_values()
@argument_clinic('iterator[, default], /')
def builtins_iter(iterators_or_callables, defaults):
# TODO implement this if it's a callable.
return iterators_or_callables.py__getattribute__('__iter__').execute_evaluated()
return iterators_or_callables.py__getattribute__('__iter__').execute_with_values()
@argument_clinic('object, name[, default], /')
@@ -231,38 +231,38 @@ def builtins_getattr(objects, names, defaults=None):
continue
else:
return obj.py__getattribute__(force_unicode(string))
return NO_CONTEXTS
return NO_VALUES
@argument_clinic('object[, bases, dict], /')
def builtins_type(objects, bases, dicts):
if bases or dicts:
# It's a type creation... maybe someday...
return NO_CONTEXTS
return NO_VALUES
else:
return objects.py__class__()
class SuperInstance(LazyContextWrapper):
class SuperInstance(LazyValueWrapper):
"""To be used like the object ``super`` returns."""
def __init__(self, evaluator, instance):
self.evaluator = evaluator
def __init__(self, inference_state, instance):
self.inference_state = inference_state
self._instance = instance # Corresponds to super().__self__
def _get_bases(self):
return self._instance.py__class__().py__bases__()
def _get_wrapped_context(self):
objs = self._get_bases()[0].infer().execute_evaluated()
def _get_wrapped_value(self):
objs = self._get_bases()[0].infer().execute_with_values()
if not objs:
# This is just a fallback and will only be used, if it's not
# possible to find a class
return self._instance
return next(iter(objs))
def get_filters(self, search_global=False, until_position=None, origin_scope=None):
def get_filters(self, origin_scope=None):
for b in self._get_bases():
for obj in b.infer().execute_evaluated():
for obj in b.infer().execute_with_values():
for f in obj.get_filters():
yield f
@@ -274,9 +274,9 @@ def builtins_super(types, objects, context):
instance = context.var_args.instance
# TODO if a class is given it doesn't have to be the direct super
# class, it can be an anecestor from long ago.
return ContextSet({SuperInstance(instance.evaluator, instance)})
return ValueSet({SuperInstance(instance.inference_state, instance)})
return NO_CONTEXTS
return NO_VALUES
class ReversedObject(AttributeOverwrite):
@@ -291,8 +291,8 @@ class ReversedObject(AttributeOverwrite):
@publish_method('next', python_version_match=2)
@publish_method('__next__', python_version_match=3)
def py__next__(self):
return ContextSet.from_sets(
lazy_context.infer() for lazy_context in self._iter_list
return ValueSet.from_sets(
lazy_value.infer() for lazy_value in self._iter_list
)
@@ -301,23 +301,22 @@ def builtins_reversed(sequences, obj, arguments):
# While we could do without this variable (just by using sequences), we
# want static analysis to work well. Therefore we need to generated the
# values again.
key, lazy_context = next(arguments.unpack())
key, lazy_value = next(arguments.unpack())
cn = None
if isinstance(lazy_context, LazyTreeContext):
# TODO access private
cn = ContextualizedNode(lazy_context.context, lazy_context.data)
if isinstance(lazy_value, LazyTreeValue):
cn = ContextualizedNode(lazy_value.context, lazy_value.data)
ordered = list(sequences.iterate(cn))
# Repack iterator values and then run it the normal way. This is
# necessary, because `reversed` is a function and autocompletion
# would fail in certain cases like `reversed(x).__iter__` if we
# just returned the result directly.
seq, = obj.evaluator.typing_module.py__getattribute__('Iterator').execute_evaluated()
return ContextSet([ReversedObject(seq, list(reversed(ordered)))])
seq, = obj.inference_state.typing_module.py__getattribute__('Iterator').execute_with_values()
return ValueSet([ReversedObject(seq, list(reversed(ordered)))])
@argument_clinic('obj, type, /', want_arguments=True, want_evaluator=True)
def builtins_isinstance(objects, types, arguments, evaluator):
@argument_clinic('obj, type, /', want_arguments=True, want_inference_state=True)
def builtins_isinstance(objects, types, arguments, inference_state):
bool_results = set()
for o in objects:
cls = o.py__class__()
@@ -336,57 +335,57 @@ def builtins_isinstance(objects, types, arguments, evaluator):
if cls_or_tup.is_class():
bool_results.add(cls_or_tup in mro)
elif cls_or_tup.name.string_name == 'tuple' \
and cls_or_tup.get_root_context() == evaluator.builtins_module:
and cls_or_tup.get_root_context().is_builtins_module():
# Check for tuples.
classes = ContextSet.from_sets(
lazy_context.infer()
for lazy_context in cls_or_tup.iterate()
classes = ValueSet.from_sets(
lazy_value.infer()
for lazy_value in cls_or_tup.iterate()
)
bool_results.add(any(cls in mro for cls in classes))
else:
_, lazy_context = list(arguments.unpack())[1]
if isinstance(lazy_context, LazyTreeContext):
node = lazy_context.data
_, lazy_value = list(arguments.unpack())[1]
if isinstance(lazy_value, LazyTreeValue):
node = lazy_value.data
message = 'TypeError: isinstance() arg 2 must be a ' \
'class, type, or tuple of classes and types, ' \
'not %s.' % cls_or_tup
analysis.add(lazy_context.context, 'type-error-isinstance', node, message)
analysis.add(lazy_value.context, 'type-error-isinstance', node, message)
return ContextSet(
compiled.builtin_from_name(evaluator, force_unicode(str(b)))
return ValueSet(
compiled.builtin_from_name(inference_state, force_unicode(str(b)))
for b in bool_results
)
class StaticMethodObject(AttributeOverwrite, ContextWrapper):
class StaticMethodObject(AttributeOverwrite, ValueWrapper):
def get_object(self):
return self._wrapped_context
return self._wrapped_value
def py__get__(self, instance, klass):
return ContextSet([self._wrapped_context])
return ValueSet([self._wrapped_value])
@argument_clinic('sequence, /')
def builtins_staticmethod(functions):
return ContextSet(StaticMethodObject(f) for f in functions)
return ValueSet(StaticMethodObject(f) for f in functions)
class ClassMethodObject(AttributeOverwrite, ContextWrapper):
class ClassMethodObject(AttributeOverwrite, ValueWrapper):
def __init__(self, class_method_obj, function):
super(ClassMethodObject, self).__init__(class_method_obj)
self._function = function
def get_object(self):
return self._wrapped_context
return self._wrapped_value
def py__get__(self, obj, class_context):
return ContextSet([
ClassMethodGet(__get__, class_context, self._function)
for __get__ in self._wrapped_context.py__getattribute__('__get__')
def py__get__(self, obj, class_value):
return ValueSet([
ClassMethodGet(__get__, class_value, self._function)
for __get__ in self._wrapped_value.py__getattribute__('__get__')
])
class ClassMethodGet(AttributeOverwrite, ContextWrapper):
class ClassMethodGet(AttributeOverwrite, ValueWrapper):
def __init__(self, get_method, klass, function):
super(ClassMethodGet, self).__init__(get_method)
self._class = klass
@@ -396,7 +395,7 @@ class ClassMethodGet(AttributeOverwrite, ContextWrapper):
return self._function.get_signatures()
def get_object(self):
return self._wrapped_context
return self._wrapped_value
def py__call__(self, arguments):
return self._function.execute(ClassMethodArguments(self._class, arguments))
@@ -408,14 +407,14 @@ class ClassMethodArguments(TreeArgumentsWrapper):
self._class = klass
def unpack(self, func=None):
yield None, LazyKnownContext(self._class)
yield None, LazyKnownValue(self._class)
for values in self._wrapped_arguments.unpack(func):
yield values
@argument_clinic('sequence, /', want_obj=True, want_arguments=True)
def builtins_classmethod(functions, obj, arguments):
return ContextSet(
return ValueSet(
ClassMethodObject(class_method_object, function)
for class_method_object in obj.py__call__(arguments=arguments)
for function in functions
@@ -427,36 +426,36 @@ def collections_namedtuple(obj, arguments, callback):
Implementation of the namedtuple function.
This has to be done by processing the namedtuple class template and
evaluating the result.
inferring the result.
"""
evaluator = obj.evaluator
inference_state = obj.inference_state
# Process arguments
name = u'jedi_unknown_namedtuple'
for c in _follow_param(evaluator, arguments, 0):
for c in _follow_param(inference_state, arguments, 0):
x = get_str_or_none(c)
if x is not None:
name = force_unicode(x)
break
# TODO here we only use one of the types, we should use all.
param_contexts = _follow_param(evaluator, arguments, 1)
if not param_contexts:
return NO_CONTEXTS
_fields = list(param_contexts)[0]
param_values = _follow_param(inference_state, arguments, 1)
if not param_values:
return NO_VALUES
_fields = list(param_values)[0]
string = get_str_or_none(_fields)
if string is not None:
fields = force_unicode(string).replace(',', ' ').split()
elif isinstance(_fields, iterable.Sequence):
fields = [
force_unicode(get_str_or_none(v))
for lazy_context in _fields.py__iter__()
for v in lazy_context.infer()
for lazy_value in _fields.py__iter__()
for v in lazy_value.infer()
]
fields = [f for f in fields if f is not None]
else:
return NO_CONTEXTS
return NO_VALUES
# Build source code
code = _NAMEDTUPLE_CLASS_TEMPLATE.format(
@@ -470,32 +469,32 @@ def collections_namedtuple(obj, arguments, callback):
)
# Parse source code
module = evaluator.grammar.parse(code)
module = inference_state.grammar.parse(code)
generated_class = next(module.iter_classdefs())
parent_context = ModuleContext(
evaluator, module,
parent_context = ModuleValue(
inference_state, module,
file_io=None,
string_names=None,
code_lines=parso.split_lines(code, keepends=True),
)
).as_context()
return ContextSet([ClassContext(evaluator, parent_context, generated_class)])
return ValueSet([ClassValue(inference_state, parent_context, generated_class)])
class PartialObject(object):
def __init__(self, actual_context, arguments):
self._actual_context = actual_context
def __init__(self, actual_value, arguments):
self._actual_value = actual_value
self._arguments = arguments
def __getattr__(self, name):
return getattr(self._actual_context, name)
return getattr(self._actual_value, name)
def _get_function(self, unpacked_arguments):
key, lazy_context = next(unpacked_arguments, (None, None))
if key is not None or lazy_context is None:
key, lazy_value = next(unpacked_arguments, (None, None))
if key is not None or lazy_value is None:
debug.warning("Partial should have a proper function %s", self._arguments)
return None
return lazy_context.infer()
return lazy_value.infer()
def get_signatures(self):
unpacked_arguments = self._arguments.unpack()
@@ -515,7 +514,7 @@ class PartialObject(object):
def py__call__(self, arguments):
func = self._get_function(self._arguments.unpack())
if func is None:
return NO_CONTEXTS
return NO_VALUES
return func.execute(
MergedPartialArguments(self._arguments, arguments)
@@ -543,14 +542,14 @@ class MergedPartialArguments(AbstractArguments):
# Ignore this one, it's the function. It was checked before that it's
# there.
next(unpacked)
for key_lazy_context in unpacked:
yield key_lazy_context
for key_lazy_context in self._call_arguments.unpack(funcdef):
yield key_lazy_context
for key_lazy_value in unpacked:
yield key_lazy_value
for key_lazy_value in self._call_arguments.unpack(funcdef):
yield key_lazy_value
def functools_partial(obj, arguments, callback):
return ContextSet(
return ValueSet(
PartialObject(instance, arguments)
for instance in obj.py__call__(arguments)
)
@@ -563,28 +562,28 @@ def _return_first_param(firsts):
@argument_clinic('seq')
def _random_choice(sequences):
return ContextSet.from_sets(
lazy_context.infer()
return ValueSet.from_sets(
lazy_value.infer()
for sequence in sequences
for lazy_context in sequence.py__iter__()
for lazy_value in sequence.py__iter__()
)
def _dataclass(obj, arguments, callback):
for c in _follow_param(obj.evaluator, arguments, 0):
for c in _follow_param(obj.inference_state, arguments, 0):
if c.is_class():
return ContextSet([DataclassWrapper(c)])
return ValueSet([DataclassWrapper(c)])
else:
return ContextSet([obj])
return NO_CONTEXTS
return ValueSet([obj])
return NO_VALUES
class DataclassWrapper(ContextWrapper, ClassMixin):
class DataclassWrapper(ValueWrapper, ClassMixin):
def get_signatures(self):
param_names = []
for cls in reversed(list(self.py__mro__())):
if isinstance(cls, DataclassWrapper):
filter_ = cls.get_global_filter()
filter_ = cls.as_context().get_global_filter()
# .values ordering is not guaranteed, at least not in
# Python < 3.6, when dicts where not ordered, which is an
# implementation detail anyway.
@@ -606,8 +605,8 @@ class DataclassWrapper(ContextWrapper, ClassMixin):
class DataclassSignature(AbstractSignature):
def __init__(self, context, param_names):
super(DataclassSignature, self).__init__(context)
def __init__(self, value, param_names):
super(DataclassSignature, self).__init__(value)
self._param_names = param_names
def get_param_names(self, resolve_stars=False):
@@ -625,51 +624,51 @@ class DataclassParamName(BaseTreeParamName):
def infer(self):
if self.annotation_node is None:
return NO_CONTEXTS
return NO_VALUES
else:
return self.parent_context.eval_node(self.annotation_node)
return self.parent_context.infer_node(self.annotation_node)
class ItemGetterCallable(ContextWrapper):
def __init__(self, instance, args_context_set):
class ItemGetterCallable(ValueWrapper):
def __init__(self, instance, args_value_set):
super(ItemGetterCallable, self).__init__(instance)
self._args_context_set = args_context_set
self._args_value_set = args_value_set
@repack_with_argument_clinic('item, /')
def py__call__(self, item_context_set):
context_set = NO_CONTEXTS
for args_context in self._args_context_set:
lazy_contexts = list(args_context.py__iter__())
if len(lazy_contexts) == 1:
# TODO we need to add the contextualized context.
context_set |= item_context_set.get_item(lazy_contexts[0].infer(), None)
def py__call__(self, item_value_set):
value_set = NO_VALUES
for args_value in self._args_value_set:
lazy_values = list(args_value.py__iter__())
if len(lazy_values) == 1:
# TODO we need to add the contextualized value.
value_set |= item_value_set.get_item(lazy_values[0].infer(), None)
else:
context_set |= ContextSet([iterable.FakeSequence(
self._wrapped_context.evaluator,
value_set |= ValueSet([iterable.FakeSequence(
self._wrapped_value.inference_state,
'list',
[
LazyKnownContexts(item_context_set.get_item(lazy_context.infer(), None))
for lazy_context in lazy_contexts
LazyKnownValues(item_value_set.get_item(lazy_value.infer(), None))
for lazy_value in lazy_values
],
)])
return context_set
return value_set
@argument_clinic('func, /')
def _functools_wraps(funcs):
return ContextSet(WrapsCallable(func) for func in funcs)
return ValueSet(WrapsCallable(func) for func in funcs)
class WrapsCallable(ContextWrapper):
# XXX this is not the correct wrapped context, it should be a weird
class WrapsCallable(ValueWrapper):
# XXX this is not the correct wrapped value, it should be a weird
# partials object, but it doesn't matter, because it's always used as a
# decorator anyway.
@repack_with_argument_clinic('func, /')
def py__call__(self, funcs):
return ContextSet({Wrapped(func, self._wrapped_context) for func in funcs})
return ValueSet({Wrapped(func, self._wrapped_value) for func in funcs})
class Wrapped(ContextWrapper, FunctionMixin):
class Wrapped(ValueWrapper, FunctionMixin):
def __init__(self, func, original_function):
super(Wrapped, self).__init__(func)
self._original_function = original_function
@@ -683,9 +682,9 @@ class Wrapped(ContextWrapper, FunctionMixin):
@argument_clinic('*args, /', want_obj=True, want_arguments=True)
def _operator_itemgetter(args_context_set, obj, arguments):
return ContextSet([
ItemGetterCallable(instance, args_context_set)
def _operator_itemgetter(args_value_set, obj, arguments):
return ValueSet([
ItemGetterCallable(instance, args_value_set)
for instance in obj.py__call__(arguments)
])
@@ -694,14 +693,14 @@ def _create_string_input_function(func):
@argument_clinic('string, /', want_obj=True, want_arguments=True)
def wrapper(strings, obj, arguments):
def iterate():
for context in strings:
s = get_str_or_none(context)
for value in strings:
s = get_str_or_none(value)
if s is not None:
s = func(s)
yield compiled.create_simple_object(context.evaluator, s)
contexts = ContextSet(iterate())
if contexts:
return contexts
yield compiled.create_simple_object(value.inference_state, s)
values = ValueSet(iterate())
if values:
return values
return obj.py__call__(arguments)
return wrapper
@@ -712,11 +711,11 @@ def _os_path_join(args_set, callback):
string = u''
sequence, = args_set
is_first = True
for lazy_context in sequence.py__iter__():
string_contexts = lazy_context.infer()
if len(string_contexts) != 1:
for lazy_value in sequence.py__iter__():
string_values = lazy_value.infer()
if len(string_values) != 1:
break
s = get_str_or_none(next(iter(string_contexts)))
s = get_str_or_none(next(iter(string_values)))
if s is None:
break
if not is_first:
@@ -724,7 +723,7 @@ def _os_path_join(args_set, callback):
string += force_unicode(s)
is_first = False
else:
return ContextSet([compiled.create_simple_object(sequence.evaluator, string)])
return ValueSet([compiled.create_simple_object(sequence.inference_state, string)])
return callback()
@@ -745,8 +744,8 @@ _implemented = {
'deepcopy': _return_first_param,
},
'json': {
'load': lambda obj, arguments, callback: NO_CONTEXTS,
'loads': lambda obj, arguments, callback: NO_CONTEXTS,
'load': lambda obj, arguments, callback: NO_VALUES,
'loads': lambda obj, arguments, callback: NO_VALUES,
},
'collections': {
'namedtuple': collections_namedtuple,
@@ -773,7 +772,7 @@ _implemented = {
# The _alias function just leads to some annoying type inference.
# Therefore, just make it return nothing, which leads to the stubs
# being used instead. This only matters for 3.7+.
'_alias': lambda obj, arguments, callback: NO_CONTEXTS,
'_alias': lambda obj, arguments, callback: NO_VALUES,
},
'dataclasses': {
# For now this works at least better than Jedi trying to understand it.
@@ -793,7 +792,7 @@ def get_metaclass_filters(func):
for metaclass in metaclasses:
if metaclass.py__name__() == 'EnumMeta' \
and metaclass.get_root_context().py__name__() == 'enum':
filter_ = ParserTreeFilter(cls.evaluator, context=cls)
filter_ = ParserTreeFilter(parent_context=cls.as_context())
return [DictFilter({
name.string_name: EnumInstance(cls, name).name for name in filter_.values()
})]
@@ -801,35 +800,35 @@ def get_metaclass_filters(func):
return wrapper
class EnumInstance(LazyContextWrapper):
class EnumInstance(LazyValueWrapper):
def __init__(self, cls, name):
self.evaluator = cls.evaluator
self.inference_state = cls.inference_state
self._cls = cls # Corresponds to super().__self__
self._name = name
self.tree_node = self._name.tree_name
@safe_property
def name(self):
return ContextName(self, self._name.tree_name)
return ValueName(self, self._name.tree_name)
def _get_wrapped_context(self):
obj, = self._cls.execute_evaluated()
def _get_wrapped_value(self):
obj, = self._cls.execute_with_values()
return obj
def get_filters(self, search_global=False, position=None, origin_scope=None):
def get_filters(self, origin_scope=None):
yield DictFilter(dict(
name=compiled.create_simple_object(self.evaluator, self._name.string_name).name,
name=compiled.create_simple_object(self.inference_state, self._name.string_name).name,
value=self._name,
))
for f in self._get_wrapped_context().get_filters():
for f in self._get_wrapped_value().get_filters():
yield f
def tree_name_to_contexts(func):
def wrapper(evaluator, context, tree_name):
if tree_name.value == 'sep' and context.is_module() and context.py__name__() == 'os.path':
return ContextSet({
compiled.create_simple_object(evaluator, os.path.sep),
def tree_name_to_values(func):
def wrapper(inference_state, value, tree_name):
if tree_name.value == 'sep' and value.is_module() and value.py__name__() == 'os.path':
return ValueSet({
compiled.create_simple_object(inference_state, os.path.sep),
})
return func(evaluator, context, tree_name)
return func(inference_state, value, tree_name)
return wrapper
jedi/refactoring.py
+1 -1
View File
@@ -18,7 +18,7 @@ following functions (sometimes bug-prone):
import difflib
from parso import python_bytes_to_unicode, split_lines
from jedi.evaluate import helpers
from jedi.inference import helpers
class Refactoring(object):
test/completion/completion.py
+1 -1
View File
@@ -1,6 +1,6 @@
"""
Special cases of completions (typically special positions that caused issues
with context parsing.
with value parsing.
"""
def pass_decorator(func):
test/completion/generators.py
+1 -1
View File
@@ -24,7 +24,7 @@ next(gen_ret(1))
#? []
next(gen_ret()).
# generators evaluate to true if cast by bool.
# generators infer to true if cast by bool.
a = ''
if gen_ret():
a = 3
test/completion/parser.py
+1 -1
View File
@@ -36,7 +36,7 @@ definition = 0
str(def
# It might be hard to determine the context
# It might be hard to determine the value
class Foo(object):
@property
#? ['str']
test/completion/pep0484_basic.py
+1 -1
View File
@@ -24,7 +24,7 @@ def function_parameters(a: A, b, c: str, d: int, e: str, f: str, g: int=4):
d
#? str()
e
#? int() str()
#? str()
f
# int()
g
test/completion/sys_path.py
+4 -4
View File
@@ -10,11 +10,11 @@ sys.path.append(dirname(os.path.abspath('thirdparty' + os.path.sep + 'asdf')))
# syntax err
sys.path.append('a' +* '/thirdparty')
#? ['evaluate']
import evaluate
#? ['inference']
import inference
#? ['evaluator_function_cache']
evaluate.Evaluator_fu
#? ['inference_state_function_cache']
inference.inference_state_fu
# Those don't work because dirname and abspath are not properly understood.
##? ['jedi_']
test/completion/thirdparty/jedi_.py
Vendored
+11 -11
View File
@@ -1,5 +1,5 @@
from jedi import functions, evaluate, parsing
from jedi import functions, inference, parsing
el = functions.completions()[0]
#? ['description']
@@ -18,17 +18,17 @@ el = scopes
# get_names_for_scope is also recursion stuff
#? tuple()
el = list(evaluate.get_names_for_scope())[0]
el = list(inference.get_names_for_scope())[0]
#? int() parsing.Module()
el = list(evaluate.get_names_for_scope(1))[0][0]
el = list(inference.get_names_for_scope(1))[0][0]
#? parsing.Module()
el = list(evaluate.get_names_for_scope())[0][0]
el = list(inference.get_names_for_scope())[0][0]
#? list()
el = list(evaluate.get_names_for_scope(1))[0][1]
el = list(inference.get_names_for_scope(1))[0][1]
#? list()
el = list(evaluate.get_names_for_scope())[0][1]
el = list(inference.get_names_for_scope())[0][1]
#? list()
parsing.Scope((0,0)).get_set_vars()
@@ -39,14 +39,14 @@ parsing.Scope((0,0)).get_set_vars()[0]
parsing.Scope((0,0)).get_set_vars()[0].parent
#? parsing.Import() parsing.Name()
el = list(evaluate.get_names_for_scope())[0][1][0]
el = list(inference.get_names_for_scope())[0][1][0]
#? evaluate.Array() evaluate.Class() evaluate.Function() evaluate.Instance()
list(evaluate.follow_call())[0]
#? inference.Array() inference.Class() inference.Function() inference.Instance()
list(inference.follow_call())[0]
# With the right recursion settings, this should be possible (and maybe more):
# Array Class Function Generator Instance Module
# However, this was produced with the recursion settings 10/350/10000, and
# lasted 18.5 seconds. So we just have to be content with the results.
#? evaluate.Class() evaluate.Function()
evaluate.get_scopes_for_name()[0]
#? inference.Class() inference.Function()
inference.get_scopes_for_name()[0]
test/completion/usages.py
+5 -5
View File
@@ -83,18 +83,18 @@ import module_not_exists
module_not_exists
#< ('rename1', 1,0), (0,24), (3,0), (6,17), ('rename2', 4,17), (11,17), (14,17), ('imports', 72, 16)
#< ('import_tree.rename1', 1,0), (0,24), (3,0), (6,17), ('import_tree.rename2', 4,17), (11,17), (14,17), ('imports', 72, 16)
from import_tree import rename1
#< (0,8), ('rename1',3,0), ('rename2',4,32), ('rename2',6,0), (3,32), (8,32), (5,0)
#< (0,8), ('import_tree.rename1',3,0), ('import_tree.rename2',4,32), ('import_tree.rename2',6,0), (3,32), (8,32), (5,0)
rename1.abc
#< (-3,8), ('rename1', 3,0), ('rename2', 4,32), ('rename2', 6,0), (0,32), (5,32), (2,0)
#< (-3,8), ('import_tree.rename1', 3,0), ('import_tree.rename2', 4,32), ('import_tree.rename2', 6,0), (0,32), (5,32), (2,0)
from import_tree.rename1 import abc
#< (-5,8), (-2,32), ('rename1', 3,0), ('rename2', 4,32), ('rename2', 6,0), (0,0), (3,32)
#< (-5,8), (-2,32), ('import_tree.rename1', 3,0), ('import_tree.rename2', 4,32), ('import_tree.rename2', 6,0), (0,0), (3,32)
abc
#< 20 ('rename1', 1,0), ('rename2', 4,17), (-11,24), (-8,0), (-5,17), (0,17), (3,17), ('imports', 72, 16)
#< 20 ('import_tree.rename1', 1,0), ('import_tree.rename2', 4,17), (-11,24), (-8,0), (-5,17), (0,17), (3,17), ('imports', 72, 16)
from import_tree.rename1 import abc
#< (0, 32),
test/conftest.py
+5 -5
View File
@@ -10,7 +10,7 @@ from . import refactor
import jedi
from jedi.api.environment import InterpreterEnvironment
from jedi.evaluate.analysis import Warning
from jedi.inference.analysis import Warning
def pytest_addoption(parser):
@@ -162,10 +162,10 @@ def cwd_tmpdir(monkeypatch, tmpdir):
@pytest.fixture
def evaluator(Script):
return Script('')._evaluator
def inference_state(Script):
return Script('')._inference_state
@pytest.fixture
def same_process_evaluator(Script):
return Script('', environment=InterpreterEnvironment())._evaluator
def same_process_inference_state(Script):
return Script('', environment=InterpreterEnvironment())._inference_state
test/run.py
+16 -15
View File
@@ -123,10 +123,10 @@ import jedi
from jedi import debug
from jedi._compatibility import unicode, is_py3
from jedi.api.classes import Definition
from jedi.api.completion import get_user_scope
from jedi.api.completion import get_user_context
from jedi import parser_utils
from jedi.api.environment import get_default_environment, get_system_environment
from jedi.evaluate.gradual.conversion import convert_contexts
from jedi.inference.gradual.conversion import convert_values
TEST_COMPLETIONS = 0
@@ -212,7 +212,7 @@ class IntegrationTestCase(object):
def run_goto_definitions(self, compare_cb, environment):
script = self.script(environment)
evaluator = script._evaluator
inference_state = script._inference_state
def comparison(definition):
suffix = '()' if definition.type == 'instance' else ''
@@ -224,21 +224,19 @@ class IntegrationTestCase(object):
string = match.group(0)
parser = grammar36.parse(string, start_symbol='eval_input', error_recovery=False)
parser_utils.move(parser.get_root_node(), self.line_nr)
element = parser.get_root_node()
module_context = script._get_module()
# The context shouldn't matter for the test results.
user_context = get_user_scope(module_context, (self.line_nr, 0))
if user_context.api_type == 'function':
user_context = user_context.get_function_execution()
element.parent = user_context.tree_node
results = convert_contexts(
evaluator.eval_element(user_context, element),
)
node = parser.get_root_node()
module_context = script._get_module_context()
user_context = get_user_context(module_context, (self.line_nr, 0))
# TODO needed?
#if user_context._value.api_type == 'function':
# user_context = user_context.get_function_execution()
node.parent = user_context.tree_node
results = convert_values(user_context.infer_node(node))
if not results:
raise Exception('Could not resolve %s on line %s'
% (match.string, self.line_nr - 1))
should_be |= set(Definition(evaluator, r.name) for r in results)
should_be |= set(Definition(inference_state, r.name) for r in results)
debug.dbg('Finished getting types', color='YELLOW')
# Because the objects have different ids, `repr`, then compare.
@@ -258,7 +256,10 @@ class IntegrationTestCase(object):
def run_usages(self, compare_cb, environment):
result = self.script(environment).usages()
self.correct = self.correct.strip()
compare = sorted((r.module_name, r.line, r.column) for r in result)
compare = sorted(
(re.sub(r'^test\.completion\.', '', r.module_name), r.line, r.column)
for r in result
)
wanted = []
if not self.correct:
positions = []
test/test_api/test_api.py
+4 -3
View File
@@ -9,7 +9,7 @@ from pytest import raises
from parso import cache
from jedi import preload_module
from jedi.evaluate.gradual import typeshed
from jedi.inference.gradual import typeshed
def test_preload_modules():
@@ -219,15 +219,16 @@ def test_goto_assignments_follow_imports(Script):
def test_goto_module(Script):
def check(line, expected):
def check(line, expected, follow_imports=False):
script = Script(path=path, line=line)
module, = script.goto_assignments()
module, = script.goto_assignments(follow_imports=follow_imports)
assert module.module_path == expected
base_path = os.path.join(os.path.dirname(__file__), 'simple_import')
path = os.path.join(base_path, '__init__.py')
check(1, os.path.join(base_path, 'module.py'))
check(1, os.path.join(base_path, 'module.py'), follow_imports=True)
check(5, os.path.join(base_path, 'module2.py'))
test/test_api/test_api_classes_follow_definition.py
+1 -1
View File
@@ -21,7 +21,7 @@ def check_follow_definition_types(Script, source):
def test_follow_import_incomplete(Script, environment):
"""
Completion on incomplete imports should always take the full completion
to do any evaluation.
to do any type inference.
"""
datetime = check_follow_definition_types(Script, "import itertool")
assert datetime == ['module']
test/test_api/test_classes.py
+19 -5
View File
@@ -8,7 +8,7 @@ import pytest
import jedi
from jedi import __doc__ as jedi_doc
from jedi.evaluate.compiled import CompiledContextName
from jedi.inference.compiled import CompiledValueName
def test_is_keyword(Script):
@@ -287,6 +287,20 @@ def test_parent_on_completion(Script):
assert parent.type == 'class'
def test_parent_on_comprehension():
ns = jedi.names('''\
def spam():
return [i for i in range(5)]
''', all_scopes=True)
assert [name.name for name in ns] == ['spam', 'i']
assert ns[0].parent().name == ''
assert ns[0].parent().type == 'module'
assert ns[1].parent().name == 'spam'
assert ns[1].parent().type == 'function'
def test_type(Script):
for c in Script('a = [str()]; a[0].').completions():
if c.name == '__class__' and False: # TODO fix.
@@ -386,7 +400,7 @@ def test_import(names):
n = nms[1].goto_assignments()[0]
# This is very special, normally the name doesn't chance, but since
# os.path is a sys.modules hack, it does.
assert n.name in ('ntpath', 'posixpath', 'os2emxpath')
assert n.name in ('macpath', 'ntpath', 'posixpath', 'os2emxpath')
assert n.type == 'module'
@@ -398,7 +412,7 @@ def test_import_alias(names):
n = nms[0].goto_assignments()[0]
assert n.name == 'json'
assert n.type == 'module'
assert n._name._context.tree_node.type == 'file_input'
assert n._name._value.tree_node.type == 'file_input'
assert nms[1].name == 'foo'
assert nms[1].type == 'module'
@@ -407,7 +421,7 @@ def test_import_alias(names):
assert len(ass) == 1
assert ass[0].name == 'json'
assert ass[0].type == 'module'
assert ass[0]._name._context.tree_node.type == 'file_input'
assert ass[0]._name._value.tree_node.type == 'file_input'
def test_added_equals_to_params(Script):
@@ -436,7 +450,7 @@ def test_builtin_module_with_path(Script):
confusing.
"""
semlock, = Script('from _multiprocessing import SemLock').goto_definitions()
assert isinstance(semlock._name, CompiledContextName)
assert isinstance(semlock._name, CompiledValueName)
assert semlock.module_path is None
assert semlock.in_builtin_module() is True
assert semlock.name == 'SemLock'
test/test_api/test_completion.py
+6 -6
View File
@@ -34,7 +34,7 @@ def test_in_empty_space(Script):
assert def_.name == 'X'
def test_indent_context(Script):
def test_indent_value(Script):
"""
If an INDENT is the next supposed token, we should still be able to
complete.
@@ -44,7 +44,7 @@ def test_indent_context(Script):
assert comp.name == 'isinstance'
def test_keyword_context(Script):
def test_keyword_value(Script):
def get_names(*args, **kwargs):
return [d.name for d in Script(*args, **kwargs).completions()]
@@ -89,7 +89,7 @@ def test_fake_subnodes(Script):
Test the number of subnodes of a fake object.
There was a bug where the number of child nodes would grow on every
call to :func:``jedi.evaluate.compiled.fake.get_faked``.
call to :func:``jedi.inference.compiled.fake.get_faked``.
See Github PR#649 and isseu #591.
"""
@@ -101,8 +101,8 @@ def test_fake_subnodes(Script):
for i in range(2):
completions = Script('').completions()
c = get_str_completion(completions)
str_context, = c._name.infer()
n = len(str_context.tree_node.children[-1].children)
str_value, = c._name.infer()
n = len(str_value.tree_node.children[-1].children)
if i == 0:
limit = n
else:
@@ -223,7 +223,7 @@ se = s * 2 if s == '\\' else s
(f2, os_path + 'join(dirname(__file__), "completion", "basi)', 33, ['on"']),
(f2, os_path + 'join(dirname(__file__), "completion", "basi")', 33, ['on"']),
# join with one argument. join will not get evaluated and the result is
# join with one argument. join will not get inferred and the result is
# that directories and in a slash. This is unfortunate, but doesn't
# really matter.
(f2, os_path + 'join("tes', 9, ['t"']),
test/test_api/test_environment.py
+10 -10
View File
@@ -42,10 +42,10 @@ def test_versions(version):
assert env.get_sys_path()
def test_load_module(evaluator):
access_path = evaluator.compiled_subprocess.load_module(
def test_load_module(inference_state):
access_path = inference_state.compiled_subprocess.load_module(
dotted_name=u'math',
sys_path=evaluator.get_sys_path()
sys_path=inference_state.get_sys_path()
)
name, access_handle = access_path.accesses[0]
@@ -55,31 +55,31 @@ def test_load_module(evaluator):
access_handle.py__mro__()
def test_error_in_environment(evaluator, Script, environment):
def test_error_in_environment(inference_state, Script, environment):
if isinstance(environment, InterpreterEnvironment):
pytest.skip("We don't catch these errors at the moment.")
# Provoke an error to show how Jedi can recover from it.
with pytest.raises(jedi.InternalError):
evaluator.compiled_subprocess._test_raise_error(KeyboardInterrupt)
inference_state.compiled_subprocess._test_raise_error(KeyboardInterrupt)
# The second time it should raise an InternalError again.
with pytest.raises(jedi.InternalError):
evaluator.compiled_subprocess._test_raise_error(KeyboardInterrupt)
inference_state.compiled_subprocess._test_raise_error(KeyboardInterrupt)
# Jedi should still work.
def_, = Script('str').goto_definitions()
assert def_.name == 'str'
def test_stdout_in_subprocess(evaluator, Script):
evaluator.compiled_subprocess._test_print(stdout='.')
def test_stdout_in_subprocess(inference_state, Script):
inference_state.compiled_subprocess._test_print(stdout='.')
Script('1').goto_definitions()
def test_killed_subprocess(evaluator, Script, environment):
def test_killed_subprocess(inference_state, Script, environment):
if isinstance(environment, InterpreterEnvironment):
pytest.skip("We cannot kill our own process")
# Just kill the subprocess.
evaluator.compiled_subprocess._compiled_subprocess._get_process().kill()
inference_state.compiled_subprocess._compiled_subprocess._get_process().kill()
# Since the process was terminated (and nobody knows about it) the first
# Jedi call fails.
with pytest.raises(jedi.InternalError):
test/test_api/test_interpreter.py
+3 -3
View File
@@ -7,7 +7,7 @@ import pytest
import jedi
from jedi._compatibility import is_py3, py_version
from jedi.evaluate.compiled import mixed, context
from jedi.inference.compiled import mixed, value
from importlib import import_module
if py_version > 30:
@@ -101,8 +101,8 @@ def test_side_effect_completion():
side_effect = get_completion('SideEffectContainer', _GlobalNameSpace.__dict__)
# It's a class that contains MixedObject.
context, = side_effect._name.infer()
assert isinstance(context, mixed.MixedObject)
value, = side_effect._name.infer()
assert isinstance(value, mixed.MixedObject)
foo = get_completion('SideEffectContainer.foo', _GlobalNameSpace.__dict__)
assert foo.name == 'foo'
test/test_api/test_project.py
+2 -2
View File
@@ -13,12 +13,12 @@ def test_django_default_project(Script):
)
c, = script.completions()
assert c.name == "SomeModel"
assert script._evaluator.project._django is True
assert script._inference_state.project._django is True
def test_interpreter_project_path():
# Run from anywhere it should be the cwd.
dir = os.path.join(root_dir, 'test')
with set_cwd(dir):
project = Interpreter('', [locals()])._evaluator.project
project = Interpreter('', [locals()])._inference_state.project
assert project._path == dir
test/test_api/test_settings.py
+2 -2
View File
@@ -3,7 +3,7 @@ import os
import pytest
from jedi import api
from jedi.evaluate import imports
from jedi.inference import imports
from ..helpers import cwd_at
@@ -17,7 +17,7 @@ def test_add_dynamic_mods(Script):
# Other fictional modules in another place in the fs.
src2 = 'from .. import setup; setup.r(1)'
script = Script(src1, path='../setup.py')
imports.load_module(script._evaluator, os.path.abspath(fname), src2)
imports.load_module(script._inference_state, os.path.abspath(fname), src2)
result = script.goto_definitions()
assert len(result) == 1
assert result[0].description == 'class int'
test/test_evaluate/test_literals.py
-46
View File
@@ -1,46 +0,0 @@
import pytest
from jedi.evaluate.context import TreeInstance
def _eval_literal(Script, code, is_fstring=False):
def_, = Script(code).goto_definitions()
if is_fstring:
assert def_.name == 'str'
assert isinstance(def_._name._context, TreeInstance)
return ''
else:
return def_._name._context.get_safe_value()
def test_f_strings(Script, environment):
"""
f literals are not really supported in Jedi. They just get ignored and an
empty string is returned.
"""
if environment.version_info < (3, 6):
pytest.skip()
assert _eval_literal(Script, 'f"asdf"', is_fstring=True) == ''
assert _eval_literal(Script, 'f"{asdf} "', is_fstring=True) == ''
assert _eval_literal(Script, 'F"{asdf} "', is_fstring=True) == ''
assert _eval_literal(Script, 'rF"{asdf} "', is_fstring=True) == ''
def test_rb_strings(Script, environment):
assert _eval_literal(Script, 'br"asdf"') == b'asdf'
obj = _eval_literal(Script, 'rb"asdf"')
# rb is not valid in Python 2. Due to error recovery we just get a
# string.
assert obj == b'asdf'
def test_thousand_separators(Script, environment):
if environment.version_info < (3, 6):
pytest.skip()
assert _eval_literal(Script, '1_2_3') == 123
assert _eval_literal(Script, '123_456_789') == 123456789
assert _eval_literal(Script, '0x3_4') == 52
assert _eval_literal(Script, '0b1_0') == 2
assert _eval_literal(Script, '0o1_0') == 8
test/test_evaluate/__init__.py → test/test_inference/__init__.py
View File
test/test_evaluate/absolute_import/local_module.py → test/test_inference/absolute_import/local_module.py
View File
test/test_evaluate/absolute_import/unittest.py → test/test_inference/absolute_import/unittest.py
View File
test/test_evaluate/flask-site-packages/flask/__init__.py → test/test_inference/flask-site-packages/flask/__init__.py
View File
test/test_evaluate/flask-site-packages/flask/ext/__init__.py → test/test_inference/flask-site-packages/flask/ext/__init__.py
View File
test/test_evaluate/flask-site-packages/flask_baz/__init__.py → test/test_inference/flask-site-packages/flask_baz/__init__.py
View File
test/test_evaluate/flask-site-packages/flask_foo.py → test/test_inference/flask-site-packages/flask_foo.py
View File
test/test_evaluate/flask-site-packages/flaskext/__init__.py → test/test_inference/flask-site-packages/flaskext/__init__.py
View File

Some files were not shown because too many files have changed in this diff Show More