diff --git a/.coveragerc b/.coveragerc index c241dc7e..ba916f02 100644 --- a/.coveragerc +++ b/.coveragerc @@ -4,7 +4,6 @@ omit = jedi/inference/compiled/subprocess/__main__.py jedi/__main__.py # For now this is not being used. - jedi/refactoring.py [report] # Regexes for lines to exclude from consideration diff --git a/.travis.yml b/.travis.yml index 324e27a6..8c0b9a38 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,20 +1,18 @@ dist: xenial language: python python: - - 2.7 - - 3.4 - - 3.5 - - 3.6 - - 3.7 - 3.8 + - 3.7 + - 3.6 + - 3.5 + - 2.7 env: - - JEDI_TEST_ENVIRONMENT=27 - - JEDI_TEST_ENVIRONMENT=34 - - JEDI_TEST_ENVIRONMENT=35 - - JEDI_TEST_ENVIRONMENT=36 - - JEDI_TEST_ENVIRONMENT=37 - JEDI_TEST_ENVIRONMENT=38 + - JEDI_TEST_ENVIRONMENT=37 + - JEDI_TEST_ENVIRONMENT=36 + - JEDI_TEST_ENVIRONMENT=35 + - JEDI_TEST_ENVIRONMENT=27 - JEDI_TEST_ENVIRONMENT=interpreter matrix: @@ -42,7 +40,8 @@ script: python_bin=python$test_env_version python_path="$(which $python_bin || true)" if [ -z "$python_path" ]; then - # Only required for JEDI_TEST_ENVIRONMENT=34. + # Only required for JEDI_TEST_ENVIRONMENT=38, because it's not always + # available. download_name=python-$test_env_version wget https://s3.amazonaws.com/travis-python-archives/binaries/ubuntu/16.04/x86_64/$download_name.tar.bz2 sudo tar xjf $download_name.tar.bz2 --directory / opt/python diff --git a/AUTHORS.txt b/AUTHORS.txt index 0cd22f7e..41e72edf 100644 --- a/AUTHORS.txt +++ b/AUTHORS.txt @@ -1,60 +1,62 @@ Main Authors -============ +------------ -David Halter (@davidhalter) -Takafumi Arakaki (@tkf) +- David Halter (@davidhalter) +- Takafumi Arakaki (@tkf) Code Contributors -================= +----------------- -Danilo Bargen (@dbrgn) -Laurens Van Houtven (@lvh) <_@lvh.cc> -Aldo Stracquadanio (@Astrac) -Jean-Louis Fuchs (@ganwell) -tek (@tek) -Yasha Borevich (@jjay) -Aaron Griffin -andviro (@andviro) -Mike Gilbert (@floppym) -Aaron Meurer (@asmeurer) -Lubos Trilety -Akinori Hattori (@hattya) -srusskih (@srusskih) -Steven Silvester (@blink1073) -Colin Duquesnoy (@ColinDuquesnoy) -Jorgen Schaefer (@jorgenschaefer) -Fredrik Bergroth (@fbergroth) -Mathias Fußenegger (@mfussenegger) -Syohei Yoshida (@syohex) -ppalucky (@ppalucky) -immerrr (@immerrr) immerrr@gmail.com -Albertas Agejevas (@alga) -Savor d'Isavano (@KenetJervet) -Phillip Berndt (@phillipberndt) -Ian Lee (@IanLee1521) -Farkhad Khatamov (@hatamov) -Kevin Kelley (@kelleyk) -Sid Shanker (@squidarth) -Reinoud Elhorst (@reinhrst) -Guido van Rossum (@gvanrossum) -Dmytro Sadovnychyi (@sadovnychyi) -Cristi Burcă (@scribu) -bstaint (@bstaint) -Mathias Rav (@Mortal) -Daniel Fiterman (@dfit99) -Simon Ruggier (@sruggier) -Élie Gouzien (@ElieGouzien) -Robin Roth (@robinro) -Malte Plath (@langsamer) -Anton Zub (@zabulazza) -Maksim Novikov (@m-novikov) -Tobias Rzepka (@TobiasRzepka) -micbou (@micbou) -Dima Gerasimov (@karlicoss) -Max Woerner Chase (@mwchase) -Johannes Maria Frank (@jmfrank63) -Shane Steinert-Threlkeld (@shanest) -Tim Gates (@timgates42) -Lior Goldberg (@goldberglior) +- Danilo Bargen (@dbrgn) +- Laurens Van Houtven (@lvh) <_@lvh.cc> +- Aldo Stracquadanio (@Astrac) +- Jean-Louis Fuchs (@ganwell) +- tek (@tek) +- Yasha Borevich (@jjay) +- Aaron Griffin +- andviro (@andviro) +- Mike Gilbert (@floppym) +- Aaron Meurer (@asmeurer) +- Lubos Trilety +- Akinori Hattori (@hattya) +- srusskih (@srusskih) +- Steven Silvester (@blink1073) +- Colin Duquesnoy (@ColinDuquesnoy) +- Jorgen Schaefer (@jorgenschaefer) +- Fredrik Bergroth (@fbergroth) +- Mathias Fußenegger (@mfussenegger) +- Syohei Yoshida (@syohex) +- ppalucky (@ppalucky) +- immerrr (@immerrr) immerrr@gmail.com +- Albertas Agejevas (@alga) +- Savor d'Isavano (@KenetJervet) +- Phillip Berndt (@phillipberndt) +- Ian Lee (@IanLee1521) +- Farkhad Khatamov (@hatamov) +- Kevin Kelley (@kelleyk) +- Sid Shanker (@squidarth) +- Reinoud Elhorst (@reinhrst) +- Guido van Rossum (@gvanrossum) +- Dmytro Sadovnychyi (@sadovnychyi) +- Cristi Burcă (@scribu) +- bstaint (@bstaint) +- Mathias Rav (@Mortal) +- Daniel Fiterman (@dfit99) +- Simon Ruggier (@sruggier) +- Élie Gouzien (@ElieGouzien) +- Robin Roth (@robinro) +- Malte Plath (@langsamer) +- Anton Zub (@zabulazza) +- Maksim Novikov (@m-novikov) +- Tobias Rzepka (@TobiasRzepka) +- micbou (@micbou) +- Dima Gerasimov (@karlicoss) +- Max Woerner Chase (@mwchase) +- Johannes Maria Frank (@jmfrank63) +- Shane Steinert-Threlkeld (@shanest) +- Tim Gates (@timgates42) +- Lior Goldberg (@goldberglior) + +And a few more "anonymous" contributors. Note: (@user) means a github user name. diff --git a/CHANGELOG.rst b/CHANGELOG.rst index 2d7bcccd..bd647c58 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -3,9 +3,32 @@ Changelog --------- -0.16.1 (2020--) +0.17.0 (2020-03-) +++++++++++++++++++ +- Added support projects. This allows user to specify which folders Jedi should + search. +- Added support for Refactoring. The following refactorings have been + implemented: ``Script.rename``, ``Script.inline``, + ``Script.extract_variable`` and ``Script.extract_function``. +- Added ``Script.get_syntax_errors`` to display syntax errors in the current + script. +- Added code search capabilities both for individual files and projects. The + new functions are ``Project.search``, ``Project.complete_search``, + ``Script.search`` and ``Script.complete_search``. +- Added ``Script.help`` to make it easier to display a help window to people. + Now returns pydoc information as well for Python keywords/operators. This + means that on the class keyword it will now return the docstring of Python's + builtin function ``help('class')``. +- The API documentation is now way more readable and complete. Check it out + under https://jedi.readthedocs.io. A lot of it has been rewritten. +- Removed Python 3.4 support +- Many bugfixes + +This is likely going to be the last minor version that supports Python 2 and +Python3.5. Bugfixes will be provided in 0.17.1+. The next minor/major version +will probably be Jedi 1.0.0. + 0.16.0 (2020-01-26) +++++++++++++++++++ @@ -28,8 +51,8 @@ Changelog - ``call_signatures`` deprecated, use ``get_signatures`` instead - ``usages`` deprecated, use ``get_references`` instead - ``jedi.names`` deprecated, use ``jedi.Script(...).get_names()`` -- ``BaseDefinition.goto_assignments`` renamed to ``BaseDefinition.goto`` -- Add follow_imports to ``Definition.goto``. Now its signature matches +- ``BaseName.goto_assignments`` renamed to ``BaseName.goto`` +- Add follow_imports to ``Name.goto``. Now its signature matches ``Script.goto``. - **Python 2 support deprecated**. For this release it is best effort. Python 2 has reached the end of its life and now it's just about a smooth transition. @@ -69,13 +92,13 @@ Changelog New APIs: -- ``Definition.get_signatures() -> List[Signature]``. Signatures are similar to - ``CallSignature``. ``Definition.params`` is therefore deprecated. +- ``Name.get_signatures() -> List[Signature]``. Signatures are similar to + ``CallSignature``. ``Name.params`` is therefore deprecated. - ``Signature.to_string()`` to format signatures. -- ``Signature.params -> List[ParamDefinition]``, ParamDefinition has the +- ``Signature.params -> List[ParamName]``, ParamName has the following additional attributes ``infer_default()``, ``infer_annotation()``, ``to_string()``, and ``kind``. -- ``Definition.execute() -> List[Definition]``, makes it possible to infer +- ``Name.execute() -> List[Name]``, makes it possible to infer return values of functions. @@ -91,7 +114,7 @@ New APIs: - Added ``goto_*(prefer_stubs=True)`` as well as ``goto_*(prefer_stubs=True)`` - Stubs are used now for type inference - Typeshed is used for better type inference -- Reworked Definition.full_name, should have more correct return values +- Reworked Name.full_name, should have more correct return values 0.13.3 (2019-02-24) +++++++++++++++++++ @@ -171,7 +194,7 @@ New APIs: - Actual semantic completions for the complete Python syntax. - Basic type inference for ``yield from`` PEP 380. - PEP 484 support (most of the important features of it). Thanks Claude! (@reinhrst) -- Added ``get_line_code`` to ``Definition`` and ``Completion`` objects. +- Added ``get_line_code`` to ``Name`` and ``Completion`` objects. - Completely rewritten the type inference engine. - A new and better parser for (fast) parsing diffs of Python code. diff --git a/README.rst b/README.rst index bf8f3c8c..581b5cb1 100644 --- a/README.rst +++ b/README.rst @@ -1,14 +1,14 @@ -################################################################### -Jedi - an awesome autocompletion/static analysis library for Python -################################################################### +#################################################################################### +Jedi - an awesome autocompletion, static analysis and refactoring library for Python +#################################################################################### -.. image:: https://img.shields.io/pypi/v/jedi.svg?style=flat - :target: https://pypi.python.org/pypi/jedi - :alt: PyPI version +.. image:: http://isitmaintained.com/badge/open/davidhalter/jedi.svg + :target: https://github.com/davidhalter/jedi/issues + :alt: The percentage of open issues and pull requests -.. image:: https://img.shields.io/pypi/pyversions/jedi.svg - :target: https://pypi.python.org/pypi/jedi - :alt: Supported Python versions +.. image:: http://isitmaintained.com/badge/resolution/davidhalter/jedi.svg + :target: https://github.com/davidhalter/jedi/issues + :alt: The resolution time is the median time an issue or pull request stays open. .. image:: https://travis-ci.org/davidhalter/jedi.svg?branch=master :target: https://travis-ci.org/davidhalter/jedi @@ -23,22 +23,15 @@ Jedi - an awesome autocompletion/static analysis library for Python :alt: Coverage status -*If you have specific questions, please add an issue or ask on* `Stack Overflow -`_ *with the label* ``python-jedi``. +Jedi is a static analysis tool for Python that is typically used in +IDEs/editors plugins. Jedi has a focus on autocompletion and goto +functionality. Other features include refactoring, code search and finding +references. - -Jedi is a static analysis tool for Python that can be used in IDEs/editors. -Jedi has a focus on autocompletion and goto functionality. Jedi is fast and is -very well tested. It understands Python and stubs on a deep level. - -Jedi has support for different goto functions. It's possible to search for -references and list names in a Python file to get information about them. - -Jedi uses a very simple API to connect with IDE's. There's a reference -implementation as a `VIM-Plugin `_, -which uses Jedi's autocompletion. We encourage you to use Jedi in your IDEs. -Autocompletion in your REPL is also possible, IPython uses it natively and for -the CPython REPL you have to install it. +Jedi has a simple API to work with. There is a reference implementation as a +`VIM-Plugin `_. Autocompletion in your +REPL is also possible, IPython uses it natively and for the CPython REPL you +can install it. Jedi is well tested and bugs should be rare. Jedi can currently be used with the following editors/projects: @@ -47,7 +40,7 @@ Jedi can currently be used with the following editors/projects: - Emacs (Jedi.el_, company-mode_, elpy_, anaconda-mode_, ycmd_) - Sublime Text (SublimeJEDI_ [ST2 + ST3], anaconda_ [only ST3]) - TextMate_ (Not sure if it's actually working) -- Kate_ version 4.13+ supports it natively, you have to enable it, though. [`proof +- Kate_ version 4.13+ supports it natively, you have to enable it, though. [`see `_] - Atom_ (autocomplete-python-jedi_) - `GNOME Builder`_ (with support for GObject Introspection) @@ -58,7 +51,6 @@ Jedi can currently be used with the following editors/projects: and many more! - Here are some pictures taken from jedi-vim_: .. image:: https://github.com/davidhalter/jedi/raw/master/docs/_screenshots/screenshot_complete.png @@ -71,47 +63,41 @@ Display of function/class bodies, docstrings. .. image:: https://github.com/davidhalter/jedi/raw/master/docs/_screenshots/screenshot_pydoc.png -Pydoc support (Shift+k). - -There is also support for goto and renaming. +Documentation support (Shift+k). Get the latest version from `github `_ (master branch should always be kind of stable/working). Docs are available at `https://jedi.readthedocs.org/en/latest/ -`_. Pull requests with documentation -enhancements and/or fixes are awesome and most welcome. Jedi uses `semantic -versioning `_. +`_. Pull requests with enhancements +and/or fixes are awesome and most welcome. Jedi uses `semantic versioning +`_. If you want to stay up-to-date (News / RFCs), please subscribe to this `github thread `_.: +Issues & Questions +================== +You can file issues and questions in the `issue tracker +`. Alternatively you can also ask on +`Stack Overflow `_ with +the label ``python-jedi``. Installation ============ - pip install jedi +`Check out the docs `_. -Note: This just installs the Jedi library, not the editor plugins. For -information about how to make it work with your editor, refer to the -corresponding documentation. +Features and Limitations +======================== -You don't want to use ``pip``? Please refer to the `manual -`_. +Jedi's features are listed here: +`Features `_. - -Feature Support and Caveats -=========================== - -Jedi really understands your Python code. For a comprehensive list what Jedi -understands, see: `Features -`_. A list of -caveats can be found on the same page. - -You can run Jedi on CPython 2.7 or 3.4+ but it should also -understand/parse code older than those versions. Additionally you should be able -to use `Virtualenvs `_ +You can run Jedi on CPython 2.7 or 3.5+ but it should also +understand code that is older than those versions. Additionally you should be +able to use `Virtualenvs `_ very well. Tips on how to use Jedi efficiently can be found `here @@ -120,47 +106,62 @@ Tips on how to use Jedi efficiently can be found `here API --- -You can find the documentation for the `API here `_. +You can find a comprehensive documentation for the +`API here `_. +Autocompletion / Goto / Documentation +------------------------------------- -Autocompletion / Goto / Pydoc ------------------------------ - -Please check the API for a good explanation. There are the following commands: +There are the following commands: - ``jedi.Script.goto`` +- ``jedi.Script.infer`` +- ``jedi.Script.help`` - ``jedi.Script.complete`` - ``jedi.Script.get_references`` +- ``jedi.Script.get_signatures`` +- ``jedi.Script.get_context`` -The returned objects are very powerful and really all you might need. - +The returned objects are very powerful and are really all you might need. Autocompletion in your REPL (IPython, etc.) ------------------------------------------- -Starting with IPython `6.0.0` Jedi is a dependency of IPython. Autocompletion -in IPython is therefore possible without additional configuration. +Jedi is a dependency of IPython. Autocompletion in IPython with Jedi is +therefore possible without additional configuration. -It's possible to have Jedi autocompletion in REPL modes - `example video `_. -This means that in Python you can enable tab completion in a `REPL +Here is an `example video `_ how REPL completion +can look like. +For the ``python`` shell you can enable tab completion in a `REPL `_. - Static Analysis ------------------------- +--------------- -To do all forms of static analysis, please try to use -``jedi.Script(...).get_names``. It will return a list of names that you can use -to infer types and so on. +For a lot of forms of static analysis, you can try to use +``jedi.Script(...).get_names``. It will return a list of names that you can +then filter and work with. There is also a way to list the syntax errors in a +file: ``jedi.Script.get_syntax_errors``. Refactoring ----------- -Jedi's parser would support refactoring, but there's no API to use it right -now. If you're interested in helping out here, let me know. With the latest -parser changes, it should be very easy to actually make it work. +Jedi supports the following refactorings: +- ``jedi.Script.inline`` +- ``jedi.Script.rename`` +- ``jedi.Script.extract_function`` +- ``jedi.Script.extract_variable`` + +Code Search +----------- + +There is support for module search with ``jedi.Script.search``, and project +search for ``jedi.Project.search``. The way to search is either by providing a +name like ``foo`` or by using dotted syntax like ``foo.bar``. Additionally you +can provide the API type like ``class foo.bar.Bar``. There are also the +functions ``jedi.Script.complete_search`` and ``jedi.Project.complete_search``. Development =========== @@ -168,39 +169,26 @@ Development There's a pretty good and extensive `development documentation `_. - Testing ======= -The test suite depends on ``tox`` and ``pytest``:: +The test suite uses ``pytest``:: - pip install tox pytest + pip install pytest -To run the tests for all supported Python versions:: +If you want to test only a specific Python version (e.g. Python 3.8), it is as +easy as:: - tox - -If you want to test only a specific Python version (e.g. Python 2.7), it's as -easy as :: - - tox -e py27 - -Tests are also run automatically on `Travis CI -`_. + python3.8 -m pytest For more detailed information visit the `testing documentation `_. - Acknowledgements ================ -- Takafumi Arakaki (@tkf) for creating a solid test environment and a lot of - other things. -- Danilo Bargen (@dbrgn) for general housekeeping and being a good friend :). -- Guido van Rossum (@gvanrossum) for creating the parser generator pgen2 - (originally used in lib2to3). - +Thanks a lot to all the +`contributors `_! .. _jedi-vim: https://github.com/davidhalter/jedi-vim diff --git a/appveyor.yml b/appveyor.yml index 18605d8a..a68fa11e 100644 --- a/appveyor.yml +++ b/appveyor.yml @@ -1,68 +1,56 @@ environment: matrix: - - TOXENV: py27 - PYTHON_PATH: C:\Python27 - JEDI_TEST_ENVIRONMENT: 27 - - TOXENV: py27 - PYTHON_PATH: C:\Python27 - JEDI_TEST_ENVIRONMENT: 34 - - TOXENV: py27 - PYTHON_PATH: C:\Python27 - JEDI_TEST_ENVIRONMENT: 35 - - TOXENV: py27 - PYTHON_PATH: C:\Python27 - JEDI_TEST_ENVIRONMENT: 36 - - TOXENV: py27 - PYTHON_PATH: C:\Python27 + - TOXENV: py37 + PYTHON_PATH: C:\Python37 JEDI_TEST_ENVIRONMENT: 37 - - - TOXENV: py35 - PYTHON_PATH: C:\Python35 - JEDI_TEST_ENVIRONMENT: 27 - - TOXENV: py35 - PYTHON_PATH: C:\Python35 - JEDI_TEST_ENVIRONMENT: 34 - - TOXENV: py35 - PYTHON_PATH: C:\Python35 - JEDI_TEST_ENVIRONMENT: 35 - - TOXENV: py35 - PYTHON_PATH: C:\Python35 + - TOXENV: py37 + PYTHON_PATH: C:\Python37 JEDI_TEST_ENVIRONMENT: 36 - - TOXENV: py35 - PYTHON_PATH: C:\Python35 - JEDI_TEST_ENVIRONMENT: 37 + - TOXENV: py37 + PYTHON_PATH: C:\Python37 + JEDI_TEST_ENVIRONMENT: 35 + - TOXENV: py37 + PYTHON_PATH: C:\Python37 + JEDI_TEST_ENVIRONMENT: 27 - TOXENV: py36 PYTHON_PATH: C:\Python36 - JEDI_TEST_ENVIRONMENT: 27 - - TOXENV: py36 - PYTHON_PATH: C:\Python36 - JEDI_TEST_ENVIRONMENT: 34 - - TOXENV: py36 - PYTHON_PATH: C:\Python36 - JEDI_TEST_ENVIRONMENT: 35 + JEDI_TEST_ENVIRONMENT: 37 - TOXENV: py36 PYTHON_PATH: C:\Python36 JEDI_TEST_ENVIRONMENT: 36 - TOXENV: py36 PYTHON_PATH: C:\Python36 - JEDI_TEST_ENVIRONMENT: 37 + JEDI_TEST_ENVIRONMENT: 35 + - TOXENV: py36 + PYTHON_PATH: C:\Python36 + JEDI_TEST_ENVIRONMENT: 27 - - TOXENV: py37 - PYTHON_PATH: C:\Python37 - JEDI_TEST_ENVIRONMENT: 27 - - TOXENV: py37 - PYTHON_PATH: C:\Python37 - JEDI_TEST_ENVIRONMENT: 34 - - TOXENV: py37 - PYTHON_PATH: C:\Python37 - JEDI_TEST_ENVIRONMENT: 35 - - TOXENV: py37 - PYTHON_PATH: C:\Python37 - JEDI_TEST_ENVIRONMENT: 36 - - TOXENV: py37 - PYTHON_PATH: C:\Python37 + - TOXENV: py35 + PYTHON_PATH: C:\Python35 JEDI_TEST_ENVIRONMENT: 37 + - TOXENV: py35 + PYTHON_PATH: C:\Python35 + JEDI_TEST_ENVIRONMENT: 36 + - TOXENV: py35 + PYTHON_PATH: C:\Python35 + JEDI_TEST_ENVIRONMENT: 35 + - TOXENV: py35 + PYTHON_PATH: C:\Python35 + JEDI_TEST_ENVIRONMENT: 27 + + - TOXENV: py27 + PYTHON_PATH: C:\Python27 + JEDI_TEST_ENVIRONMENT: 37 + - TOXENV: py27 + PYTHON_PATH: C:\Python27 + JEDI_TEST_ENVIRONMENT: 36 + - TOXENV: py27 + PYTHON_PATH: C:\Python27 + JEDI_TEST_ENVIRONMENT: 35 + - TOXENV: py27 + PYTHON_PATH: C:\Python27 + JEDI_TEST_ENVIRONMENT: 27 install: - git submodule update --init --recursive - set PATH=%PYTHON_PATH%;%PYTHON_PATH%\Scripts;%PATH% diff --git a/docs/_static/custom_style.css b/docs/_static/custom_style.css new file mode 100644 index 00000000..a9e8b807 --- /dev/null +++ b/docs/_static/custom_style.css @@ -0,0 +1,9 @@ +div.version { + color: black !important; + margin-top: -1.2em !important; + margin-bottom: .6em !important; +} + +div.wy-side-nav-search { + padding-top: 0 !important; +} diff --git a/docs/_themes/flask/LICENSE b/docs/_themes/flask/LICENSE deleted file mode 100644 index 8daab7ee..00000000 --- a/docs/_themes/flask/LICENSE +++ /dev/null @@ -1,37 +0,0 @@ -Copyright (c) 2010 by Armin Ronacher. - -Some rights reserved. - -Redistribution and use in source and binary forms of the theme, with or -without modification, are permitted provided that the following conditions -are met: - -* Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - -* Redistributions in binary form must reproduce the above - copyright notice, this list of conditions and the following - disclaimer in the documentation and/or other materials provided - with the distribution. - -* The names of the contributors may not be used to endorse or - promote products derived from this software without specific - prior written permission. - -We kindly ask you to only use these themes in an unmodified manner just -for Flask and Flask-related products, not for unrelated projects. If you -like the visual style and want to use it for your own projects, please -consider making some larger changes to the themes (such as changing -font faces, sizes, colors or margins). - -THIS THEME IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE -LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR -CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF -SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS -INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN -CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) -ARISING IN ANY WAY OUT OF THE USE OF THIS THEME, EVEN IF ADVISED OF THE -POSSIBILITY OF SUCH DAMAGE. diff --git a/docs/_themes/flask/layout.html b/docs/_themes/flask/layout.html deleted file mode 100644 index 48cb4d5f..00000000 --- a/docs/_themes/flask/layout.html +++ /dev/null @@ -1,27 +0,0 @@ -{%- extends "basic/layout.html" %} -{%- block extrahead %} - {{ super() }} - {% if theme_touch_icon %} - - {% endif %} - - - Fork me on GitHub - -{% endblock %} -{%- block relbar2 %}{% endblock %} -{% block header %} - {{ super() }} - {% if pagename == 'index' %} -
- {% endif %} -{% endblock %} -{%- block footer %} -
- Created using Sphinx. -
- {% if pagename == 'index' %} -
- {% endif %} -{%- endblock %} diff --git a/docs/_themes/flask/relations.html b/docs/_themes/flask/relations.html deleted file mode 100644 index 3bbcde85..00000000 --- a/docs/_themes/flask/relations.html +++ /dev/null @@ -1,19 +0,0 @@ -

Related Topics

- diff --git a/docs/_themes/flask/static/flasky.css_t b/docs/_themes/flask/static/flasky.css_t deleted file mode 100644 index 79ab4787..00000000 --- a/docs/_themes/flask/static/flasky.css_t +++ /dev/null @@ -1,394 +0,0 @@ -/* - * flasky.css_t - * ~~~~~~~~~~~~ - * - * :copyright: Copyright 2010 by Armin Ronacher. - * :license: Flask Design License, see LICENSE for details. - */ - -{% set page_width = '940px' %} -{% set sidebar_width = '220px' %} - -@import url("basic.css"); - -/* -- page layout ----------------------------------------------------------- */ - -body { - font-family: 'Georgia', serif; - font-size: 17px; - background-color: white; - color: #000; - margin: 0; - padding: 0; -} - -div.document { - width: {{ page_width }}; - margin: 30px auto 0 auto; -} - -div.documentwrapper { - float: left; - width: 100%; -} - -div.bodywrapper { - margin: 0 0 0 {{ sidebar_width }}; -} - -div.sphinxsidebar { - width: {{ sidebar_width }}; -} - -hr { - border: 1px solid #B1B4B6; -} - -div.body { - background-color: #ffffff; - color: #3E4349; - padding: 0 30px 0 30px; -} - -img.floatingflask { - padding: 0 0 10px 10px; - float: right; -} - -div.footer { - width: {{ page_width }}; - margin: 20px auto 30px auto; - font-size: 14px; - color: #888; - text-align: right; -} - -div.footer a { - color: #888; -} - -div.related { - display: none; -} - -div.sphinxsidebar a { - color: #444; - text-decoration: none; - border-bottom: 1px dotted #999; -} - -div.sphinxsidebar a:hover { - border-bottom: 1px solid #999; -} - -div.sphinxsidebar { - font-size: 14px; - line-height: 1.5; -} - -div.sphinxsidebarwrapper { - padding: 18px 10px; -} - -div.sphinxsidebarwrapper p.logo { - padding: 0 0 20px 0; - margin: 0; - text-align: center; -} - -div.sphinxsidebar h3, -div.sphinxsidebar h4 { - font-family: 'Garamond', 'Georgia', serif; - color: #444; - font-size: 24px; - font-weight: normal; - margin: 0 0 5px 0; - padding: 0; -} - -div.sphinxsidebar h4 { - font-size: 20px; -} - -div.sphinxsidebar h3 a { - color: #444; -} - -div.sphinxsidebar p.logo a, -div.sphinxsidebar h3 a, -div.sphinxsidebar p.logo a:hover, -div.sphinxsidebar h3 a:hover { - border: none; -} - -div.sphinxsidebar p { - color: #555; - margin: 10px 0; -} - -div.sphinxsidebar ul { - margin: 10px 0; - padding: 0; - color: #000; -} - -div.sphinxsidebar input { - border: 1px solid #ccc; - font-family: 'Georgia', serif; - font-size: 1em; -} - -/* -- body styles ----------------------------------------------------------- */ - -a { - color: #004B6B; - text-decoration: underline; -} - -a:hover { - color: #6D4100; - text-decoration: underline; -} - -div.body h1, -div.body h2, -div.body h3, -div.body h4, -div.body h5, -div.body h6 { - font-family: 'Garamond', 'Georgia', serif; - font-weight: normal; - margin: 30px 0px 10px 0px; - padding: 0; -} - -{% if theme_index_logo %} -div.indexwrapper h1 { - text-indent: -999999px; - background: url({{ theme_index_logo }}) no-repeat center center; - height: {{ theme_index_logo_height }}; -} -{% endif %} - -div.body h1 { margin-top: 0; padding-top: 0; font-size: 240%; } -div.body h2 { font-size: 180%; } -div.body h3 { font-size: 150%; } -div.body h4 { font-size: 130%; } -div.body h5 { font-size: 100%; } -div.body h6 { font-size: 100%; } - -a.headerlink { - color: #ddd; - padding: 0 4px; - text-decoration: none; -} - -a.headerlink:hover { - color: #444; -} - -div.body p, div.body dd, div.body li { - line-height: 1.4em; -} - -div.admonition { - background: #fafafa; - margin: 20px -30px; - padding: 10px 30px; - border-top: 1px solid #ccc; - border-bottom: 1px solid #ccc; -} - -div.admonition tt.xref, div.admonition a tt { - border-bottom: 1px solid #fafafa; -} - -dd div.admonition { - margin-left: -60px; - padding-left: 60px; -} - -div.admonition p.admonition-title { - font-family: 'Garamond', 'Georgia', serif; - font-weight: normal; - font-size: 24px; - margin: 0 0 10px 0; - padding: 0; - line-height: 1; -} - -div.admonition p.last { - margin-bottom: 0; -} - -div.highlight { - background-color: white; -} - -dt:target, .highlight { - background: #FAF3E8; -} - -div.note { - background-color: #eee; - border: 1px solid #ccc; -} - -div.seealso { - background-color: #ffc; - border: 1px solid #ff6; -} - -div.topic { - background-color: #eee; -} - -p.admonition-title { - display: inline; -} - -p.admonition-title:after { - content: ":"; -} - -pre, tt { - font-family: 'Consolas', 'Menlo', 'Deja Vu Sans Mono', 'Bitstream Vera Sans Mono', monospace; - font-size: 0.9em; -} - -img.screenshot { -} - -tt.descname, tt.descclassname { - font-size: 0.95em; -} - -tt.descname { - padding-right: 0.08em; -} - -img.screenshot { - -moz-box-shadow: 2px 2px 4px #eee; - -webkit-box-shadow: 2px 2px 4px #eee; - box-shadow: 2px 2px 4px #eee; -} - -table.docutils { - border: 1px solid #888; - -moz-box-shadow: 2px 2px 4px #eee; - -webkit-box-shadow: 2px 2px 4px #eee; - box-shadow: 2px 2px 4px #eee; -} - -table.docutils td, table.docutils th { - border: 1px solid #888; - padding: 0.25em 0.7em; -} - -table.field-list, table.footnote { - border: none; - -moz-box-shadow: none; - -webkit-box-shadow: none; - box-shadow: none; -} - -table.footnote { - margin: 15px 0; - width: 100%; - border: 1px solid #eee; - background: #fdfdfd; - font-size: 0.9em; -} - -table.footnote + table.footnote { - margin-top: -15px; - border-top: none; -} - -table.field-list th { - padding: 0 0.8em 0 0; -} - -table.field-list td { - padding: 0; -} - -table.footnote td.label { - width: 0px; - padding: 0.3em 0 0.3em 0.5em; -} - -table.footnote td { - padding: 0.3em 0.5em; -} - -dl { - margin: 0; - padding: 0; -} - -dl dd { - margin-left: 30px; -} - -blockquote { - margin: 0 0 0 30px; - padding: 0; -} - -ul, ol { - margin: 10px 0 10px 30px; - padding: 0; -} - -pre { - background: #eee; - padding: 7px 30px; - margin: 15px -30px; - line-height: 1.3em; -} - -dl pre, blockquote pre, li pre { - margin-left: -60px; - padding-left: 60px; -} - -dl dl pre { - margin-left: -90px; - padding-left: 90px; -} - -tt { - background-color: #ecf0f3; - color: #222; - /* padding: 1px 2px; */ -} - -tt.xref, a tt { - background-color: #FBFBFB; - border-bottom: 1px solid white; -} - -a.reference { - text-decoration: none; - border-bottom: 1px dotted #004B6B; -} - -a.reference:hover { - border-bottom: 1px solid #6D4100; -} - -a.footnote-reference { - text-decoration: none; - font-size: 0.7em; - vertical-align: top; - border-bottom: 1px dotted #004B6B; -} - -a.footnote-reference:hover { - border-bottom: 1px solid #6D4100; -} - -a:hover tt { - background: #EEE; -} diff --git a/docs/_themes/flask/static/small_flask.css b/docs/_themes/flask/static/small_flask.css deleted file mode 100644 index 1c6df309..00000000 --- a/docs/_themes/flask/static/small_flask.css +++ /dev/null @@ -1,70 +0,0 @@ -/* - * small_flask.css_t - * ~~~~~~~~~~~~~~~~~ - * - * :copyright: Copyright 2010 by Armin Ronacher. - * :license: Flask Design License, see LICENSE for details. - */ - -body { - margin: 0; - padding: 20px 30px; -} - -div.documentwrapper { - float: none; - background: white; -} - -div.sphinxsidebar { - display: block; - float: none; - width: 102.5%; - margin: 50px -30px -20px -30px; - padding: 10px 20px; - background: #333; - color: white; -} - -div.sphinxsidebar h3, div.sphinxsidebar h4, div.sphinxsidebar p, -div.sphinxsidebar h3 a { - color: white; -} - -div.sphinxsidebar a { - color: #aaa; -} - -div.sphinxsidebar p.logo { - display: none; -} - -div.document { - width: 100%; - margin: 0; -} - -div.related { - display: block; - margin: 0; - padding: 10px 0 20px 0; -} - -div.related ul, -div.related ul li { - margin: 0; - padding: 0; -} - -div.footer { - display: none; -} - -div.bodywrapper { - margin: 0; -} - -div.body { - min-height: 0; - padding: 0; -} diff --git a/docs/_themes/flask/theme.conf b/docs/_themes/flask/theme.conf deleted file mode 100644 index 1d5657f2..00000000 --- a/docs/_themes/flask/theme.conf +++ /dev/null @@ -1,9 +0,0 @@ -[theme] -inherit = basic -stylesheet = flasky.css -pygments_style = flask_theme_support.FlaskyStyle - -[options] -index_logo = -index_logo_height = 120px -touch_icon = diff --git a/docs/_themes/flask_theme_support.py b/docs/_themes/flask_theme_support.py deleted file mode 100644 index d3e33c06..00000000 --- a/docs/_themes/flask_theme_support.py +++ /dev/null @@ -1,125 +0,0 @@ -""" -Copyright (c) 2010 by Armin Ronacher. - -Some rights reserved. - -Redistribution and use in source and binary forms of the theme, with or -without modification, are permitted provided that the following conditions -are met: - -* Redistributions of source code must retain the above copyright - notice, this list of conditions and the following disclaimer. - -* Redistributions in binary form must reproduce the above - copyright notice, this list of conditions and the following - disclaimer in the documentation and/or other materials provided - with the distribution. - -* The names of the contributors may not be used to endorse or - promote products derived from this software without specific - prior written permission. - -We kindly ask you to only use these themes in an unmodified manner just -for Flask and Flask-related products, not for unrelated projects. If you -like the visual style and want to use it for your own projects, please -consider making some larger changes to the themes (such as changing -font faces, sizes, colors or margins). - -THIS THEME IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" -AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE -IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE -ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE -LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR -CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF -SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS -INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN -CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) -ARISING IN ANY WAY OUT OF THE USE OF THIS THEME, EVEN IF ADVISED OF THE -POSSIBILITY OF SUCH DAMAGE. -""" -# flasky extensions. flasky pygments style based on tango style -from pygments.style import Style -from pygments.token import Keyword, Name, Comment, String, Error, \ - Number, Operator, Generic, Whitespace, Punctuation, Other, Literal - - -class FlaskyStyle(Style): - background_color = "#f8f8f8" - default_style = "" - - styles = { - # No corresponding class for the following: - #Text: "", # class: '' - Whitespace: "underline #f8f8f8", # class: 'w' - Error: "#a40000 border:#ef2929", # class: 'err' - Other: "#000000", # class 'x' - - Comment: "italic #8f5902", # class: 'c' - Comment.Preproc: "noitalic", # class: 'cp' - - Keyword: "bold #004461", # class: 'k' - Keyword.Constant: "bold #004461", # class: 'kc' - Keyword.Declaration: "bold #004461", # class: 'kd' - Keyword.Namespace: "bold #004461", # class: 'kn' - Keyword.Pseudo: "bold #004461", # class: 'kp' - Keyword.Reserved: "bold #004461", # class: 'kr' - Keyword.Type: "bold #004461", # class: 'kt' - - Operator: "#582800", # class: 'o' - Operator.Word: "bold #004461", # class: 'ow' - like keywords - - Punctuation: "bold #000000", # class: 'p' - - # because special names such as Name.Class, Name.Function, etc. - # are not recognized as such later in the parsing, we choose them - # to look the same as ordinary variables. - Name: "#000000", # class: 'n' - Name.Attribute: "#c4a000", # class: 'na' - to be revised - Name.Builtin: "#004461", # class: 'nb' - Name.Builtin.Pseudo: "#3465a4", # class: 'bp' - Name.Class: "#000000", # class: 'nc' - to be revised - Name.Constant: "#000000", # class: 'no' - to be revised - Name.Decorator: "#888", # class: 'nd' - to be revised - Name.Entity: "#ce5c00", # class: 'ni' - Name.Exception: "bold #cc0000", # class: 'ne' - Name.Function: "#000000", # class: 'nf' - Name.Property: "#000000", # class: 'py' - Name.Label: "#f57900", # class: 'nl' - Name.Namespace: "#000000", # class: 'nn' - to be revised - Name.Other: "#000000", # class: 'nx' - Name.Tag: "bold #004461", # class: 'nt' - like a keyword - Name.Variable: "#000000", # class: 'nv' - to be revised - Name.Variable.Class: "#000000", # class: 'vc' - to be revised - Name.Variable.Global: "#000000", # class: 'vg' - to be revised - Name.Variable.Instance: "#000000", # class: 'vi' - to be revised - - Number: "#990000", # class: 'm' - - Literal: "#000000", # class: 'l' - Literal.Date: "#000000", # class: 'ld' - - String: "#4e9a06", # class: 's' - String.Backtick: "#4e9a06", # class: 'sb' - String.Char: "#4e9a06", # class: 'sc' - String.Doc: "italic #8f5902", # class: 'sd' - like a comment - String.Double: "#4e9a06", # class: 's2' - String.Escape: "#4e9a06", # class: 'se' - String.Heredoc: "#4e9a06", # class: 'sh' - String.Interpol: "#4e9a06", # class: 'si' - String.Other: "#4e9a06", # class: 'sx' - String.Regex: "#4e9a06", # class: 'sr' - String.Single: "#4e9a06", # class: 's1' - String.Symbol: "#4e9a06", # class: 'ss' - - Generic: "#000000", # class: 'g' - Generic.Deleted: "#a40000", # class: 'gd' - Generic.Emph: "italic #000000", # class: 'ge' - Generic.Error: "#ef2929", # class: 'gr' - Generic.Heading: "bold #000080", # class: 'gh' - Generic.Inserted: "#00A000", # class: 'gi' - Generic.Output: "#888", # class: 'go' - Generic.Prompt: "#745334", # class: 'gp' - Generic.Strong: "bold #000000", # class: 'gs' - Generic.Subheading: "bold #800080", # class: 'gu' - Generic.Traceback: "bold #a40000", # class: 'gt' - } diff --git a/docs/conf.py b/docs/conf.py index 4199bca8..57015f3f 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -13,13 +13,11 @@ import sys import os -import datetime # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the # documentation root, use os.path.abspath to make it absolute, like shown here. sys.path.insert(0, os.path.abspath('..')) -sys.path.append(os.path.abspath('_themes')) # -- General configuration ----------------------------------------------------- @@ -29,7 +27,8 @@ sys.path.append(os.path.abspath('_themes')) # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = ['sphinx.ext.autodoc', 'sphinx.ext.viewcode', 'sphinx.ext.todo', - 'sphinx.ext.intersphinx', 'sphinx.ext.inheritance_diagram'] + 'sphinx.ext.intersphinx', 'sphinx.ext.inheritance_diagram', + 'sphinx_rtd_theme', 'sphinx.ext.autosummary'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -54,8 +53,8 @@ from jedi.utils import version_info # |version| and |release|, also used in various other places throughout the # built documents. # -# The short X.Y version. -version = '.'.join(str(x) for x in version_info()[:2]) +# The short X.Y.Z version. +version = '.'.join(str(x) for x in version_info()[:3]) # The full version, including alpha/beta/rc tags. release = jedi.__version__ @@ -98,12 +97,15 @@ pygments_style = 'sphinx' # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. -html_theme = 'flask' +html_theme = 'sphinx_rtd_theme' # Theme options are theme-specific and customize the look and feel of a theme # further. For a list of options available for each theme, see the # documentation. -#html_theme_options = {} +html_theme_options = { + 'logo_only': True, + 'style_nav_header_background': 'white', +} # Add any paths that contain custom themes here, relative to this directory. html_theme_path = ['_themes'] @@ -117,7 +119,7 @@ html_theme_path = ['_themes'] # The name of an image file (relative to this directory) to place at the top # of the sidebar. -#html_logo = None +html_logo = '_static/logo.png' # The name of an image file (within the static path) to use as favicon of the # docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 @@ -129,6 +131,8 @@ html_theme_path = ['_themes'] # so a file named "default.css" will overwrite the builtin "default.css". html_static_path = ['_static'] +html_css_files = ['custom_style.css'] + # If not '', a 'Last updated on:' timestamp is inserted at every page bottom, # using the given strftime format. #html_last_updated_fmt = '%b %d, %Y' @@ -145,7 +149,7 @@ html_sidebars = { #'relations.html', 'ghbuttons.html', #'sourcelink.html', - #'searchbox.html' + 'searchbox.html' ] } @@ -163,13 +167,13 @@ html_sidebars = { #html_split_index = False # If true, links to the reST sources are added to the pages. -#html_show_sourcelink = True +html_show_sourcelink = False # If true, "Created using Sphinx" is shown in the HTML footer. Default is True. -#html_show_sphinx = True +html_show_sphinx = False # If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. -#html_show_copyright = True +html_show_copyright = False # If true, an OpenSearch description file will be output, and all pages will # contain a tag referring to it. The value of this option must be the @@ -274,7 +278,8 @@ autodoc_default_flags = [] # -- Options for intersphinx module -------------------------------------------- intersphinx_mapping = { - 'https://docs.python.org/': None, + 'python': ('https://docs.python.org/', None), + 'parso': ('https://parso.readthedocs.io/en/latest/', None), } diff --git a/docs/docs/acknowledgements.rst b/docs/docs/acknowledgements.rst new file mode 100644 index 00000000..52fa533d --- /dev/null +++ b/docs/docs/acknowledgements.rst @@ -0,0 +1,66 @@ +.. include global.rst + +History & Acknowledgements +========================== + +Acknowledgements +---------------- + +- Dave Halter for creating and maintaining Jedi & Parso. +- Takafumi Arakaki (@tkf) for creating a solid test environment and a lot of + other things. +- Danilo Bargen (@dbrgn) for general housekeeping and being a good friend :). +- Guido van Rossum (@gvanrossum) for creating the parser generator pgen2 + (originally used in lib2to3). +- Thanks to all the :ref:`contributors `. + +A Little Bit of History +----------------------- + +Written by Dave. + +The Star Wars Jedi are awesome. My Jedi software tries to imitate a little bit +of the precognition the Jedi have. There's even an awesome `scene +`_ of Monty Python Jedis :-). + +But actually the name has not much to do with Star Wars. It's part of my +second name Jedidjah. + +I actually started Jedi back in 2012, because there were no good solutions +available for VIM. Most auto-completion solutions just did not work well. The +only good solution was PyCharm. But I liked my good old VIM very much. There +was also a solution called Rope that did not work at all for me. So I decided +to write my own version of a completion engine. + +The first idea was to execute non-dangerous code. But I soon realized, that +this would not work. So I started to build a static analysis tool. +The biggest problem that I had at the time was that I did not know a thing +about parsers.I did not did not even know the word static analysis. It turns +out they are the foundation of a good static analysis tool. I of course did not +know that and tried to write my own poor version of a parser that I ended up +throwing away two years later. + +Because of my lack of knowledge, everything after 2012 and before 2020 was +basically refactoring. I rewrote the core parts of Jedi probably like 5-10 +times. The last big rewrite (that I did twice) was the inclusion of +gradual typing and stubs. + +I learned during that time that it is crucial to have a good understanding of +your problem. Otherwise you just end up doing it again. I only wrote features +in the beginning and in the end. Everything else was bugfixing and refactoring. +However now I am really happy with the result. It works well, bugfixes can be +quick and is pretty much feature complete. + +-------- + +I will leave you with a small annectote that happend in 2012, if I remember +correctly. After I explained Guido van Rossum, how some parts of my +auto-completion work, he said: + + *"Oh, that worries me..."* + +Now that it is finished, I hope he likes it :-). + +.. _contributors: + +.. include:: ../../AUTHORS.txt diff --git a/docs/docs/api-classes.rst b/docs/docs/api-classes.rst index 9feec1c6..681a312b 100644 --- a/docs/docs/api-classes.rst +++ b/docs/docs/api-classes.rst @@ -5,6 +5,49 @@ API Return Classes ------------------ -.. automodule:: jedi.api.classes +Abstract Base Class +~~~~~~~~~~~~~~~~~~~ +.. autoclass:: jedi.api.classes.BaseName :members: - :undoc-members: + :show-inheritance: + +Name +~~~~ +.. autoclass:: jedi.api.classes.Name + :members: + :show-inheritance: + +Completion +~~~~~~~~~~ +.. autoclass:: jedi.api.classes.Completion + :members: + :show-inheritance: + +BaseSignature +~~~~~~~~~~~~~ +.. autoclass:: jedi.api.classes.BaseSignature + :members: + :show-inheritance: + +Signature +~~~~~~~~~ +.. autoclass:: jedi.api.classes.Signature + :members: + :show-inheritance: + +ParamName +~~~~~~~~~ +.. autoclass:: jedi.api.classes.ParamName + :members: + :show-inheritance: + +Refactoring +~~~~~~~~~~~ + +.. autoclass:: jedi.api.refactoring.Refactoring + :members: + :show-inheritance: + +.. autoclass:: jedi.api.errors.SyntaxError + :members: + :show-inheritance: diff --git a/docs/docs/api.rst b/docs/docs/api.rst index 111fdf26..3171ab32 100644 --- a/docs/docs/api.rst +++ b/docs/docs/api.rst @@ -3,56 +3,74 @@ API Overview ============ -.. currentmodule:: jedi - -Note: This documentation is for Plugin developers, who want to improve their -editors/IDE autocompletion - -If you want to use |jedi|, you first need to ``import jedi``. You then have -direct access to the :class:`.Script`. You can then call the functions -documented here. These functions return :ref:`API classes -`. - - -Deprecations ------------- - -The deprecation process is as follows: - -1. A deprecation is announced in the next major/minor release. -2. We wait either at least a year & at least two minor releases until we remove - the deprecated functionality. - - -API Documentation ------------------ - -The API consists of a few different parts: - -- The main starting points for complete/goto: :class:`.Script` and :class:`.Interpreter` -- Helpful functions: :func:`.preload_module` and :func:`.set_debug_function` -- :ref:`API Result Classes ` -- :ref:`Python Versions/Virtualenv Support ` with functions like - :func:`.find_system_environments` and :func:`.find_virtualenvs` +.. note:: This documentation is mostly for Plugin developers, who want to + improve their editors/IDE with Jedi. .. _api: -Static Analysis Interface -~~~~~~~~~~~~~~~~~~~~~~~~~ +The API consists of a few different parts: -.. automodule:: jedi +- The main starting points for complete/goto: :class:`.Script` and + :class:`.Interpreter`. If you work with Jedi you want to understand these + classes first. +- :ref:`API Result Classes ` +- :ref:`Python Versions/Virtualenv Support ` with functions like + :func:`.find_system_environments` and :func:`.find_virtualenvs` +- A way to work with different :ref:`Folders / Projects ` +- Helpful functions: :func:`.preload_module` and :func:`.set_debug_function` + +The methods that you are most likely going to use to work with Jedi are the +following ones: + +.. currentmodule:: jedi + +.. autosummary:: + :nosignatures: + + Script.complete + Script.goto + Script.infer + Script.help + Script.get_signatures + Script.get_references + Script.get_context + Script.get_names + Script.get_syntax_errors + Script.rename + Script.inline + Script.extract_variable + Script.extract_function + Script.search + Script.complete_search + Project.search + Project.complete_search + +Script +------ .. autoclass:: jedi.Script :members: + +Interpreter +----------- .. autoclass:: jedi.Interpreter :members: -.. autofunction:: jedi.preload_module -.. autofunction:: jedi.set_debug_function + +.. _projects: + +Projects +-------- + +.. automodule:: jedi.api.project + +.. autofunction:: jedi.get_default_project +.. autoclass:: jedi.Project + :members: .. _environments: Environments -~~~~~~~~~~~~ +------------ .. automodule:: jedi.api.environment @@ -65,18 +83,31 @@ Environments .. autoclass:: jedi.api.environment.Environment :members: +Helper Functions +---------------- + +.. autofunction:: jedi.preload_module +.. autofunction:: jedi.set_debug_function + +Errors +------ + +.. autoexception:: jedi.InternalError +.. autoexception:: jedi.RefactoringError + Examples -------- -Completions: +Completions +~~~~~~~~~~~ .. sourcecode:: python >>> import jedi - >>> source = '''import json; json.l''' - >>> script = jedi.Script(source, path='') + >>> code = '''import json; json.l''' + >>> script = jedi.Script(code, path='example.py') >>> script - + > >>> completions = script.complete(1, 19) >>> completions [, ] @@ -87,12 +118,14 @@ Completions: >>> completions[1].name 'loads' -Definitions / Goto: +Type Inference / Goto +~~~~~~~~~~~~~~~~~~~~~ .. sourcecode:: python >>> import jedi - >>> source = '''def my_func(): + >>> code = '''\ + ... def my_func(): ... print 'called' ... ... alias = my_func @@ -100,30 +133,42 @@ Definitions / Goto: ... inception = my_list[2] ... ... inception()''' - >>> script = jedi.Script(source, path='') + >>> script = jedi.Script(code) >>> >>> script.goto(8, 1) - [] + [] >>> >>> script.infer(8, 1) - [] + [] -References: +References +~~~~~~~~~~ .. sourcecode:: python >>> import jedi - >>> source = '''x = 3 + >>> code = '''\ + ... x = 3 ... if 1 == 2: ... x = 4 ... else: ... del x''' - >>> script = jedi.Script(source, '') + >>> script = jedi.Script(code) >>> rns = script.get_references(5, 8) >>> rns - [, - ] + [, + , + ] >>> rns[1].line - 5 - >>> rns[0].column - 8 + 3 + >>> rns[1].column + 4 + +Deprecations +------------ + +The deprecation process is as follows: + +1. A deprecation is announced in the next major/minor release. +2. We wait either at least a year and at least two minor releases until we + remove the deprecated functionality. diff --git a/docs/docs/changelog.rst b/docs/docs/changelog.rst new file mode 100644 index 00000000..09929fe4 --- /dev/null +++ b/docs/docs/changelog.rst @@ -0,0 +1 @@ +.. include:: ../../CHANGELOG.rst diff --git a/docs/docs/development.rst b/docs/docs/development.rst index d129ce68..ec981e8b 100644 --- a/docs/docs/development.rst +++ b/docs/docs/development.rst @@ -22,16 +22,12 @@ couldn't get rid of complexity. I know that **simple is better than complex**, but unfortunately it sometimes requires complex solutions to understand complex systems. -Since most of the Jedi internals have been written by me (David Halter), this -introduction will be written mostly by me, because no one else understands to -the same level how Jedi works. Actually this is also the reason for exactly this -part of the documentation. To make multiple people able to edit the Jedi core. - -In five chapters I'm trying to describe the internals of |jedi|: +In six chapters I'm trying to describe the internals of |jedi|: - :ref:`The Jedi Core ` - :ref:`Core Extensions ` - :ref:`Imports & Modules ` +- :ref:`Stubs & Annotations ` - :ref:`Caching & Recursions ` - :ref:`Helper modules ` @@ -59,17 +55,17 @@ because that's where all the magic happens. I need to introduce the :ref:`parser Parser ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Jedi used to have it's internal parser, however this is now a separate project +Jedi used to have its internal parser, however this is now a separate project and is called `parso `_. The parser creates a syntax tree that |jedi| analyses and tries to understand. -The grammar that this parsers uses is very similar to the official Python +The grammar that this parser uses is very similar to the official Python `grammar files `_. .. _inference: Type inference of python code (inference/__init__.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: jedi.inference @@ -80,7 +76,7 @@ Inference Values (inference/base_value.py) .. inheritance-diagram:: jedi.inference.value.instance.TreeInstance - jedi.inference.value.klass.Classvalue + jedi.inference.value.klass.ClassValue jedi.inference.value.function.FunctionValue jedi.inference.value.function.FunctionExecutionContext :parts: 1 @@ -89,7 +85,7 @@ Inference Values (inference/base_value.py) .. _name_resolution: Name resolution (inference/finder.py) -++++++++++++++++++++++++++++++++++++ ++++++++++++++++++++++++++++++++++++++ .. automodule:: jedi.inference.finder @@ -114,7 +110,7 @@ Core Extensions Core Extensions is a summary of the following topics: - :ref:`Iterables & Dynamic Arrays ` -- :ref:`Dynamic Parameters ` +- :ref:`Dynamic Parameters ` - :ref:`Docstrings ` - :ref:`Refactoring ` @@ -125,7 +121,7 @@ without some features. .. _iterables: Iterables & Dynamic Arrays (inference/value/iterable.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ To understand Python on a deeper level, |jedi| needs to understand some of the dynamic features of Python like lists that are filled after creation: @@ -133,33 +129,33 @@ dynamic features of Python like lists that are filled after creation: .. automodule:: jedi.inference.value.iterable -.. _dynamic: +.. _dynamic_params: -Parameter completion (inference/dynamic.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Parameter completion (inference/dynamic_params.py) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: jedi.inference.dynamic +.. automodule:: jedi.inference.dynamic_params .. _docstrings: Docstrings (inference/docstrings.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: jedi.inference.docstrings .. _refactoring: -Refactoring (inference/refactoring.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Refactoring (inference/api/refactoring.py) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. automodule:: jedi.refactoring +.. automodule:: jedi.api.refactoring .. _imports-modules: Imports & Modules -------------------- +----------------- - :ref:`Modules ` @@ -170,7 +166,7 @@ Imports & Modules .. _builtin: Compiled Modules (inference/compiled.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: jedi.inference.compiled @@ -178,10 +174,16 @@ Compiled Modules (inference/compiled.py) .. _imports: Imports (inference/imports.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: jedi.inference.imports +.. _stubs: + +Stubs & Annotations (inference/gradual) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. automodule:: jedi.inference.gradual .. _caching-recursions: @@ -210,13 +212,8 @@ Recursions (recursion.py) .. _dev-helpers: Helper Modules ---------------- +-------------- Most other modules are not really central to how Jedi works. They all contain relevant code, but you if you understand the modules above, you pretty much understand Jedi. - -Python 2/3 compatibility (_compatibility.py) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. automodule:: jedi._compatibility diff --git a/docs/docs/features.rst b/docs/docs/features.rst index 617b7408..c77418db 100644 --- a/docs/docs/features.rst +++ b/docs/docs/features.rst @@ -1,29 +1,30 @@ .. include:: ../global.rst -Features and Caveats -==================== +Features and Limitations +======================== -Jedi obviously supports autocompletion. It's also possible to get it working in -(:ref:`your REPL (IPython, etc.) `). +Jedi's main API calls and features are: -Static analysis is also possible by using ``jedi.Script(...).get_names``. +- Autocompletion: :meth:`.Script.complete`; It's also possible to get it + working in :ref:`your REPL (IPython, etc.) ` +- Goto/Type Inference: :meth:`.Script.goto` and :meth:`.Script.infer` +- Static Analysis: :meth:`.Script.get_names` and :meth:`.Script.get_syntax_errors` +- Refactorings: :meth:`.Script.rename`, :meth:`.Script.inline`, + :meth:`.Script.extract_variable` and :meth:`.Script.extract_function` +- Code Search: :meth:`.Script.search` and :meth:`.Project.search` -Jedi would in theory support refactoring, but we have never publicized it, -because it's not production ready. If you're interested in helping out here, -let me know. With the latest parser changes, it should be very easy to actually -make it work. +Basic Features +-------------- - -General Features ----------------- - -- Python 2.7 and 3.4+ support +- Python 2.7 and 3.5+ support - Ignores syntax errors and wrong indentation - Can deal with complex module / function / class structures -- Great Virtualenv support -- Can infer function arguments from sphinx, epydoc and basic numpydoc docstrings, - and PEP0484-style type hints (:ref:`type hinting `) -- Stub files +- Great ``virtualenv``/``venv`` support +- Works great with Python's :ref:`type hinting `, +- Understands stub files +- Can infer function arguments for sphinx, epydoc and basic numpydoc docstrings +- Is overall a very solid piece of software that has been refined for a long + time. Bug reports are very welcome and are usually fixed within a few weeks. Supported Python Features @@ -38,7 +39,7 @@ Supported Python Features - ``*args`` / ``**kwargs`` - decorators / lambdas / closures - generators / iterators -- some descriptors: property / staticmethod / classmethod +- descriptors: property / staticmethod / classmethod / custom descriptors - some magic methods: ``__call__``, ``__iter__``, ``__next__``, ``__get__``, ``__getitem__``, ``__init__`` - ``list.append()``, ``set.add()``, ``list.extend()``, etc. @@ -46,191 +47,64 @@ Supported Python Features - relative imports - ``getattr()`` / ``__getattr__`` / ``__getattribute__`` - function annotations -- class decorators (py3k feature, are being ignored too, until I find a use - case, that doesn't work with |jedi|) -- simple/usual ``sys.path`` modifications +- simple/typical ``sys.path`` modifications - ``isinstance`` checks for if/while/assert - namespace packages (includes ``pkgutil``, ``pkg_resources`` and PEP420 namespaces) - Django / Flask / Buildout support - Understands Pytest fixtures -Not Supported -------------- +Limitations +----------- -Not yet implemented: +In general Jedi's limit are quite high, but for very big projects or very +complex code, sometimes Jedi intentionally stops type inference, to avoid +hanging for a long time. -- manipulations of instances outside the instance variables without using - methods +Additionally there are some Python patterns Jedi does not support. This is +intentional and below should be a complete list: -Will probably never be implemented: - -- metaclasses (how could an auto-completion ever support this) +- Arbitrary metaclasses: Some metaclasses like enums and dataclasses are + reimplemented in Jedi to make them work. Most of the time stubs are good + enough to get type inference working, even when metaclasses are involved. - ``setattr()``, ``__import__()`` -- writing to some dicts: ``globals()``, ``locals()``, ``object.__dict__`` +- Writing to some dicts: ``globals()``, ``locals()``, ``object.__dict__`` +- Manipulations of instances outside the instance variables without using + methods - -Caveats -------- - -**Slow Performance** +Performance Issues +~~~~~~~~~~~~~~~~~~ Importing ``numpy`` can be quite slow sometimes, as well as loading the -builtins the first time. If you want to speed things up, you could write import -hooks in |jedi|, which preload stuff. However, once loaded, this is not a -problem anymore. The same is true for huge modules like ``PySide``, ``wx``, -etc. +builtins the first time. If you want to speed things up, you could preload +libriaries in |jedi|, with :func:`.preload_module`. However, once loaded, this +should not be a problem anymore. The same is true for huge modules like +``PySide``, ``wx``, ``tensorflow``, ``pandas``, etc. -**Security** +Jedi does not have a very good cache layer. This is probably the biggest and +only architectural `issue `_ in +Jedi. Unfortunately it is not easy to change that. Dave Halter is thinking +about rewriting Jedi in Rust, but it has taken Jedi more than 8 years to reach +version 1.0, a rewrite will probably also take years. -Security is an important issue for |jedi|. Therefore no Python code is -executed. As long as you write pure Python, everything is inferred -statically. But: If you use builtin modules (``c_builtin``) there is no other -option than to execute those modules. However: Execute isn't that critical (as -e.g. in pythoncomplete, which used to execute *every* import!), because it -means one import and no more. So basically the only dangerous thing is using -the import itself. If your ``c_builtin`` uses some strange initializations, it -might be dangerous. But if it does you're screwed anyways, because eventually -you're going to execute your code, which executes the import. +Security +-------- +For :class:`.Script` +~~~~~~~~~~~~~~~~~~~~ -Recipes -------- +Security is an important topic for |jedi|. By default, no code is executed +within Jedi. As long as you write pure Python, everything is inferred +statically. If you enable ``load_unsafe_extensions=True`` for your +:class:`.Project` and you use builtin modules (``c_builtin``) Jedi will execute +those modules. If you don't trust a code base, please do not enable that +option. It might lead to arbitrary code execution. -Here are some tips on how to use |jedi| efficiently. +For :class:`.Interpreter` +~~~~~~~~~~~~~~~~~~~~~~~~~ - -.. _type-hinting: - -Type Hinting -~~~~~~~~~~~~ - -If |jedi| cannot detect the type of a function argument correctly (due to the -dynamic nature of Python), you can help it by hinting the type using -one of the following docstring/annotation syntax styles: - -**PEP-0484 style** - -https://www.python.org/dev/peps/pep-0484/ - -function annotations - -:: - - def myfunction(node: ProgramNode, foo: str) -> None: - """Do something with a ``node``. - - """ - node.| # complete here - - -assignment, for-loop and with-statement type hints (all Python versions). -Note that the type hints must be on the same line as the statement - -:: - - x = foo() # type: int - x, y = 2, 3 # type: typing.Optional[int], typing.Union[int, str] # typing module is mostly supported - for key, value in foo.items(): # type: str, Employee # note that Employee must be in scope - pass - with foo() as f: # type: int - print(f + 3) - -Most of the features in PEP-0484 are supported including the typing module -(for Python < 3.5 you have to do ``pip install typing`` to use these), -and forward references. - -You can also use stub files. - - -**Sphinx style** - -http://www.sphinx-doc.org/en/stable/domains.html#info-field-lists - -:: - - def myfunction(node, foo): - """Do something with a ``node``. - - :type node: ProgramNode - :param str foo: foo parameter description - - """ - node.| # complete here - -**Epydoc** - -http://epydoc.sourceforge.net/manual-fields.html - -:: - - def myfunction(node): - """Do something with a ``node``. - - @type node: ProgramNode - - """ - node.| # complete here - -**Numpydoc** - -https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt - -In order to support the numpydoc format, you need to install the `numpydoc -`__ package. - -:: - - def foo(var1, var2, long_var_name='hi'): - r"""A one-line summary that does not use variable names or the - function name. - - ... - - Parameters - ---------- - var1 : array_like - Array_like means all those objects -- lists, nested lists, - etc. -- that can be converted to an array. We can also - refer to variables like `var1`. - var2 : int - The type above can either refer to an actual Python type - (e.g. ``int``), or describe the type of the variable in more - detail, e.g. ``(N,) ndarray`` or ``array_like``. - long_variable_name : {'hi', 'ho'}, optional - Choices in brackets, default first when optional. - - ... - - """ - var2.| # complete here - -A little history ----------------- - -The Star Wars Jedi are awesome. My Jedi software tries to imitate a little bit -of the precognition the Jedi have. There's even an awesome `scene -`_ of Monty Python Jedis :-). - -But actually the name hasn't so much to do with Star Wars. It's part of my -second name. - -After I explained Guido van Rossum, how some parts of my auto-completion work, -he said (we drank a beer or two): - - *"Oh, that worries me..."* - -When it's finished, I hope he'll like it :-) - -I actually started Jedi, because there were no good solutions available for VIM. -Most auto-completions just didn't work well. The only good solution was PyCharm. -But I like my good old VIM. Rope was never really intended to be an -auto-completion (and also I really hate project folders for my Python scripts). -It's more of a refactoring suite. So I decided to do my own version of a -completion, which would execute non-dangerous code. But I soon realized, that -this wouldn't work. So I built an extremely recursive thing which understands -many of Python's key features. - -By the way, I really tried to program it as understandable as possible. But I -think understanding it might need quite some time, because of its recursive -nature. +If you want security for :class:`.Interpreter`, ``do not`` use it. Jedi does +execute properties and in general is not very careful to avoid code execution. +This is intentional: Most people trust the code bases they have imported, +because at that point a malicious code base would have had code execution +already. diff --git a/docs/docs/installation.rst b/docs/docs/installation.rst index e3e16e98..46e5ba90 100644 --- a/docs/docs/installation.rst +++ b/docs/docs/installation.rst @@ -3,6 +3,15 @@ Installation and Configuration ============================== +.. warning:: Most people will want to install Jedi as a submodule/vendored and + not through pip/system wide. The reason for this is that it makes sense that + the plugin that uses Jedi has always access to it. Otherwise Jedi will not + work properly when virtualenvs are activated. So please read the + documentation of your editor/IDE plugin to install Jedi. + + For plugin developers, Jedi works best if it is always available. Vendoring + is a pretty good option for that. + You can either include |jedi| as a submodule in your text editor plugin (like jedi-vim_ does by default), or you can install it systemwide. diff --git a/docs/docs/testing.rst b/docs/docs/testing.rst index 0c666a13..fdb36658 100644 --- a/docs/docs/testing.rst +++ b/docs/docs/testing.rst @@ -3,18 +3,14 @@ Jedi Testing ============ -The test suite depends on ``tox`` and ``pytest``:: +The test suite depends on ``pytest``:: - pip install tox pytest + pip install pytest -To run the tests for all supported Python versions:: - - tox - -If you want to test only a specific Python version (e.g. Python 2.7), it's as +If you want to test only a specific Python version (e.g. Python 3.8), it is as easy as:: - tox -e py27 + python3.8 -m pytest Tests are also run automatically on `Travis CI `_. @@ -28,8 +24,8 @@ simple and readable testing structure. .. _blackbox: -Blackbox Tests (run.py) -~~~~~~~~~~~~~~~~~~~~~~~ +Integration Tests (run.py) +~~~~~~~~~~~~~~~~~~~~~~~~~~ .. automodule:: test.run diff --git a/docs/docs/usage.rst b/docs/docs/usage.rst index 3ebbb752..b148ad76 100644 --- a/docs/docs/usage.rst +++ b/docs/docs/usage.rst @@ -1,15 +1,12 @@ .. include:: ../global.rst -End User Usage -============== +Using Jedi +========== -If you are a not an IDE Developer, the odds are that you just want to use -|jedi| as a browser plugin or in the shell. Yes that's :ref:`also possible -`! +|jedi| is can be used with a variety of plugins and software. It is also possible +to use |jedi| in the :ref:`Python shell or with IPython `. -|jedi| is relatively young and can be used in a variety of Plugins and -Software. If your Editor/IDE is not among them, recommend |jedi| to your IDE -developers. +Below you can also find a list of :ref:`recipes for type hinting `. .. _editor-plugins: @@ -17,60 +14,72 @@ developers. Editor Plugins -------------- -Vim: +Vim +~~~ - jedi-vim_ - YouCompleteMe_ - deoplete-jedi_ -Visual Studio Code: +Visual Studio Code +~~~~~~~~~~~~~~~~~~ - `Python Extension`_ -Emacs: +Emacs +~~~~~ - Jedi.el_ - elpy_ - anaconda-mode_ -Sublime Text 2/3: +Sublime Text 2/3 +~~~~~~~~~~~~~~~~ - SublimeJEDI_ (ST2 & ST3) - anaconda_ (only ST3) -SynWrite: +SynWrite +~~~~~~~~ - SynJedi_ -TextMate: +TextMate +~~~~~~~~ - Textmate_ (Not sure if it's actually working) -Kate: +Kate +~~~~ - Kate_ version 4.13+ `supports it natively `__, you have to enable it, though. -Atom: +Atom +~~~~ - autocomplete-python-jedi_ -GNOME Builder: +GNOME Builder +~~~~~~~~~~~~~ - `GNOME Builder`_ `supports it natively `__, and is enabled by default. -Gedit: +Gedit +~~~~~ - gedi_ -Eric IDE: +Eric IDE +~~~~~~~~ - `Eric IDE`_ (Available as a plugin) -Web Debugger: +Web Debugger +~~~~~~~~~~~~ - wdb_ @@ -81,11 +90,14 @@ and many more! Tab Completion in the Python Shell ---------------------------------- -Starting with Ipython `6.0.0` Jedi is a dependency of IPython. Autocompletion -in IPython is therefore possible without additional configuration. +Jedi is a dependency of IPython. Autocompletion in IPython is therefore +possible without additional configuration. + +Here is an `example video `_ how REPL completion +can look like in a different shell. There are two different options how you can use Jedi autocompletion in -your Python interpreter. One with your custom ``$HOME/.pythonrc.py`` file +your ``python`` interpreter. One with your custom ``$HOME/.pythonrc.py`` file and one that uses ``PYTHONSTARTUP``. Using ``PYTHONSTARTUP`` @@ -93,11 +105,137 @@ Using ``PYTHONSTARTUP`` .. automodule:: jedi.api.replstartup -Using a custom ``$HOME/.pythonrc.py`` +Using a Custom ``$HOME/.pythonrc.py`` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. autofunction:: jedi.utils.setup_readline +.. _recipes: + +Recipes +------- + +Here are some tips on how to use |jedi| efficiently. + + +.. _type-hinting: + +Type Hinting +~~~~~~~~~~~~ + +If |jedi| cannot detect the type of a function argument correctly (due to the +dynamic nature of Python), you can help it by hinting the type using +one of the docstring/annotation styles below. **Only gradual typing will +always work**, all the docstring solutions are glorified hacks and more +complicated cases will probably not work. + +Official Gradual Typing (Recommended) ++++++++++++++++++++++++++++++++++++++ + +You can read a lot about Python's gradual typing system in the corresponding +PEPs like: + +- `PEP 484 `_ as an introduction +- `PEP 526 `_ for variable annotations +- `PEP 589 `_ for ``TypeDict`` +- There are probably more :) + +Below you can find a few examples how you can use this feature. + +Function annotations:: + + def myfunction(node: ProgramNode, foo: str) -> None: + """Do something with a ``node``. + + """ + node.| # complete here + + +Assignment, for-loop and with-statement type hints:: + + import typing + x: int = foo() + y: typing.Optional[int] = 3 + + key: str + value: Employee + for key, value in foo.items(): + pass + + f: Union[int, float] + with foo() as f: + print(f + 3) + +PEP-0484 should be supported in its entirety. Feel free to open issues if that +is not the case. You can also use stub files. + + +Sphinx style +++++++++++++ + +http://www.sphinx-doc.org/en/stable/domains.html#info-field-lists + +:: + + def myfunction(node, foo): + """ + Do something with a ``node``. + + :type node: ProgramNode + :param str foo: foo parameter description + """ + node.| # complete here + +Epydoc +++++++ + +http://epydoc.sourceforge.net/manual-fields.html + +:: + + def myfunction(node): + """ + Do something with a ``node``. + + @type node: ProgramNode + """ + node.| # complete here + +Numpydoc +++++++++ + +https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt + +In order to support the numpydoc format, you need to install the `numpydoc +`__ package. + +:: + + def foo(var1, var2, long_var_name='hi'): + r""" + A one-line summary that does not use variable names or the + function name. + + ... + + Parameters + ---------- + var1 : array_like + Array_like means all those objects -- lists, nested lists, + etc. -- that can be converted to an array. We can also + refer to variables like `var1`. + var2 : int + The type above can either refer to an actual Python type + (e.g. ``int``), or describe the type of the variable in more + detail, e.g. ``(N,) ndarray`` or ``array_like``. + long_variable_name : {'hi', 'ho'}, optional + Choices in brackets, default first when optional. + + ... + + """ + var2.| # complete here + .. _jedi-vim: https://github.com/davidhalter/jedi-vim .. _youcompleteme: https://valloric.github.io/YouCompleteMe/ .. _deoplete-jedi: https://github.com/zchee/deoplete-jedi diff --git a/docs/global.rst b/docs/global.rst index 0279842a..c0c90b40 100644 --- a/docs/global.rst +++ b/docs/global.rst @@ -1,3 +1,3 @@ :orphan: -.. |jedi| replace:: *Jedi* +.. |jedi| replace:: Jedi diff --git a/docs/index.rst b/docs/index.rst index 57072abf..2a1adac0 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,13 +1,40 @@ .. include global.rst -Jedi - an awesome autocompletion/static analysis library for Python -=================================================================== +.. meta:: + :github_url: https://github.com/davidhalter/jedi -Release v\ |release|. (:doc:`Installation `) +Jedi - an awesome autocompletion, static analysis and refactoring library for Python +==================================================================================== + +.. image:: https://img.shields.io/github/stars/davidhalter/jedi.svg?style=social&label=Star&maxAge=2592000 + :target: https://github.com/davidhalter/jedi + :alt: GitHub stars + +.. image:: http://isitmaintained.com/badge/open/davidhalter/jedi.svg + :target: https://github.com/davidhalter/jedi/issues + :alt: The percentage of open issues and pull requests + +.. image:: http://isitmaintained.com/badge/resolution/davidhalter/jedi.svg + :target: https://github.com/davidhalter/jedi/issues + :alt: The resolution time is the median time an issue or pull request stays open. + +.. image:: https://travis-ci.org/davidhalter/jedi.svg?branch=master + :target: https://travis-ci.org/davidhalter/jedi + :alt: Linux Tests + +.. image:: https://ci.appveyor.com/api/projects/status/mgva3bbawyma1new/branch/master?svg=true + :target: https://ci.appveyor.com/project/davidhalter/jedi/branch/master + :alt: Windows Tests + +.. image:: https://coveralls.io/repos/davidhalter/jedi/badge.svg?branch=master + :target: https://coveralls.io/r/davidhalter/jedi + :alt: Coverage status + +`Github Repository `_ .. automodule:: jedi -Autocompletion can look like this (e.g. VIM plugin): +Autocompletion can for example look like this in jedi-vim: .. figure:: _screenshots/screenshot_complete.png @@ -18,16 +45,18 @@ Docs ---- .. toctree:: - :maxdepth: 2 + :maxdepth: 1 docs/usage - docs/installation docs/features docs/api docs/api-classes + docs/installation docs/settings docs/development docs/testing + docs/acknowledgements + docs/changelog .. _resources: diff --git a/jedi/__init__.py b/jedi/__init__.py index b35f3ec3..6824efe3 100644 --- a/jedi/__init__.py +++ b/jedi/__init__.py @@ -1,16 +1,13 @@ """ -Jedi is a static analysis tool for Python that can be used in IDEs/editors. -Jedi has a focus on autocompletion and goto functionality. Jedi is fast and is -very well tested. It understands Python and stubs on a deep level. +Jedi is a static analysis tool for Python that is typically used in +IDEs/editors plugins. Jedi has a focus on autocompletion and goto +functionality. Other features include refactoring, code search and finding +references. -Jedi has support for different goto functions. It's possible to search for -references and list names in a Python file to get information about them. - -Jedi uses a very simple API to connect with IDE's. There's a reference -implementation as a `VIM-Plugin `_, -which uses Jedi's autocompletion. We encourage you to use Jedi in your IDEs. -Autocompletion in your REPL is also possible, IPython uses it natively and for -the CPython REPL you have to install it. +Jedi has a simple API to work with. There is a reference implementation as a +`VIM-Plugin `_. Autocompletion in your +REPL is also possible, IPython uses it natively and for the CPython REPL you +can install it. Jedi is well tested and bugs should be rare. Here's a simple example of the autocompletion feature: @@ -28,12 +25,9 @@ Here's a simple example of the autocompletion feature: ad >>> print(completions[0].name) load - -As you see Jedi is pretty simple and allows you to concentrate on writing a -good text editor, while still having very good IDE features for Python. """ -__version__ = '0.16.1' +__version__ = '0.17.0' from jedi.api import Script, Interpreter, set_debug_function, \ preload_module, names @@ -42,7 +36,8 @@ from jedi.api.environment import find_virtualenvs, find_system_environments, \ get_default_environment, InvalidPythonEnvironment, create_environment, \ get_system_environment, InterpreterEnvironment from jedi.api.project import Project, get_default_project -from jedi.api.exceptions import InternalError +from jedi.api.exceptions import InternalError, RefactoringError + # Finally load the internal plugins. This is only internal. from jedi.plugins import registry del registry diff --git a/jedi/_compatibility.py b/jedi/_compatibility.py index d8950018..d7008941 100644 --- a/jedi/_compatibility.py +++ b/jedi/_compatibility.py @@ -11,7 +11,6 @@ import os import re import pkgutil import warnings -import inspect import subprocess import weakref try: @@ -219,65 +218,6 @@ if the module is contained in a package. """ -def _iter_modules(paths, prefix=''): - # Copy of pkgutil.iter_modules adapted to work with namespaces - - for path in paths: - importer = pkgutil.get_importer(path) - - if not isinstance(importer, importlib.machinery.FileFinder): - # We're only modifying the case for FileFinder. All the other cases - # still need to be checked (like zip-importing). Do this by just - # calling the pkgutil version. - for mod_info in pkgutil.iter_modules([path], prefix): - yield mod_info - continue - - # START COPY OF pkutils._iter_file_finder_modules. - if importer.path is None or not os.path.isdir(importer.path): - return - - yielded = {} - - try: - filenames = os.listdir(importer.path) - except OSError: - # ignore unreadable directories like import does - filenames = [] - filenames.sort() # handle packages before same-named modules - - for fn in filenames: - modname = inspect.getmodulename(fn) - if modname == '__init__' or modname in yielded: - continue - - # jedi addition: Avoid traversing special directories - if fn.startswith('.') or fn == '__pycache__': - continue - - path = os.path.join(importer.path, fn) - ispkg = False - - if not modname and os.path.isdir(path) and '.' not in fn: - modname = fn - # A few jedi modifications: Don't check if there's an - # __init__.py - try: - os.listdir(path) - except OSError: - # ignore unreadable directories like import does - continue - ispkg = True - - if modname and '.' not in modname: - yielded[modname] = 1 - yield importer, prefix + modname, ispkg - # END COPY - - -iter_modules = _iter_modules if py_version >= 34 else pkgutil.iter_modules - - class ImplicitNSInfo(object): """Stores information returned from an implicit namespace spec""" def __init__(self, name, paths): @@ -442,64 +382,6 @@ try: import cPickle as pickle except ImportError: import pickle -if sys.version_info[:2] == (3, 3): - """ - Monkeypatch the unpickler in Python 3.3. This is needed, because the - argument `encoding='bytes'` is not supported in 3.3, but badly needed to - communicate with Python 2. - """ - - class NewUnpickler(pickle._Unpickler): - dispatch = dict(pickle._Unpickler.dispatch) - - def _decode_string(self, value): - # Used to allow strings from Python 2 to be decoded either as - # bytes or Unicode strings. This should be used only with the - # STRING, BINSTRING and SHORT_BINSTRING opcodes. - if self.encoding == "bytes": - return value - else: - return value.decode(self.encoding, self.errors) - - def load_string(self): - data = self.readline()[:-1] - # Strip outermost quotes - if len(data) >= 2 and data[0] == data[-1] and data[0] in b'"\'': - data = data[1:-1] - else: - raise pickle.UnpicklingError("the STRING opcode argument must be quoted") - self.append(self._decode_string(pickle.codecs.escape_decode(data)[0])) - dispatch[pickle.STRING[0]] = load_string - - def load_binstring(self): - # Deprecated BINSTRING uses signed 32-bit length - len, = pickle.struct.unpack('`); + - Otherwise ``sys.path`` will match that of the default environment of + Jedi, which typically matches the sys path that was used at the time + when Jedi was imported. - - if `sys_path` parameter is ``None`` and ``VIRTUAL_ENV`` environment - variable is defined, ``sys.path`` for the specified environment will be - guessed (see :func:`jedi.inference.sys_path.get_venv_path`) and used for - the script; + Most methods have a ``line`` and a ``column`` parameter. Lines in Jedi are + always 1-based and columns are always zero based. To avoid repetition they + are not always documented. You can omit both line and column. Jedi will + then just do whatever action you are calling at the end of the file. If you + provide only the line, just will complete at the end of that line. - - otherwise ``sys.path`` will match that of |jedi|. + .. warning:: By default :attr:`jedi.settings.fast_parser` is enabled, which means + that parso reuses modules (i.e. they are not immutable). With this setting + Jedi is **not thread safe** and it is also not safe to use multiple + :class:`.Script` instances and its definitions at the same time. - :param source: The source code of the current file, separated by newlines. - :type source: str - :param line: Deprecated, please use it directly on e.g. `.complete` + If you are a normal plugin developer this should not be an issue. It is + an issue for people that do more complex stuff with Jedi. + + This is purely a performance optimization and works pretty well for all + typical usages, however consider to turn the setting of if it causes + you problems. See also + `this discussion `_. + + :param code: The source code of the current file, separated by newlines. + :type code: str + :param line: Deprecated, please use it directly on e.g. ``.complete`` :type line: int - :param column: Deprecated, please use it directly on e.g. `.complete` + :param column: Deprecated, please use it directly on e.g. ``.complete`` :type column: int :param path: The path of the file in the file system, or ``''`` if it hasn't been saved yet. :type path: str or None - :param encoding: The encoding of ``source``, if it is not a - ``unicode`` object (default ``'utf-8'``). + :param encoding: Deprecated, cast to unicode yourself. The encoding of + ``code``, if it is not a ``unicode`` object (default ``'utf-8'``). :type encoding: str - :param sys_path: ``sys.path`` to use during analysis of the script - :type sys_path: list - :param environment: TODO - :type environment: Environment + :param sys_path: Deprecated, use the project parameter. + :type sys_path: typing.List[str] + :param Environment environment: Provide a predefined :ref:`Environment ` + to work with a specific Python version or virtualenv. + :param Project project: Provide a :class:`.Project` to make sure finding + references works well, because the right folder is searched. There are + also ways to modify the sys path and other things. """ - def __init__(self, source=None, line=None, column=None, path=None, - encoding='utf-8', sys_path=None, environment=None, - project=None): + def __init__(self, code=None, line=None, column=None, path=None, + encoding=None, sys_path=None, environment=None, + project=None, source=None): self._orig_path = path # An empty path (also empty string) should always result in no path. self.path = os.path.abspath(path) if path else None - if source is None: - # TODO add a better warning than the traceback! - with open(path, 'rb') as f: - source = f.read() - - # Load the Python grammar of the current interpreter. - self._grammar = parso.load_grammar() - - if sys_path is not None and not is_py3: - sys_path = list(map(force_unicode, sys_path)) - - if project is None: - # Load the Python grammar of the current interpreter. - project = get_default_project( - os.path.dirname(self.path) if path else None - ) # TODO deprecate and remove sys_path from the Script API. if sys_path is not None: project._sys_path = sys_path @@ -118,12 +137,56 @@ class Script(object): DeprecationWarning, stacklevel=2 ) + if encoding is None: + encoding = 'utf-8' + else: + warnings.warn( + "Deprecated since version 0.17.0. You should cast to valid " + "unicode yourself, especially if you are not using utf-8.", + DeprecationWarning, + stacklevel=2 + ) + if line is not None: + warnings.warn( + "Providing the line is now done in the functions themselves " + "like `Script(...).complete(line, column)`", + DeprecationWarning, + stacklevel=2 + ) + if column is not None: + warnings.warn( + "Providing the column is now done in the functions themselves " + "like `Script(...).complete(line, column)`", + DeprecationWarning, + stacklevel=2 + ) + if source is not None: + code = source + warnings.warn( + "Use the code keyword argument instead.", + DeprecationWarning, + stacklevel=2 + ) + if code is None: + # TODO add a better warning than the traceback! + with open(path, 'rb') as f: + code = f.read() + + if sys_path is not None and not is_py3: + sys_path = list(map(force_unicode, sys_path)) + + if project is None: + # Load the Python grammar of the current interpreter. + project = get_default_project( + os.path.dirname(self.path) if path else None + ) + self._inference_state = InferenceState( project, environment=environment, script_path=self.path ) debug.speed('init') - self._module_node, source = self._inference_state.parse_and_get_code( - code=source, + self._module_node, code = self._inference_state.parse_and_get_code( + code=code, path=self.path, encoding=encoding, use_latest_grammar=path and path.endswith('.pyi'), @@ -132,8 +195,8 @@ class Script(object): cache_path=settings.cache_directory, ) debug.speed('parsed') - self._code_lines = parso.split_lines(source, keepends=True) - self._code = source + self._code_lines = parso.split_lines(code, keepends=True) + self._code = code self._pos = line, column cache.clear_time_caches() @@ -197,13 +260,17 @@ class Script(object): @validate_line_column def complete(self, line=None, column=None, **kwargs): """ - Return :class:`classes.Completion` objects. Those objects contain - information about the completions, more than just names. + Completes objects under the cursor. + + Those objects contain information about the completions, more than just + names. :param fuzzy: Default False. Will return fuzzy completions, which means that e.g. ``ooa`` will match ``foobar``. - :return: Completion objects, sorted by name and ``__`` comes last. - :rtype: list of :class:`classes.Completion` + :return: Completion objects, sorted by name. Normal names appear + before "private" names that start with ``_`` and those appear + before magic methods and name mangled names that start with ``__``. + :rtype: list of :class:`.Completion` """ return self._complete(line, column, **kwargs) @@ -216,30 +283,39 @@ class Script(object): return completion.complete() def completions(self, fuzzy=False): - # Deprecated, will be removed. + warnings.warn( + "Deprecated since version 0.16.0. Use Script(...).complete instead.", + DeprecationWarning, + stacklevel=2 + ) return self.complete(*self._pos, fuzzy=fuzzy) @validate_line_column def infer(self, line=None, column=None, **kwargs): """ - Return the definitions of a the path under the cursor. goto function! - This follows complicated paths and returns the end, not the first - definition. The big difference between :meth:`goto` and + Return the definitions of under the cursor. It is basically a wrapper + around Jedi's type inference. + + This method follows complicated paths and returns the end, not the + first definition. The big difference between :meth:`goto` and :meth:`infer` is that :meth:`goto` doesn't follow imports and statements. Multiple objects may be returned, - because Python itself is a dynamic language, which means depending on - an option you can have two different versions of a function. + because depending on an option you can have two different versions of a + function. - :param only_stubs: Only return stubs for this goto call. - :param prefer_stubs: Prefer stubs to Python objects for this type - inference call. - :rtype: list of :class:`classes.Definition` + :param only_stubs: Only return stubs for this method. + :param prefer_stubs: Prefer stubs to Python objects for this method. + :rtype: list of :class:`.Name` """ with debug.increase_indent_cm('infer'): return self._infer(line, column, **kwargs) def goto_definitions(self, **kwargs): - # Deprecated, will be removed. + warnings.warn( + "Deprecated since version 0.16.0. Use Script(...).infer instead.", + DeprecationWarning, + stacklevel=2 + ) return self.infer(*self._pos, **kwargs) def _infer(self, line, column, only_stubs=False, prefer_stubs=False): @@ -259,14 +335,18 @@ class Script(object): prefer_stubs=prefer_stubs, ) - defs = [classes.Definition(self._inference_state, c.name) for c in values] + defs = [classes.Name(self._inference_state, c.name) for c in values] # The additional set here allows the definitions to become unique in an # API sense. In the internals we want to separate more things than in # the API. return helpers.sorted_definitions(set(defs)) def goto_assignments(self, follow_imports=False, follow_builtin_imports=False, **kwargs): - # Deprecated, will be removed. + warnings.warn( + "Deprecated since version 0.16.0. Use Script(...).goto instead.", + DeprecationWarning, + stacklevel=2 + ) return self.goto(*self._pos, follow_imports=follow_imports, follow_builtin_imports=follow_builtin_imports, @@ -275,17 +355,17 @@ class Script(object): @validate_line_column def goto(self, line=None, column=None, **kwargs): """ - Return the first definition found, while optionally following imports. - Multiple objects may be returned, because Python itself is a - dynamic language, which means you can have two different versions of a - function. + Goes to the name that defined the object under the cursor. Optionally + you can follow imports. + Multiple objects may be returned, depending on an if you can have two + different versions of a function. - :param follow_imports: The goto call will follow imports. - :param follow_builtin_imports: If follow_imports is True will try to - look up names in builtins (i.e. compiled or extension modules). - :param only_stubs: Only return stubs for this goto call. - :param prefer_stubs: Prefer stubs to Python objects for this goto call. - :rtype: list of :class:`classes.Definition` + :param follow_imports: The method will follow imports. + :param follow_builtin_imports: If ``follow_imports`` is True will try + to look up names in builtins (i.e. compiled or extension modules). + :param only_stubs: Only return stubs for this method. + :param prefer_stubs: Prefer stubs to Python objects for this method. + :rtype: list of :class:`.Name` """ with debug.increase_indent_cm('goto'): return self._goto(line, column, **kwargs) @@ -323,48 +403,100 @@ class Script(object): prefer_stubs=prefer_stubs, ) - defs = [classes.Definition(self._inference_state, d) for d in set(names)] + defs = [classes.Name(self._inference_state, d) for d in set(names)] # Avoid duplicates return list(set(helpers.sorted_definitions(defs))) + @_no_python2_support + def search(self, string, **kwargs): + """ + Searches a name in the current file. For a description of how the + search string should look like, please have a look at + :meth:`.Project.search`. + + :param bool all_scopes: Default False; searches not only for + definitions on the top level of a module level, but also in + functions and classes. + :yields: :class:`.Name` + """ + return self._search(string, **kwargs) # Python 2 ... + + def _search(self, string, all_scopes=False): + return self._search_func(string, all_scopes=all_scopes) + + @to_list + def _search_func(self, string, all_scopes=False, complete=False, fuzzy=False): + names = self._names(all_scopes=all_scopes) + wanted_type, wanted_names = helpers.split_search_string(string) + return search_in_module( + self._inference_state, + self._get_module_context(), + names=names, + wanted_type=wanted_type, + wanted_names=wanted_names, + complete=complete, + fuzzy=fuzzy, + ) + + def complete_search(self, string, **kwargs): + """ + Like :meth:`.Script.search`, but completes that string. If you want to + have all possible definitions in a file you can also provide an empty + string. + + :param bool all_scopes: Default False; searches not only for + definitions on the top level of a module level, but also in + functions and classes. + :param fuzzy: Default False. Will return fuzzy completions, which means + that e.g. ``ooa`` will match ``foobar``. + :yields: :class:`.Completion` + """ + return self._search_func(string, complete=True, **kwargs) + @validate_line_column def help(self, line=None, column=None): """ - Works like goto and returns a list of Definition objects. Returns - additional definitions for keywords and operators. + Used to display a help window to users. Uses :meth:`.Script.goto` and + returns additional definitions for keywords and operators. - The additional definitions are of ``Definition(...).type == 'keyword'``. + Typically you will want to display :meth:`.BaseName.docstring` to the + user for all the returned definitions. + + The additional definitions are ``Name(...).type == 'keyword'``. These definitions do not have a lot of value apart from their docstring - attribute, which contains the output of Python's ``help()`` function. + attribute, which contains the output of Python's :func:`help` function. - :rtype: list of :class:`classes.Definition` + :rtype: list of :class:`.Name` """ definitions = self.goto(line, column, follow_imports=True) if definitions: return definitions leaf = self._module_node.get_leaf_for_position((line, column)) if leaf.type in ('keyword', 'operator', 'error_leaf'): - reserved = self._grammar._pgen_grammar.reserved_syntax_strings.keys() + reserved = self._inference_state.grammar._pgen_grammar.reserved_syntax_strings.keys() if leaf.value in reserved: name = KeywordName(self._inference_state, leaf.value) - return [classes.Definition(self._inference_state, name)] + return [classes.Name(self._inference_state, name)] return [] def usages(self, **kwargs): - # Deprecated, will be removed. + warnings.warn( + "Deprecated since version 0.16.0. Use Script(...).get_references instead.", + DeprecationWarning, + stacklevel=2 + ) return self.get_references(*self._pos, **kwargs) @validate_line_column def get_references(self, line=None, column=None, **kwargs): """ - Return :class:`classes.Definition` objects, which contain all - names that point to the definition of the name under the cursor. This - is very useful for refactoring (renaming), or to show all references of - a variable. + Lists all references of a variable in a project. Since this can be + quite hard to do for Jedi, if it is too complicated, Jedi will stop + searching. :param include_builtins: Default True, checks if a reference is a builtin (e.g. ``sys``) and in that case does not return it. - :rtype: list of :class:`classes.Definition` + :rtype: list of :class:`.Name` """ def _references(include_builtins=True): @@ -375,20 +507,24 @@ class Script(object): names = find_references(self._get_module_context(), tree_name) - definitions = [classes.Definition(self._inference_state, n) for n in names] + definitions = [classes.Name(self._inference_state, n) for n in names] if not include_builtins: definitions = [d for d in definitions if not d.in_builtin_module()] return helpers.sorted_definitions(definitions) return _references(**kwargs) def call_signatures(self): - # Deprecated, will be removed. + warnings.warn( + "Deprecated since version 0.16.0. Use Script(...).get_signatures instead.", + DeprecationWarning, + stacklevel=2 + ) return self.get_signatures(*self._pos) @validate_line_column def get_signatures(self, line=None, column=None): """ - Return the function object of the call you're currently in. + Return the function object of the call under the cursor. E.g. if the cursor is here:: @@ -400,7 +536,7 @@ class Script(object): This would return an empty list.. - :rtype: list of :class:`classes.Signature` + :rtype: list of :class:`.Signature` """ pos = line, column call_details = helpers.get_signature_details(self._module_node, pos) @@ -424,6 +560,12 @@ class Script(object): @validate_line_column def get_context(self, line=None, column=None): + """ + Returns the scope context under the cursor. This basically means the + function, class or module where the cursor is at. + + :rtype: :class:`.Name` + """ pos = (line, column) leaf = self._module_node.get_leaf_for_position(pos, include_prefixes=True) if leaf.start_pos > pos or leaf.type == 'endmarker': @@ -446,7 +588,7 @@ class Script(object): while context.name is None: context = context.parent_context # comprehensions - definition = classes.Definition(self._inference_state, context.name) + definition = classes.Name(self._inference_state, context.name) while definition.type != 'module': name = definition._name # TODO private access tree_name = name.tree_name @@ -493,69 +635,197 @@ class Script(object): def get_names(self, **kwargs): """ - Returns a list of `Definition` objects, containing name parts. - This means you can call ``Definition.goto()`` and get the - reference of a name. + Returns names defined in the current file. - :param all_scopes: If True lists the names of all scopes instead of only - the module namespace. + :param all_scopes: If True lists the names of all scopes instead of + only the module namespace. :param definitions: If True lists the names that have been defined by a class, function or a statement (``a = b`` returns ``a``). :param references: If True lists all the names that are not listed by ``definitions=True``. E.g. ``a = b`` returns ``b``. + :rtype: list of :class:`.Name` """ - return self._names(**kwargs) # Python 2... + names = self._names(**kwargs) + return [classes.Name(self._inference_state, n) for n in names] def get_syntax_errors(self): - return parso_to_jedi_errors(self._grammar, self._module_node) + """ + Lists all syntax errors in the current file. + + :rtype: list of :class:`.SyntaxError` + """ + return parso_to_jedi_errors(self._inference_state.grammar, self._module_node) def _names(self, all_scopes=False, definitions=True, references=False): - def def_ref_filter(_def): - is_def = _def._name.tree_name.is_definition() - return definitions and is_def or references and not is_def - # Set line/column to a random position, because they don't matter. module_context = self._get_module_context() defs = [ - classes.Definition( - self._inference_state, - module_context.create_name(name) - ) for name in get_module_names(self._module_node, all_scopes) + module_context.create_name(name) + for name in helpers.get_module_names( + self._module_node, + all_scopes=all_scopes, + definitions=definitions, + references=references, + ) ] - return sorted(filter(def_ref_filter, defs), key=lambda x: (x.line, x.column)) + return sorted(defs, key=lambda x: x.start_pos) + + @_no_python2_support + def rename(self, line=None, column=None, **kwargs): + """ + Renames all references of the variable under the cursor. + + :param new_name: The variable under the cursor will be renamed to this + string. + :raises: :exc:`.RefactoringError` + :rtype: :class:`.Refactoring` + """ + return self._rename(line, column, **kwargs) + + def _rename(self, line, column, new_name): # Python 2... + definitions = self.get_references(line, column, include_builtins=False) + return refactoring.rename(self._inference_state, definitions, new_name) + + @_no_python2_support + @validate_line_column + def extract_variable(self, line=None, column=None, **kwargs): + """ + Moves an expression to a new statemenet. + + For example if you have the cursor on ``foo`` and provide a + ``new_name`` called ``bar``:: + + foo = 3.1 + x = int(foo + 1) + + the code above will become:: + + foo = 3.1 + bar = foo + 1 + x = int(bar) + + :param new_name: The expression under the cursor will be renamed to + this string. + :param int until_line: The the selection range ends at this line, when + omitted, Jedi will be clever and try to define the range itself. + :param int until_column: The the selection range ends at this column, when + omitted, Jedi will be clever and try to define the range itself. + :raises: :exc:`.RefactoringError` + :rtype: :class:`.Refactoring` + """ + return self._extract_variable(line, column, **kwargs) # Python 2... + + def _extract_variable(self, line, column, new_name, until_line=None, until_column=None): + if until_line is None and until_column is None: + until_pos = None + else: + if until_line is None: + until_line = line + if until_column is None: + until_column = len(self._code_lines[until_line - 1]) + until_pos = until_line, until_column + return extract_variable( + self._inference_state, self.path, self._module_node, + new_name, (line, column), until_pos + ) + + @_no_python2_support + def extract_function(self, line, column, **kwargs): + """ + Moves an expression to a new function. + + For example if you have the cursor on ``foo`` and provide a + ``new_name`` called ``bar``:: + + global_var = 3 + + def x(): + foo = 3.1 + x = int(foo + 1 + global_var) + + the code above will become:: + + global_var = 3 + + def bar(foo): + return foo + 1 + global_var + + def x(foo): + x = int(bar(foo)) + + :param new_name: The expression under the cursor will be replaced with + a function with this name. + :param int until_line: The the selection range ends at this line, when + omitted, Jedi will be clever and try to define the range itself. + :param int until_column: The the selection range ends at this column, when + omitted, Jedi will be clever and try to define the range itself. + :raises: :exc:`.RefactoringError` + :rtype: :class:`.Refactoring` + """ + return self._extract_function(line, column, **kwargs) # Python 2... + + def _extract_function(self, line, column, new_name, until_line=None, until_column=None): + if until_line is None and until_column is None: + until_pos = None + else: + if until_line is None: + until_line = line + if until_column is None: + until_column = len(self._code_lines[until_line - 1]) + until_pos = until_line, until_column + return extract_function( + self._inference_state, self.path, self._get_module_context(), + new_name, (line, column), until_pos + ) + + @_no_python2_support + def inline(self, line=None, column=None): + """ + Inlines a variable under the cursor. This is basically the opposite of + extracting a variable. For example with the cursor on bar:: + + foo = 3.1 + bar = foo + 1 + x = int(bar) + + the code above will become:: + + foo = 3.1 + x = int(foo + 1) + + :raises: :exc:`.RefactoringError` + :rtype: :class:`.Refactoring` + """ + names = [d._name for d in self.get_references(line, column, include_builtins=True)] + return refactoring.inline(self._inference_state, names) class Interpreter(Script): """ - Jedi API for Python REPLs. + Jedi's API for Python REPLs. - In addition to completion of simple attribute access, Jedi - supports code completion based on static code analysis. - Jedi can complete attributes of object which is not initialized - yet. + Implements all of the methods that are present in :class:`.Script` as well. + + In addition to completions that normal REPL completion does like + ``str.upper``, Jedi also supports code completion based on static code + analysis. For example Jedi will complete ``str().upper``. >>> from os.path import join >>> namespace = locals() >>> script = Interpreter('join("").up', [namespace]) >>> print(script.complete()[0].name) upper + + All keyword arguments are same as the arguments for :class:`.Script`. + + :param str code: Code to parse. + :type namespaces: typing.List[dict] + :param namespaces: A list of namespace dictionaries such as the one + returned by :func:`globals` and :func:`locals`. """ _allow_descriptor_getattr_default = True - def __init__(self, source, namespaces, **kwds): - """ - Parse `source` and mixin interpreted Python objects from `namespaces`. - - :type source: str - :arg source: Code to parse. - :type namespaces: list of dict - :arg namespaces: a list of namespace dictionaries such as the one - returned by :func:`locals`. - - Other optional arguments are same as the ones for :class:`Script`. - If `line` and `column` are None, they are assumed be at the end of - `source`. - """ + def __init__(self, code, namespaces, **kwds): try: namespaces = [dict(n) for n in namespaces] except Exception: @@ -568,7 +838,7 @@ class Interpreter(Script): if not isinstance(environment, InterpreterEnvironment): raise TypeError("The environment needs to be an InterpreterEnvironment subclass.") - super(Interpreter, self).__init__(source, environment=environment, + super(Interpreter, self).__init__(code, environment=environment, project=Project(os.getcwd()), **kwds) self.namespaces = namespaces self._inference_state.allow_descriptor_getattr = self._allow_descriptor_getattr_default @@ -605,7 +875,8 @@ def names(source=None, path=None, encoding='utf-8', all_scopes=False, def preload_module(*modules): """ Preloading modules tells Jedi to load a module now, instead of lazy parsing - of modules. Usful for IDEs, to control which modules to load on startup. + of modules. This can be useful for IDEs, to control which modules to load + on startup. :param modules: different module names, list of string. """ @@ -621,7 +892,7 @@ def set_debug_function(func_cb=debug.print_to_stdout, warnings=True, If you don't specify any arguments, debug messages will be printed to stdout. - :param func_cb: The callback function for debug messages, with n params. + :param func_cb: The callback function for debug messages. """ debug.debug_function = func_cb debug.enable_warning = warnings diff --git a/jedi/api/classes.py b/jedi/api/classes.py index a4978d1b..a57a1613 100644 --- a/jedi/api/classes.py +++ b/jedi/api/classes.py @@ -1,7 +1,17 @@ """ -The :mod:`jedi.api.classes` module contains the return classes of the API. -These classes are the much bigger part of the whole API, because they contain -the interesting information about completion and goto operations. +There are a couple of classes documented in here: + +- :class:`.BaseName` as an abstact base class for almost everything. +- :class:`.Name` used in a lot of places +- :class:`.Completion` for completions +- :class:`.BaseSignature` as a base class for signatures +- :class:`.Signature` for :meth:`.Script.get_signatures` only +- :class:`.ParamName` used for parameters of signatures +- :class:`.Refactoring` for refactorings +- :class:`.SyntaxError` for :meth:`.Script.get_syntax_errors` only + +These classes are the much biggest part of the API, because they contain +the interesting information about all operations. """ import re import sys @@ -32,18 +42,21 @@ def defined_names(inference_state, context): List sub-definitions (e.g., methods in class). :type scope: Scope - :rtype: list of Definition + :rtype: list of Name """ filter = next(context.get_filters()) names = [name for name in filter.values()] - return [Definition(inference_state, n) for n in _sort_names_by_start_pos(names)] + return [Name(inference_state, n) for n in _sort_names_by_start_pos(names)] def _values_to_definitions(values): - return [Definition(c.inference_state, c.name) for c in values] + return [Name(c.inference_state, c.name) for c in values] -class BaseDefinition(object): +class BaseName(object): + """ + The base class for all definitions, completions and signatures. + """ _mapping = { 'posixpath': 'os.path', 'riscospath': 'os.path', @@ -138,10 +151,10 @@ class BaseDefinition(object): >>> defs = sorted(defs, key=lambda d: d.line) >>> no_unicode_pprint(defs) # doctest: +NORMALIZE_WHITESPACE - [, - , - , - ] + [, + , + , + ] Finally, here is what you can get from :attr:`type`: @@ -176,7 +189,8 @@ class BaseDefinition(object): @property def module_name(self): """ - The module name. + The module name, a bit similar to what ``__name__`` is in a random + Python module. >>> from jedi import Script >>> source = 'import json' @@ -188,7 +202,9 @@ class BaseDefinition(object): return self._get_module_context().py__name__() def in_builtin_module(self): - """Whether this is a builtin module.""" + """ + Returns True, if this is a builtin module. + """ value = self._get_module_context().get_value() if isinstance(value, StubModuleValue): return any(v.is_compiled() for v in value.non_stub_value_set) @@ -265,7 +281,7 @@ class BaseDefinition(object): @property def description(self): """ - A description of the :class:`.Definition` object, which is heavily used + A description of the :class:`.Name` object, which is heavily used in testing. e.g. for ``isinstance`` it returns ``def isinstance``. Example: @@ -284,8 +300,8 @@ class BaseDefinition(object): >>> defs = script.infer(column=3) >>> defs = sorted(defs, key=lambda d: d.line) >>> no_unicode_pprint(defs) # doctest: +NORMALIZE_WHITESPACE - [, - ] + [, + ] >>> str(defs[0].description) # strip literals in python2 'def f' >>> str(defs[1].description) @@ -341,7 +357,7 @@ class BaseDefinition(object): names = self._name.get_qualified_names(include_module_names=True) if names is None: - return names + return None names = list(names) try: @@ -352,12 +368,37 @@ class BaseDefinition(object): return '.'.join(names) def is_stub(self): + """ + Returns True if the current name is defined in a stub file. + """ if not self._name.is_value_name: return False return self._name.get_root_context().is_stub() + def is_side_effect(self): + """ + Checks if a name is defined as ``self.foo = 3``. In case of self, this + function would return False, for foo it would return True. + """ + tree_name = self._name.tree_name + if tree_name is None: + return False + return tree_name.is_definition() and tree_name.parent.type == 'trailer' + def goto(self, **kwargs): + """ + Like :meth:`.Script.goto` (also supports the same params), but does it + for the current name. This is typically useful if you are using + something like :meth:`.Script.get_names()`. + + :param follow_imports: The goto call will follow imports. + :param follow_builtin_imports: If follow_imports is True will try to + look up names in builtins (i.e. compiled or extension modules). + :param only_stubs: Only return stubs for this goto call. + :param prefer_stubs: Prefer stubs to Python objects for this goto call. + :rtype: list of :class:`Name` + """ with debug.increase_indent_cm('goto for %s' % self._name): return self._goto(**kwargs) @@ -383,10 +424,26 @@ class BaseDefinition(object): only_stubs=only_stubs, prefer_stubs=prefer_stubs, ) - return [self if n == self._name else Definition(self._inference_state, n) + return [self if n == self._name else Name(self._inference_state, n) for n in names] def infer(self, **kwargs): # Python 2... + """ + Like :meth:`.Script.infer`, it can be useful to understand which type + the current name has. + + Return the actual definitions. I strongly recommend not using it for + your completions, because it might slow down |jedi|. If you want to + read only a few objects (<=20), it might be useful, especially to get + the original docstrings. The basic problem of this function is that it + follows all results. This means with 1000 completions (e.g. numpy), + it's just very, very slow. + + :param only_stubs: Only return stubs for this goto call. + :param prefer_stubs: Prefer stubs to Python objects for this type + inference call. + :rtype: list of :class:`Name` + """ with debug.increase_indent_cm('infer for %s' % self._name): return self._infer(**kwargs) @@ -406,18 +463,12 @@ class BaseDefinition(object): prefer_stubs=prefer_stubs, ) resulting_names = [c.name for c in values] - return [self if n == self._name else Definition(self._inference_state, n) + return [self if n == self._name else Name(self._inference_state, n) for n in resulting_names] @property @memoize_method def params(self): - """ - Deprecated! Will raise a warning soon. Use get_signatures()[...].params. - - Raises an ``AttributeError`` if the definition is not callable. - Otherwise returns a list of `Definition` that represents the params. - """ warnings.warn( "Deprecated since version 0.16.0. Use get_signatures()[...].params", DeprecationWarning, @@ -427,7 +478,7 @@ class BaseDefinition(object): # with overloading. for signature in self._get_signatures(): return [ - Definition(self._inference_state, n) + Name(self._inference_state, n) for n in signature.get_param_names(resolve_stars=True) ] @@ -438,6 +489,11 @@ class BaseDefinition(object): raise AttributeError('There are no params defined on this.') def parent(self): + """ + Returns the parent scope of this identifier. + + :rtype: Name + """ if not self._name.is_value_name: return None @@ -463,7 +519,7 @@ class BaseDefinition(object): # Happens for comprehension contexts context = context.parent_context - return Definition(self._inference_state, context.name) + return Name(self._inference_state, context.name) def __repr__(self): return "<%s %sname=%r, description=%r>" % ( @@ -505,12 +561,24 @@ class BaseDefinition(object): return [sig for name in names for sig in name.infer().get_signatures()] def get_signatures(self): + """ + Returns all potential signatures for a function or a class. Multiple + signatures are typical if you use Python stubs with ``@overload``. + + :rtype: list of :class:`BaseSignature` + """ return [ BaseSignature(self._inference_state, s) for s in self._get_signatures() ] def execute(self): + """ + Uses type inference to "execute" this identifier and returns the + executed objects. + + :rtype: list of :class:`Name` + """ return _values_to_definitions(self._name.infer().execute_with_values()) def get_type_hint(self): @@ -520,13 +588,15 @@ class BaseDefinition(object): This method might be quite slow, especially for functions. The problem is finding executions for those functions to return something like ``Callable[[int, str], str]``. + + :rtype: str """ return self._name.infer().get_type_hint() -class Completion(BaseDefinition): +class Completion(BaseName): """ - `Completion` objects are returned from :meth:`api.Script.complete`. They + ``Completion`` objects are returned from :meth:`.Script.complete`. They provide additional information about a completion. """ def __init__(self, inference_state, name, stack, like_name_length, @@ -564,15 +634,15 @@ class Completion(BaseDefinition): isinstan# <-- Cursor is here would return the string 'ce'. It also adds additional stuff, depending - on your `settings.py`. + on your ``settings.py``. Assuming the following function definition:: def foo(param=0): pass - completing ``foo(par`` would give a ``Completion`` which `complete` - would be `am=` + completing ``foo(par`` would give a ``Completion`` which ``complete`` + would be ``am=``. """ if self._is_fuzzy: return None @@ -581,7 +651,7 @@ class Completion(BaseDefinition): @property def name_with_symbols(self): """ - Similar to :attr:`name`, but like :attr:`name` returns also the + Similar to :attr:`.name`, but like :attr:`.name` returns also the symbols, for example assuming the following function definition:: def foo(param=0): @@ -594,6 +664,9 @@ class Completion(BaseDefinition): return self._complete(False) def docstring(self, raw=False, fast=True): + """ + Documentated under :meth:`BaseName.docstring`. + """ if self._like_name_length >= 3: # In this case we can just resolve the like name, because we # wouldn't load like > 100 Python modules anymore. @@ -629,6 +702,9 @@ class Completion(BaseDefinition): @property def type(self): + """ + Documentated under :meth:`BaseName.type`. + """ # Purely a speed optimization. if self._cached_name is not None: return completion_cache.get_type( @@ -642,45 +718,22 @@ class Completion(BaseDefinition): def __repr__(self): return '<%s: %s>' % (type(self).__name__, self._name.get_public_name()) - @memoize_method - def follow_definition(self): - """ - Deprecated! - Return the original definitions. I strongly recommend not using it for - your completions, because it might slow down |jedi|. If you want to - read only a few objects (<=20), it might be useful, especially to get - the original docstrings. The basic problem of this function is that it - follows all results. This means with 1000 completions (e.g. numpy), - it's just PITA-slow. - """ - warnings.warn( - "Deprecated since version 0.14.0. Use .infer.", - DeprecationWarning, - stacklevel=2 - ) - return self.infer() - - -class Definition(BaseDefinition): +class Name(BaseName): """ - *Definition* objects are returned from :meth:`api.Script.goto` - or :meth:`api.Script.infer`. + *Name* objects are returned from many different APIs including + :meth:`.Script.goto` or :meth:`.Script.infer`. """ def __init__(self, inference_state, definition): - super(Definition, self).__init__(inference_state, definition) + super(Name, self).__init__(inference_state, definition) @property def desc_with_module(self): - """ - In addition to the definition, also return the module. - - .. warning:: Don't use this function yet, its behaviour may change. If - you really need it, talk to me. - - .. todo:: Add full path. This function is should return a - `module.class.function` path. - """ + warnings.warn( + "Deprecated since version 0.17.0. No replacement for now, maybe .full_name helps", + DeprecationWarning, + stacklevel=2 + ) position = '' if self.in_builtin_module else '@%s' % self.line return "%s:%s%s" % (self.module_name, self.description, position) @@ -689,7 +742,7 @@ class Definition(BaseDefinition): """ List sub-definitions (e.g., methods in class). - :rtype: list of Definition + :rtype: list of :class:`Name` """ defs = self._name.infer() return sorted( @@ -707,16 +760,6 @@ class Definition(BaseDefinition): else: return self._name.tree_name.is_definition() - def is_side_effect(self): - """ - Checks if a name is defined as ``self.foo = 3``. In case of self, this - function would return False, for foo it would return True. - """ - tree_name = self._name.tree_name - if tree_name is None: - return False - return tree_name.is_definition() and tree_name.parent.type == 'trailer' - def __eq__(self, other): return self._name.start_pos == other._name.start_pos \ and self.module_path == other.module_path \ @@ -730,11 +773,10 @@ class Definition(BaseDefinition): return hash((self._name.start_pos, self.module_path, self.name, self._inference_state)) -class BaseSignature(Definition): +class BaseSignature(Name): """ - `BaseSignature` objects is the return value of `Script.function_definition`. - It knows what functions you are currently in. e.g. `isinstance(` would - return the `isinstance` function. without `(` it would return nothing. + These signatures are returned by :meth:`BaseName.get_signatures` + calls. """ def __init__(self, inference_state, signature): super(BaseSignature, self).__init__(inference_state, signature.name) @@ -743,21 +785,28 @@ class BaseSignature(Definition): @property def params(self): """ - :return list of ParamDefinition: + Returns definitions for all parameters that a signature defines. + This includes stuff like ``*args`` and ``**kwargs``. + + :rtype: list of :class:`.ParamName` """ - return [ParamDefinition(self._inference_state, n) + return [ParamName(self._inference_state, n) for n in self._signature.get_param_names(resolve_stars=True)] def to_string(self): + """ + Returns a text representation of the signature. This could for example + look like ``foo(bar, baz: int, **kwargs)``. + + :return str + """ return self._signature.to_string() class Signature(BaseSignature): """ - `Signature` objects is the return value of `Script.get_signatures`. - It knows what functions you are currently in. e.g. `isinstance(` would - return the `isinstance` function with its params. Without `(` it would - return nothing. + A full signature object is the return value of + :meth:`.Script.get_signatures`. """ def __init__(self, inference_state, signature, call_details): super(Signature, self).__init__(inference_state, signature) @@ -767,8 +816,10 @@ class Signature(BaseSignature): @property def index(self): """ - The Param index of the current call. + Returns the param index of the current cursor position. Returns None if the index cannot be found in the curent call. + + :rtype: int """ return self._call_details.calculate_index( self._signature.get_param_names(resolve_stars=True) @@ -777,8 +828,10 @@ class Signature(BaseSignature): @property def bracket_start(self): """ - The line/column of the bracket that is responsible for the last - function call. + Returns a line/column tuple of the bracket that is responsible for the + last function call. The first line is 1 and the first column 0. + + :rtype: int, int """ return self._call_details.bracket_leaf.start_pos @@ -790,32 +843,38 @@ class Signature(BaseSignature): ) -class ParamDefinition(Definition): +class ParamName(Name): def infer_default(self): """ - :return list of Definition: + Returns default values like the ``1`` of ``def foo(x=1):``. + + :rtype: list of :class:`.Name` """ return _values_to_definitions(self._name.infer_default()) def infer_annotation(self, **kwargs): """ - :return list of Definition: - - :param execute_annotation: If False, the values are not executed and - you get classes instead of instances. + :param execute_annotation: Default True; If False, values are not + executed and classes are returned instead of instances. + :rtype: list of :class:`.Name` """ return _values_to_definitions(self._name.infer_annotation(ignore_stars=True, **kwargs)) def to_string(self): + """ + Returns a simple representation of a param, like + ``f: Callable[..., Any]``. + + :rtype: :class:`str` + """ return self._name.to_string() @property def kind(self): """ - Returns an enum instance. Returns the same values as the builtin - :py:attr:`inspect.Parameter.kind`. + Returns an enum instance of :mod:`inspect`'s ``Parameter`` enum. - No support for Python < 3.4 anymore. + :rtype: :py:attr:`inspect.Parameter.kind` """ if sys.version_info < (3, 5): raise NotImplementedError( diff --git a/jedi/api/completion.py b/jedi/api/completion.py index f8853cdf..332cab0f 100644 --- a/jedi/api/completion.py +++ b/jedi/api/completion.py @@ -19,8 +19,8 @@ from jedi.inference.base_value import ValueSet from jedi.inference.helpers import infer_call_of_leaf, parse_dotted_names from jedi.inference.context import get_global_filters from jedi.inference.value import TreeInstance, ModuleValue -from jedi.inference.names import ParamNameWrapper -from jedi.inference.gradual.conversion import convert_values +from jedi.inference.names import ParamNameWrapper, SubModuleName +from jedi.inference.gradual.conversion import convert_values, convert_names from jedi.parser_utils import cut_value_at_position from jedi.plugins import plugin_manager @@ -48,11 +48,7 @@ def filter_names(inference_state, completion_names, stack, like_name, fuzzy, cac string = name.string_name if settings.case_insensitive_completion: string = string.lower() - if fuzzy: - match = helpers.fuzzy_match(string, like_name) - else: - match = helpers.start_match(string, like_name) - if match: + if helpers.match(string, like_name, fuzzy=fuzzy): new = classes.Completion( inference_state, name, @@ -135,7 +131,7 @@ class Completion: if string is not None and not prefixed_completions: prefixed_completions = list(complete_file_name( - self._inference_state, self._module_context, start_leaf, string, + self._inference_state, self._module_context, start_leaf, quote, string, self._like_name, self._signatures_callback, self._code_lines, self._original_position, self._fuzzy @@ -361,80 +357,7 @@ class Completion: def _complete_trailer_for_values(self, values): user_context = get_user_context(self._module_context, self._position) - completion_names = [] - for value in values: - for filter in value.get_filters(origin_scope=user_context.tree_node): - completion_names += filter.values() - - if not value.is_stub() and isinstance(value, TreeInstance): - completion_names += self._complete_getattr(value) - - python_values = convert_values(values) - for c in python_values: - if c not in values: - for filter in c.get_filters(origin_scope=user_context.tree_node): - completion_names += filter.values() - return completion_names - - def _complete_getattr(self, instance): - """ - A heuristic to make completion for proxy objects work. This is not - intended to work in all cases. It works exactly in this case: - - def __getattr__(self, name): - ... - return getattr(any_object, name) - - It is important that the return contains getattr directly, otherwise it - won't work anymore. It's really just a stupid heuristic. It will not - work if you write e.g. `return (getatr(o, name))`, because of the - additional parentheses. It will also not work if you move the getattr - to some other place that is not the return statement itself. - - It is intentional that it doesn't work in all cases. Generally it's - really hard to do even this case (as you can see below). Most people - will write it like this anyway and the other ones, well they are just - out of luck I guess :) ~dave. - """ - names = (instance.get_function_slot_names(u'__getattr__') - or instance.get_function_slot_names(u'__getattribute__')) - functions = ValueSet.from_sets( - name.infer() - for name in names - ) - for func in functions: - tree_node = func.tree_node - for return_stmt in tree_node.iter_return_stmts(): - # Basically until the next comment we just try to find out if a - # return statement looks exactly like `return getattr(x, name)`. - if return_stmt.type != 'return_stmt': - continue - atom_expr = return_stmt.children[1] - if atom_expr.type != 'atom_expr': - continue - atom = atom_expr.children[0] - trailer = atom_expr.children[1] - if len(atom_expr.children) != 2 or atom.type != 'name' \ - or atom.value != 'getattr': - continue - arglist = trailer.children[1] - if arglist.type != 'arglist' or len(arglist.children) < 3: - continue - context = func.as_context() - object_node = arglist.children[0] - - # Make sure it's a param: foo in __getattr__(self, foo) - name_node = arglist.children[2] - name_list = context.goto(name_node, name_node.start_pos) - if not any(n.api_type == 'param' for n in name_list): - continue - - # Now that we know that these are most probably completion - # objects, we just infer the object and return them as - # completions. - objects = context.infer_node(object_node) - return self._complete_trailer_for_values(objects) - return [] + return complete_trailer(user_context, values) def _get_importer_names(self, names, level=0, only_modules=True): names = [n.value for n in names] @@ -572,3 +495,123 @@ def _extract_string_while_in_string(leaf, position): leaves.insert(0, leaf) leaf = leaf.get_previous_leaf() return None, None, None + + +def complete_trailer(user_context, values): + completion_names = [] + for value in values: + for filter in value.get_filters(origin_scope=user_context.tree_node): + completion_names += filter.values() + + if not value.is_stub() and isinstance(value, TreeInstance): + completion_names += _complete_getattr(user_context, value) + + python_values = convert_values(values) + for c in python_values: + if c not in values: + for filter in c.get_filters(origin_scope=user_context.tree_node): + completion_names += filter.values() + return completion_names + + +def _complete_getattr(user_context, instance): + """ + A heuristic to make completion for proxy objects work. This is not + intended to work in all cases. It works exactly in this case: + + def __getattr__(self, name): + ... + return getattr(any_object, name) + + It is important that the return contains getattr directly, otherwise it + won't work anymore. It's really just a stupid heuristic. It will not + work if you write e.g. `return (getatr(o, name))`, because of the + additional parentheses. It will also not work if you move the getattr + to some other place that is not the return statement itself. + + It is intentional that it doesn't work in all cases. Generally it's + really hard to do even this case (as you can see below). Most people + will write it like this anyway and the other ones, well they are just + out of luck I guess :) ~dave. + """ + names = (instance.get_function_slot_names(u'__getattr__') + or instance.get_function_slot_names(u'__getattribute__')) + functions = ValueSet.from_sets( + name.infer() + for name in names + ) + for func in functions: + tree_node = func.tree_node + for return_stmt in tree_node.iter_return_stmts(): + # Basically until the next comment we just try to find out if a + # return statement looks exactly like `return getattr(x, name)`. + if return_stmt.type != 'return_stmt': + continue + atom_expr = return_stmt.children[1] + if atom_expr.type != 'atom_expr': + continue + atom = atom_expr.children[0] + trailer = atom_expr.children[1] + if len(atom_expr.children) != 2 or atom.type != 'name' \ + or atom.value != 'getattr': + continue + arglist = trailer.children[1] + if arglist.type != 'arglist' or len(arglist.children) < 3: + continue + context = func.as_context() + object_node = arglist.children[0] + + # Make sure it's a param: foo in __getattr__(self, foo) + name_node = arglist.children[2] + name_list = context.goto(name_node, name_node.start_pos) + if not any(n.api_type == 'param' for n in name_list): + continue + + # Now that we know that these are most probably completion + # objects, we just infer the object and return them as + # completions. + objects = context.infer_node(object_node) + return complete_trailer(user_context, objects) + return [] + + +def search_in_module(inference_state, module_context, names, wanted_names, + wanted_type, complete=False, fuzzy=False, + ignore_imports=False, convert=False): + for s in wanted_names[:-1]: + new_names = [] + for n in names: + if s == n.string_name: + if n.tree_name is not None and n.api_type == 'module' \ + and ignore_imports: + continue + new_names += complete_trailer( + module_context, + n.infer() + ) + debug.dbg('dot lookup on search %s from %s', new_names, names[:10]) + names = new_names + + last_name = wanted_names[-1].lower() + for n in names: + string = n.string_name.lower() + if complete and helpers.match(string, last_name, fuzzy=fuzzy) \ + or not complete and string == last_name: + if isinstance(n, SubModuleName): + names = [v.name for v in n.infer()] + else: + names = [n] + if convert: + names = convert_names(names) + for n2 in names: + if complete: + def_ = classes.Completion( + inference_state, n2, + stack=None, + like_name_length=len(last_name), + is_fuzzy=fuzzy, + ) + else: + def_ = classes.Name(inference_state, n2) + if not wanted_type or wanted_type == def_.type: + yield def_ diff --git a/jedi/api/environment.py b/jedi/api/environment.py index 99643418..3ab2bf80 100644 --- a/jedi/api/environment.py +++ b/jedi/api/environment.py @@ -17,7 +17,7 @@ import parso _VersionInfo = namedtuple('VersionInfo', 'major minor micro') -_SUPPORTED_PYTHONS = ['3.8', '3.7', '3.6', '3.5', '3.4', '2.7'] +_SUPPORTED_PYTHONS = ['3.8', '3.7', '3.6', '3.5', '2.7'] _SAFE_PATHS = ['/usr/bin', '/usr/local/bin'] _CONDA_VAR = 'CONDA_PREFIX' _CURRENT_VERSION = '%s.%s' % (sys.version_info.major, sys.version_info.minor) @@ -91,8 +91,8 @@ class Environment(_BaseEnvironment): """ self.version_info = _VersionInfo(*info[2]) """ - Like ``sys.version_info``. A tuple to show the current Environment's - Python version. + Like :data:`sys.version_info`: a tuple to show the current + Environment's Python version. """ # py2 sends bytes via pickle apparently?! @@ -117,7 +117,7 @@ class Environment(_BaseEnvironment): def get_sys_path(self): """ The sys path for this environment. Does not include potential - modifications like ``sys.path.append``. + modifications from e.g. appending to :data:`sys.path`. :returns: list of str """ @@ -185,7 +185,7 @@ def get_default_environment(): makes it possible to use as many new Python features as possible when using autocompletion and other functionality. - :returns: :class:`Environment` + :returns: :class:`.Environment` """ virtual_env = _get_virtual_env_from_var() if virtual_env is not None: @@ -272,7 +272,7 @@ def find_virtualenvs(paths=None, **kwargs): CONDA_PREFIX will be checked to see if it contains a valid conda environment. - :yields: :class:`Environment` + :yields: :class:`.Environment` """ def py27_comp(paths=None, safe=True, use_environment_vars=True): if paths is None: @@ -322,7 +322,7 @@ def find_system_environments(): The environments are sorted from latest to oldest Python version. - :yields: :class:`Environment` + :yields: :class:`.Environment` """ for version_string in _SUPPORTED_PYTHONS: try: @@ -339,7 +339,7 @@ def get_system_environment(version): where X and Y are the major and minor versions of Python. :raises: :exc:`.InvalidPythonEnvironment` - :returns: :class:`Environment` + :returns: :class:`.Environment` """ exe = which('python' + version) if exe: @@ -362,7 +362,7 @@ def create_environment(path, safe=True): Virtualenv path or an executable path. :raises: :exc:`.InvalidPythonEnvironment` - :returns: :class:`Environment` + :returns: :class:`.Environment` """ if os.path.isfile(path): _assert_safe(path, safe) diff --git a/jedi/api/errors.py b/jedi/api/errors.py index e86f9212..ea104b6f 100644 --- a/jedi/api/errors.py +++ b/jedi/api/errors.py @@ -9,23 +9,30 @@ def parso_to_jedi_errors(grammar, module_node): class SyntaxError(object): + """ + Syntax errors are generated by :meth:`.Script.get_syntax_errors`. + """ def __init__(self, parso_error): self._parso_error = parso_error @property def line(self): + """The line where the error starts (starting with 1).""" return self._parso_error.start_pos[0] @property def column(self): + """The column where the error starts (starting with 0).""" return self._parso_error.start_pos[1] @property def until_line(self): + """The line where the error ends (starting with 1).""" return self._parso_error.end_pos[0] @property def until_column(self): + """The column where the error ends (starting with 0).""" return self._parso_error.end_pos[1] def __repr__(self): diff --git a/jedi/api/exceptions.py b/jedi/api/exceptions.py index 99cebdb7..db66a5f4 100644 --- a/jedi/api/exceptions.py +++ b/jedi/api/exceptions.py @@ -3,8 +3,29 @@ class _JediError(Exception): class InternalError(_JediError): - pass + """ + This error might happen a subprocess is crashing. The reason for this is + usually broken C code in third party libraries. This is not a very common + thing and it is safe to use Jedi again. However using the same calls might + result in the same error again. + """ class WrongVersion(_JediError): - pass + """ + This error is reserved for the future, shouldn't really be happening at the + moment. + """ + + +class RefactoringError(_JediError): + """ + Refactorings can fail for various reasons. So if you work with refactorings + like :meth:`.Script.rename`, :meth:`.Script.inline`, + :meth:`.Script.extract_variable` and :meth:`.Script.extract_function`, make + sure to catch these. The descriptions in the errors are ususally valuable + for end users. + + A typical ``RefactoringError`` would tell the user that inlining is not + possible if no name is under the cursor. + """ diff --git a/jedi/api/file_name.py b/jedi/api/file_name.py index 1787613c..0a2161a3 100644 --- a/jedi/api/file_name.py +++ b/jedi/api/file_name.py @@ -3,7 +3,7 @@ import os from jedi._compatibility import FileNotFoundError, force_unicode, scandir from jedi.api import classes from jedi.api.strings import StringName, get_quote_ending -from jedi.api.helpers import fuzzy_match, start_match +from jedi.api.helpers import match from jedi.inference.helpers import get_str_or_none @@ -11,7 +11,7 @@ class PathName(StringName): api_type = u'path' -def complete_file_name(inference_state, module_context, start_leaf, string, +def complete_file_name(inference_state, module_context, start_leaf, quote, string, like_name, signatures_callback, code_lines, position, fuzzy): # First we want to find out what can actually be changed as a name. like_name_length = len(os.path.basename(string)) @@ -42,15 +42,12 @@ def complete_file_name(inference_state, module_context, start_leaf, string, # OSError: [Errno 36] File name too long: '...' except (FileNotFoundError, OSError): return + quote_ending = get_quote_ending(quote, code_lines, position) for entry in listed: name = entry.name - if fuzzy: - match = fuzzy_match(name, must_start_with) - else: - match = start_match(name, must_start_with) - if match: + if match(name, must_start_with, fuzzy=fuzzy): if is_in_os_path_join or not entry.is_dir(): - name += get_quote_ending(start_leaf.value, code_lines, position) + name += quote_ending else: name += os.path.sep diff --git a/jedi/api/helpers.py b/jedi/api/helpers.py index fbfbdb31..f06ab694 100644 --- a/jedi/api/helpers.py +++ b/jedi/api/helpers.py @@ -4,6 +4,7 @@ Helpers for the API import re from collections import namedtuple from textwrap import dedent +from itertools import chain from functools import wraps from parso.python.parser import Parser @@ -15,24 +16,32 @@ from jedi.inference.syntax_tree import infer_atom from jedi.inference.helpers import infer_call_of_leaf from jedi.inference.compiled import get_string_value_set from jedi.cache import signature_time_cache +from jedi.parser_utils import get_parent_scope CompletionParts = namedtuple('CompletionParts', ['path', 'has_dot', 'name']) -def start_match(string, like_name): +def _start_match(string, like_name): return string.startswith(like_name) -def fuzzy_match(string, like_name): +def _fuzzy_match(string, like_name): if len(like_name) <= 1: return like_name in string pos = string.find(like_name[0]) if pos >= 0: - return fuzzy_match(string[pos + 1:], like_name[1:]) + return _fuzzy_match(string[pos + 1:], like_name[1:]) return False +def match(string, like_name, fuzzy=False): + if fuzzy: + return _fuzzy_match(string, like_name) + else: + return _start_match(string, like_name) + + def sorted_definitions(defs): # Note: `or ''` below is required because `module_path` could be return sorted(defs, key=lambda x: (x.module_path or '', x.line or 0, x.column or 0, x.name)) @@ -455,3 +464,37 @@ def validate_line_column(func): column, line_len, line, line_string)) return func(self, line, column, *args, **kwargs) return wrapper + + +def get_module_names(module, all_scopes, definitions=True, references=False): + """ + Returns a dictionary with name parts as keys and their call paths as + values. + """ + def def_ref_filter(name): + is_def = name.is_definition() + return definitions and is_def or references and not is_def + + names = list(chain.from_iterable(module.get_used_names().values())) + if not all_scopes: + # We have to filter all the names that don't have the module as a + # parent_scope. There's None as a parent, because nodes in the module + # node have the parent module and not suite as all the others. + # Therefore it's important to catch that case. + + def is_module_scope_name(name): + parent_scope = get_parent_scope(name) + # async functions have an extra wrapper. Strip it. + if parent_scope and parent_scope.type == 'async_stmt': + parent_scope = parent_scope.parent + return parent_scope in (module, None) + + names = [n for n in names if is_module_scope_name(n)] + return filter(def_ref_filter, names) + + +def split_search_string(name): + type, _, dotted_names = name.rpartition(' ') + if type == 'def': + type = 'function' + return type, dotted_names.split('.') diff --git a/jedi/api/project.py b/jedi/api/project.py index 45e4c09a..b71a0d09 100644 --- a/jedi/api/project.py +++ b/jedi/api/project.py @@ -1,13 +1,31 @@ +""" +Projects are a way to handle Python projects within Jedi. For simpler plugins +you might not want to deal with projects, but if you want to give the user more +flexibility to define sys paths and Python interpreters for a project, +:class:`.Project` is the perfect way to allow for that. + +Projects can be saved to disk and loaded again, to allow project definitions to +be used across repositories. +""" import os import errno import json +import sys -from jedi._compatibility import FileNotFoundError, PermissionError, IsADirectoryError +from jedi._compatibility import FileNotFoundError, PermissionError, \ + IsADirectoryError +from jedi import debug from jedi.api.environment import get_cached_default_environment, create_environment from jedi.api.exceptions import WrongVersion +from jedi.api.completion import search_in_module +from jedi.api.helpers import split_search_string, get_module_names from jedi._compatibility import force_unicode +from jedi.inference.imports import load_module_from_path, \ + load_namespace_from_path, iter_module_names from jedi.inference.sys_path import discover_buildout_paths from jedi.inference.cache import inference_state_as_method_param_cache +from jedi.inference.references import recurse_find_python_folders_and_files, search_in_file_ios +from jedi.file_io import FolderIO from jedi.common.utils import traverse_parents _CONFIG_FOLDER = '.jedi' @@ -16,6 +34,23 @@ _CONTAINS_POTENTIAL_PROJECT = 'setup.py', '.git', '.hg', 'requirements.txt', 'MA _SERIALIZER_VERSION = 1 +def _try_to_skip_duplicates(func): + def wrapper(*args, **kwargs): + found_tree_nodes = [] + found_modules = [] + for definition in func(*args, **kwargs): + tree_node = definition._name.tree_name + if tree_node is not None and tree_node in found_tree_nodes: + continue + if definition.type == 'module' and definition.module_path is not None: + if definition.module_path in found_modules: + continue + found_modules.append(definition.module_path) + yield definition + found_tree_nodes.append(tree_node) + return wrapper + + def _remove_duplicates_from_path(path): used = set() for p in path: @@ -30,6 +65,11 @@ def _force_unicode_list(lst): class Project(object): + """ + Projects are a simple way to manage Python folders and define how Jedi does + import resolution. It is mostly used as a parameter to :class:`.Script`. + Additionally there are functions to search a whole project. + """ _environment = None @staticmethod @@ -43,6 +83,9 @@ class Project(object): @classmethod def load(cls, path): """ + Loads a project from a specific path. You should not provide the path + to ``.jedi/project.json``, but rather the path to the project folder. + :param path: The path of the directory you want to use as a project. """ with open(cls._get_json_path(path)) as f: @@ -55,18 +98,36 @@ class Project(object): "The Jedi version of this project seems newer than what we can handle." ) + def save(self): + """ + Saves the project configuration in the project in ``.jedi/project.json``. + """ + data = dict(self.__dict__) + data.pop('_environment', None) + data.pop('_django', None) # TODO make django setting public? + data = {k.lstrip('_'): v for k, v in data.items()} + + # TODO when dropping Python 2 use pathlib.Path.mkdir(parents=True, exist_ok=True) + try: + os.makedirs(self._get_config_folder_path(self._path)) + except OSError as e: + if e.errno != errno.EEXIST: + raise + with open(self._get_json_path(self._path), 'w') as f: + return json.dump((_SERIALIZER_VERSION, data), f) + def __init__(self, path, **kwargs): """ :param path: The base path for this project. :param python_path: The Python executable path, typically the path of a virtual environment. - :param load_unsafe_extensions: Loads extensions that are not in the + :param load_unsafe_extensions: Default False, Loads extensions that are not in the sys path and in the local directories. With this option enabled, this is potentially unsafe if you clone a git repository and analyze it's code, because those compiled extensions will be important and therefore have execution privileges. :param sys_path: list of str. You can override the sys path if you - want. By default the ``sys.path.`` is generated from the + want. By default the ``sys.path.`` is generated by the environment (virtualenvs, etc). :param added_sys_path: list of str. Adds these paths at the end of the sys path. @@ -124,7 +185,7 @@ class Project(object): # 2. Stopping immediately when above self._path traversed = [] for parent_path in traverse_parents(inference_state.script_path): - if not parent_path.startswith(self._path): + if parent_path == self._path or not parent_path.startswith(self._path): break if not add_init_paths \ and os.path.isfile(os.path.join(parent_path, "__init__.py")): @@ -142,21 +203,6 @@ class Project(object): path = prefixed + sys_path + suffixed return list(_force_unicode_list(_remove_duplicates_from_path(path))) - def save(self): - data = dict(self.__dict__) - data.pop('_environment', None) - data.pop('_django', None) # TODO make django setting public? - data = {k.lstrip('_'): v for k, v in data.items()} - - # TODO when dropping Python 2 use pathlib.Path.mkdir(parents=True, exist_ok=True) - try: - os.makedirs(self._get_config_folder_path(self._path)) - except OSError as e: - if e.errno != errno.EEXIST: - raise - with open(self._get_json_path(self._path), 'w') as f: - return json.dump((_SERIALIZER_VERSION, data), f) - def get_environment(self): if self._environment is None: if self._python_path is not None: @@ -165,6 +211,136 @@ class Project(object): self._environment = get_cached_default_environment() return self._environment + def search(self, string, **kwargs): + """ + Searches a name in the whole project. If the project is very big, + at some point Jedi will stop searching. However it's also very much + recommended to not exhaust the generator. Just display the first ten + results to the user. + + There are currently three different search patterns: + + - ``foo`` to search for a definition foo in any file or a file called + ``foo.py`` or ``foo.pyi``. + - ``foo.bar`` to search for the ``foo`` and then an attribute ``bar`` + in it. + - ``class foo.bar.Bar`` or ``def foo.bar.baz`` to search for a specific + API type. + + :param bool all_scopes: Default False; searches not only for + definitions on the top level of a module level, but also in + functions and classes. + :yields: :class:`.Name` + """ + return self._search(string, **kwargs) + + def complete_search(self, string, **kwargs): + """ + Like :meth:`.Script.search`, but completes that string. An empty string + lists all definitions in a project, so be careful with that. + + :param bool all_scopes: Default False; searches not only for + definitions on the top level of a module level, but also in + functions and classes. + :yields: :class:`.Completion` + """ + return self._search_func(string, complete=True, **kwargs) + + def _search(self, string, all_scopes=False): # Python 2.. + return self._search_func(string, all_scopes=all_scopes) + + @_try_to_skip_duplicates + def _search_func(self, string, complete=False, all_scopes=False): + # Using a Script is they easiest way to get an empty module context. + from jedi import Script + s = Script('', project=self) + inference_state = s._inference_state + empty_module_context = s._get_module_context() + + if inference_state.grammar.version_info < (3, 6) or sys.version_info < (3, 6): + raise NotImplementedError( + "No support for refactorings/search on Python 2/3.5" + ) + debug.dbg('Search for string %s, complete=%s', string, complete) + wanted_type, wanted_names = split_search_string(string) + name = wanted_names[0] + stub_folder_name = name + '-stubs' + + ios = recurse_find_python_folders_and_files(FolderIO(self._path)) + file_ios = [] + + # 1. Search for modules in the current project + for folder_io, file_io in ios: + if file_io is None: + file_name = folder_io.get_base_name() + if file_name == name or file_name == stub_folder_name: + f = folder_io.get_file_io('__init__.py') + try: + m = load_module_from_path(inference_state, f).as_context() + except FileNotFoundError: + f = folder_io.get_file_io('__init__.pyi') + try: + m = load_module_from_path(inference_state, f).as_context() + except FileNotFoundError: + m = load_namespace_from_path(inference_state, folder_io).as_context() + else: + continue + else: + file_ios.append(file_io) + file_name = os.path.basename(file_io.path) + if file_name in (name + '.py', name + '.pyi'): + m = load_module_from_path(inference_state, file_io).as_context() + else: + continue + + debug.dbg('Search of a specific module %s', m) + for x in search_in_module( + inference_state, + m, + names=[m.name], + wanted_type=wanted_type, + wanted_names=wanted_names, + complete=complete, + convert=True, + ignore_imports=True, + ): + yield x # Python 2... + + # 2. Search for identifiers in the project. + for module_context in search_in_file_ios(inference_state, file_ios, name): + names = get_module_names(module_context.tree_node, all_scopes=all_scopes) + names = [module_context.create_name(n) for n in names] + names = _remove_imports(names) + for x in search_in_module( + inference_state, + module_context, + names=names, + wanted_type=wanted_type, + wanted_names=wanted_names, + complete=complete, + ignore_imports=True, + ): + yield x # Python 2... + + # 3. Search for modules on sys.path + sys_path = [ + p for p in self._get_sys_path(inference_state) + # Exclude folders that are handled by recursing of the Python + # folders. + if not p.startswith(self._path) + ] + names = list(iter_module_names(inference_state, empty_module_context, sys_path)) + for x in search_in_module( + inference_state, + empty_module_context, + names=names, + wanted_type=wanted_type, + wanted_names=wanted_names, + complete=complete, + convert=True, + ): + yield x # Python 2... + def __repr__(self): return '<%s: %s>' % (self.__class__.__name__, self._path) @@ -188,6 +364,15 @@ def _is_django_path(directory): def get_default_project(path=None): + """ + If a project is not defined by the user, Jedi tries to define a project by + itself as well as possible. Jedi traverses folders until it finds one of + the following: + + 1. A ``.jedi/config.json`` + 2. One of the following files: ``setup.py``, ``.git``, ``.hg``, + ``requirements.txt`` and ``MANIFEST.in``. + """ if path is None: path = os.getcwd() @@ -225,3 +410,10 @@ def get_default_project(path=None): curdir = path if os.path.isdir(path) else os.path.dirname(path) return Project(curdir) + + +def _remove_imports(names): + return [ + n for n in names + if n.tree_name is None or n.api_type != 'module' + ] diff --git a/jedi/api/refactoring/__init__.py b/jedi/api/refactoring/__init__.py new file mode 100644 index 00000000..44052cbb --- /dev/null +++ b/jedi/api/refactoring/__init__.py @@ -0,0 +1,225 @@ +from os.path import dirname, basename, join, relpath +import os +import re +import difflib + +from parso import split_lines + +from jedi.api.exceptions import RefactoringError + +EXPRESSION_PARTS = ( + 'or_test and_test not_test comparison ' + 'expr xor_expr and_expr shift_expr arith_expr term factor power atom_expr' +).split() + + +class ChangedFile(object): + def __init__(self, inference_state, from_path, to_path, + module_node, node_to_str_map): + self._inference_state = inference_state + self._from_path = from_path + self._to_path = to_path + self._module_node = module_node + self._node_to_str_map = node_to_str_map + + def get_diff(self): + old_lines = split_lines(self._module_node.get_code(), keepends=True) + new_lines = split_lines(self.get_new_code(), keepends=True) + project_path = self._inference_state.project._path + diff = difflib.unified_diff( + old_lines, new_lines, + fromfile=relpath(self._from_path, project_path), + tofile=relpath(self._to_path, project_path), + ) + # Apparently there's a space at the end of the diff - for whatever + # reason. + return ''.join(diff).rstrip(' ') + + def get_new_code(self): + return self._inference_state.grammar.refactor(self._module_node, self._node_to_str_map) + + def apply(self): + if self._from_path is None: + raise RefactoringError( + 'Cannot apply a refactoring on a Script with path=None' + ) + + with open(self._from_path, 'w', newline='') as f: + f.write(self.get_new_code()) + + def __repr__(self): + return '<%s: %s>' % (self.__class__.__name__, self._from_path) + + +class Refactoring(object): + def __init__(self, inference_state, file_to_node_changes, renames=()): + self._inference_state = inference_state + self._renames = renames + self._file_to_node_changes = file_to_node_changes + + def get_changed_files(self): + """ + Returns a path to ``ChangedFile`` map. + """ + def calculate_to_path(p): + if p is None: + return p + for from_, to in renames: + if p.startswith(from_): + p = to + p[len(from_):] + return p + + renames = self.get_renames() + return { + path: ChangedFile( + self._inference_state, + from_path=path, + to_path=calculate_to_path(path), + module_node=next(iter(map_)).get_root_node(), + node_to_str_map=map_ + ) for path, map_ in sorted(self._file_to_node_changes.items()) + } + + def get_renames(self): + """ + Files can be renamed in a refactoring. + + Returns ``Iterable[Tuple[str, str]]``. + """ + return sorted(self._renames) + + def get_diff(self): + text = '' + project_path = self._inference_state.project._path + for from_, to in self.get_renames(): + text += 'rename from %s\nrename to %s\n' \ + % (relpath(from_, project_path), relpath(to, project_path)) + + return text + ''.join(f.get_diff() for f in self.get_changed_files().values()) + + def apply(self): + """ + Applies the whole refactoring to the files, which includes renames. + """ + for f in self.get_changed_files().values(): + f.apply() + + for old, new in self.get_renames(): + os.rename(old, new) + + +def _calculate_rename(path, new_name): + name = basename(path) + dir_ = dirname(path) + if name in ('__init__.py', '__init__.pyi'): + parent_dir = dirname(dir_) + return dir_, join(parent_dir, new_name) + ending = re.search(r'\.pyi?$', name).group(0) + return path, join(dir_, new_name + ending) + + +def rename(inference_state, definitions, new_name): + file_renames = set() + file_tree_name_map = {} + + if not definitions: + raise RefactoringError("There is no name under the cursor") + + for d in definitions: + tree_name = d._name.tree_name + if d.type == 'module' and tree_name is None: + file_renames.add(_calculate_rename(d.module_path, new_name)) + else: + # This private access is ok in a way. It's not public to + # protect Jedi users from seeing it. + if tree_name is not None: + fmap = file_tree_name_map.setdefault(d.module_path, {}) + fmap[tree_name] = tree_name.prefix + new_name + return Refactoring(inference_state, file_tree_name_map, file_renames) + + +def inline(inference_state, names): + if not names: + raise RefactoringError("There is no name under the cursor") + if any(n.api_type == 'module' for n in names): + raise RefactoringError("Cannot inline imports or modules") + if any(n.tree_name is None for n in names): + raise RefactoringError("Cannot inline builtins/extensions") + + definitions = [n for n in names if n.tree_name.is_definition()] + if len(definitions) == 0: + raise RefactoringError("No definition found to inline") + if len(definitions) > 1: + raise RefactoringError("Cannot inline a name with multiple definitions") + + tree_name = definitions[0].tree_name + + expr_stmt = tree_name.get_definition() + if expr_stmt.type != 'expr_stmt': + type_ = dict( + funcdef='function', + classdef='class', + ).get(expr_stmt.type, expr_stmt.type) + raise RefactoringError("Cannot inline a %s" % type_) + + if len(expr_stmt.get_defined_names(include_setitem=True)) > 1: + raise RefactoringError("Cannot inline a statement with multiple definitions") + first_child = expr_stmt.children[1] + if first_child.type == 'annassign' and len(first_child.children) == 4: + first_child = first_child.children[2] + if first_child != '=': + if first_child.type == 'annassign': + raise RefactoringError( + 'Cannot inline a statement that is defined by an annotation' + ) + else: + raise RefactoringError( + 'Cannot inline a statement with "%s"' + % first_child.get_code(include_prefix=False) + ) + + rhs = expr_stmt.get_rhs() + replace_code = rhs.get_code(include_prefix=False) + + references = [n for n in names if not n.tree_name.is_definition()] + file_to_node_changes = {} + for name in references: + tree_name = name.tree_name + path = name.get_root_context().py__file__() + s = replace_code + if rhs.type == 'testlist_star_expr' \ + or tree_name.parent.type in EXPRESSION_PARTS \ + or tree_name.parent.type == 'trailer' \ + and tree_name.parent.get_next_sibling() is not None: + s = '(' + replace_code + ')' + + of_path = file_to_node_changes.setdefault(path, {}) + + n = tree_name + prefix = n.prefix + par = n.parent + if par.type == 'trailer' and par.children[0] == '.': + prefix = par.parent.children[0].prefix + n = par + for some_node in par.parent.children[:par.parent.children.index(par)]: + of_path[some_node] = '' + of_path[n] = prefix + s + + path = definitions[0].get_root_context().py__file__() + changes = file_to_node_changes.setdefault(path, {}) + changes[expr_stmt] = _remove_indent_of_prefix(expr_stmt.get_first_leaf().prefix) + next_leaf = expr_stmt.get_next_leaf() + + # Most of the time we have to remove the newline at the end of the + # statement, but if there's a comment we might not need to. + if next_leaf.prefix.strip(' \t') == '' \ + and (next_leaf.type == 'newline' or next_leaf == ';'): + changes[next_leaf] = '' + return Refactoring(inference_state, file_to_node_changes) + + +def _remove_indent_of_prefix(prefix): + r""" + Removes the last indentation of a prefix, e.g. " \n \n " becomes " \n \n". + """ + return ''.join(split_lines(prefix, keepends=True)[:-1]) diff --git a/jedi/api/refactoring/extract.py b/jedi/api/refactoring/extract.py new file mode 100644 index 00000000..aa44b2c3 --- /dev/null +++ b/jedi/api/refactoring/extract.py @@ -0,0 +1,386 @@ +from textwrap import dedent + +from parso import split_lines + +from jedi import debug +from jedi.api.exceptions import RefactoringError +from jedi.api.refactoring import Refactoring, EXPRESSION_PARTS +from jedi.common.utils import indent_block +from jedi.parser_utils import function_is_classmethod, function_is_staticmethod + + +_EXTRACT_USE_PARENT = EXPRESSION_PARTS + ['trailer'] +_DEFINITION_SCOPES = ('suite', 'file_input') +_VARIABLE_EXCTRACTABLE = EXPRESSION_PARTS + \ + ('atom testlist_star_expr testlist test lambdef lambdef_nocond ' + 'keyword name number string fstring').split() + + +def extract_variable(inference_state, path, module_node, name, pos, until_pos): + nodes = _find_nodes(module_node, pos, until_pos) + debug.dbg('Extracting nodes: %s', nodes) + + is_expression, message = _is_expression_with_error(nodes) + if not is_expression: + raise RefactoringError(message) + + generated_code = name + ' = ' + _expression_nodes_to_string(nodes) + file_to_node_changes = {path: _replace(nodes, name, generated_code, pos)} + return Refactoring(inference_state, file_to_node_changes) + + +def _is_expression_with_error(nodes): + """ + Returns a tuple (is_expression, error_string). + """ + if any(node.type == 'name' and node.is_definition() for node in nodes): + return False, 'Cannot extract a name that defines something' + + if nodes[0].type not in _VARIABLE_EXCTRACTABLE: + return False, 'Cannot extract a "%s"' % nodes[0].type + return True, '' + + +def _find_nodes(module_node, pos, until_pos): + """ + Looks up a module and tries to find the appropriate amount of nodes that + are in there. + """ + start_node = module_node.get_leaf_for_position(pos, include_prefixes=True) + + if until_pos is None: + if start_node.type == 'operator': + next_leaf = start_node.get_next_leaf() + if next_leaf is not None and next_leaf.start_pos == pos: + start_node = next_leaf + + if _is_not_extractable_syntax(start_node): + start_node = start_node.parent + + while start_node.parent.type in _EXTRACT_USE_PARENT: + start_node = start_node.parent + + nodes = [start_node] + else: + # Get the next leaf if we are at the end of a leaf + if start_node.end_pos == pos: + next_leaf = start_node.get_next_leaf() + if next_leaf is not None: + start_node = next_leaf + + # Some syntax is not exactable, just use its parent + if _is_not_extractable_syntax(start_node): + start_node = start_node.parent + + # Find the end + end_leaf = module_node.get_leaf_for_position(until_pos, include_prefixes=True) + if end_leaf.start_pos > until_pos: + end_leaf = end_leaf.get_previous_leaf() + if end_leaf is None: + raise RefactoringError('Cannot extract anything from that') + + parent_node = start_node + while parent_node.end_pos < end_leaf.end_pos: + parent_node = parent_node.parent + + nodes = _remove_unwanted_expression_nodes(parent_node, pos, until_pos) + + # If the user marks just a return statement, we return the expression + # instead of the whole statement, because the user obviously wants to + # extract that part. + if len(nodes) == 1 and start_node.type in ('return_stmt', 'yield_expr'): + return [nodes[0].children[1]] + return nodes + + +def _replace(nodes, expression_replacement, extracted, pos, + insert_before_leaf=None, remaining_prefix=None): + # Now try to replace the nodes found with a variable and move the code + # before the current statement. + definition = _get_parent_definition(nodes[0]) + if insert_before_leaf is None: + insert_before_leaf = definition.get_first_leaf() + first_node_leaf = nodes[0].get_first_leaf() + + lines = split_lines(insert_before_leaf.prefix, keepends=True) + if first_node_leaf is insert_before_leaf: + if remaining_prefix is not None: + # The remaining prefix has already been calculated. + lines[:-1] = remaining_prefix + lines[-1:-1] = [indent_block(extracted, lines[-1]) + '\n'] + extracted_prefix = ''.join(lines) + + replacement_dct = {} + if first_node_leaf is insert_before_leaf: + replacement_dct[nodes[0]] = extracted_prefix + expression_replacement + else: + if remaining_prefix is None: + p = first_node_leaf.prefix + else: + p = remaining_prefix + _get_indentation(nodes[0]) + replacement_dct[nodes[0]] = p + expression_replacement + replacement_dct[insert_before_leaf] = extracted_prefix + insert_before_leaf.value + + for node in nodes[1:]: + replacement_dct[node] = '' + return replacement_dct + + +def _expression_nodes_to_string(nodes): + return ''.join(n.get_code(include_prefix=i != 0) for i, n in enumerate(nodes)) + + +def _suite_nodes_to_string(nodes, pos): + n = nodes[0] + prefix, part_of_code = _split_prefix_at(n.get_first_leaf(), pos[0] - 1) + code = part_of_code + n.get_code(include_prefix=False) \ + + ''.join(n.get_code() for n in nodes[1:]) + return prefix, code + + +def _split_prefix_at(leaf, until_line): + """ + Returns a tuple of the leaf's prefix, split at the until_line + position. + """ + # second means the second returned part + second_line_count = leaf.start_pos[0] - until_line + lines = split_lines(leaf.prefix, keepends=True) + return ''.join(lines[:-second_line_count]), ''.join(lines[-second_line_count:]) + + +def _get_indentation(node): + return split_lines(node.get_first_leaf().prefix)[-1] + + +def _get_parent_definition(node): + """ + Returns the statement where a node is defined. + """ + while node is not None: + if node.parent.type in _DEFINITION_SCOPES: + return node + node = node.parent + raise NotImplementedError('We should never even get here') + + +def _remove_unwanted_expression_nodes(parent_node, pos, until_pos): + """ + This function makes it so for `1 * 2 + 3` you can extract `2 + 3`, even + though it is not part of the expression. + """ + typ = parent_node.type + is_suite_part = typ in ('suite', 'file_input') + if typ in EXPRESSION_PARTS or is_suite_part: + nodes = parent_node.children + for i, n in enumerate(nodes): + if n.end_pos > pos: + start_index = i + if n.type == 'operator': + start_index -= 1 + break + for i, n in reversed(list(enumerate(nodes))): + if n.start_pos < until_pos: + end_index = i + if n.type == 'operator': + end_index += 1 + + # Something like `not foo or bar` should not be cut after not + for n in nodes[i:]: + if _is_not_extractable_syntax(n): + end_index += 1 + else: + break + break + nodes = nodes[start_index:end_index + 1] + if not is_suite_part: + nodes[0:1] = _remove_unwanted_expression_nodes(nodes[0], pos, until_pos) + nodes[-1:] = _remove_unwanted_expression_nodes(nodes[-1], pos, until_pos) + return nodes + return [parent_node] + + +def _is_not_extractable_syntax(node): + return node.type == 'operator' \ + or node.type == 'keyword' and node.value not in ('None', 'True', 'False') + + +def extract_function(inference_state, path, module_context, name, pos, until_pos): + nodes = _find_nodes(module_context.tree_node, pos, until_pos) + assert len(nodes) + + is_expression, _ = _is_expression_with_error(nodes) + context = module_context.create_context(nodes[0]) + is_bound_method = context.is_bound_method() + params, return_variables = list(_find_inputs_and_outputs(module_context, context, nodes)) + + # Find variables + # Is a class method / method + if context.is_module(): + insert_before_leaf = None # Leaf will be determined later + else: + node = _get_code_insertion_node(context.tree_node, is_bound_method) + insert_before_leaf = node.get_first_leaf() + if is_expression: + code_block = 'return ' + _expression_nodes_to_string(nodes) + '\n' + remaining_prefix = None + has_ending_return_stmt = False + else: + has_ending_return_stmt = _is_node_ending_return_stmt(nodes[-1]) + if not has_ending_return_stmt: + # Find the actually used variables (of the defined ones). If none are + # used (e.g. if the range covers the whole function), return the last + # defined variable. + return_variables = list(_find_needed_output_variables( + context, + nodes[0].parent, + nodes[-1].end_pos, + return_variables + )) or [return_variables[-1]] if return_variables else [] + + remaining_prefix, code_block = _suite_nodes_to_string(nodes, pos) + after_leaf = nodes[-1].get_next_leaf() + first, second = _split_prefix_at(after_leaf, until_pos[0]) + code_block += first + + code_block = dedent(code_block) + if not has_ending_return_stmt: + output_var_str = ', '.join(return_variables) + code_block += 'return ' + output_var_str + '\n' + + # Check if we have to raise RefactoringError + _check_for_non_extractables(nodes[:-1] if has_ending_return_stmt else nodes) + + decorator = '' + self_param = None + if is_bound_method: + if not function_is_staticmethod(context.tree_node): + function_param_names = context.get_value().get_param_names() + if len(function_param_names): + self_param = function_param_names[0].string_name + params = [p for p in params if p != self_param] + + if function_is_classmethod(context.tree_node): + decorator = '@classmethod\n' + else: + code_block += '\n' + + function_code = '%sdef %s(%s):\n%s' % ( + decorator, + name, + ', '.join(params if self_param is None else [self_param] + params), + indent_block(code_block) + ) + + function_call = '%s(%s)' % ( + ('' if self_param is None else self_param + '.') + name, + ', '.join(params) + ) + if is_expression: + replacement = function_call + else: + if has_ending_return_stmt: + replacement = 'return ' + function_call + '\n' + else: + replacement = output_var_str + ' = ' + function_call + '\n' + + replacement_dct = _replace(nodes, replacement, function_code, pos, + insert_before_leaf, remaining_prefix) + if not is_expression: + replacement_dct[after_leaf] = second + after_leaf.value + file_to_node_changes = {path: replacement_dct} + return Refactoring(inference_state, file_to_node_changes) + + +def _check_for_non_extractables(nodes): + for n in nodes: + try: + children = n.children + except AttributeError: + if n.value == 'return': + raise RefactoringError( + 'Can only extract return statements if they are at the end.') + if n.value == 'yield': + raise RefactoringError('Cannot extract yield statements.') + else: + _check_for_non_extractables(children) + + +def _is_name_input(module_context, names, first, last): + for name in names: + if name.api_type == 'param' or not name.parent_context.is_module(): + if name.get_root_context() is not module_context: + return True + if name.start_pos is None or not (first <= name.start_pos < last): + return True + return False + + +def _find_inputs_and_outputs(module_context, context, nodes): + first = nodes[0].start_pos + last = nodes[-1].end_pos + + inputs = [] + outputs = [] + for name in _find_non_global_names(nodes): + if name.is_definition(): + if name not in outputs: + outputs.append(name.value) + else: + if name.value not in inputs: + name_definitions = context.goto(name, name.start_pos) + if not name_definitions \ + or _is_name_input(module_context, name_definitions, first, last): + inputs.append(name.value) + + # Check if outputs are really needed: + return inputs, outputs + + +def _find_non_global_names(nodes): + for node in nodes: + try: + children = node.children + except AttributeError: + if node.type == 'name': + yield node + else: + # We only want to check foo in foo.bar + if node.type == 'trailer' and node.children[0] == '.': + continue + + for x in _find_non_global_names(children): # Python 2... + yield x + + +def _get_code_insertion_node(node, is_bound_method): + if not is_bound_method or function_is_staticmethod(node): + while node.parent.type != 'file_input': + node = node.parent + + while node.parent.type in ('async_funcdef', 'decorated', 'async_stmt'): + node = node.parent + return node + + +def _find_needed_output_variables(context, search_node, at_least_pos, return_variables): + """ + Searches everything after at_least_pos in a node and checks if any of the + return_variables are used in there and returns those. + """ + for node in search_node.children: + if node.start_pos < at_least_pos: + continue + + return_variables = set(return_variables) + for name in _find_non_global_names([node]): + if not name.is_definition() and name.value in return_variables: + return_variables.remove(name.value) + yield name.value + + +def _is_node_ending_return_stmt(node): + t = node.type + if t == 'simple_stmt': + return _is_node_ending_return_stmt(node.children[0]) + return t == 'return_stmt' diff --git a/jedi/api/strings.py b/jedi/api/strings.py index 42ad10ec..6653671e 100644 --- a/jedi/api/strings.py +++ b/jedi/api/strings.py @@ -93,17 +93,16 @@ def _get_string_prefix_and_quote(string): return match.group(1), match.group(2) -def _get_string_quote(string): - return _get_string_prefix_and_quote(string)[1] - - def _matches_quote_at_position(code_lines, quote, position): string = code_lines[position[0] - 1][position[1]:position[1] + len(quote)] return string == quote def get_quote_ending(string, code_lines, position, invert_result=False): - quote = _get_string_quote(string) + _, quote = _get_string_prefix_and_quote(string) + if quote is None: + return '' + # Add a quote only if it's not already there. if _matches_quote_at_position(code_lines, quote, position) != invert_result: return '' diff --git a/jedi/common/utils.py b/jedi/common/utils.py index bc71cafd..75b69299 100644 --- a/jedi/common/utils.py +++ b/jedi/common/utils.py @@ -24,3 +24,13 @@ def monkeypatch(obj, attribute_name, new_value): yield finally: setattr(obj, attribute_name, old_value) + + +def indent_block(text, indention=' '): + """This function indents a text block with a default of four spaces.""" + temp = '' + while text and text[-1] == '\n': + temp += text[-1] + text = text[:-1] + lines = text.split('\n') + return '\n'.join(map(lambda s: indention + s, lines)) + temp diff --git a/jedi/file_io.py b/jedi/file_io.py index 19a8246a..c4a5d24a 100644 --- a/jedi/file_io.py +++ b/jedi/file_io.py @@ -7,6 +7,9 @@ class AbstractFolderIO(object): def __init__(self, path): self.path = path + def get_base_name(self): + raise NotImplementedError + def list(self): raise NotImplementedError diff --git a/jedi/inference/compiled/access.py b/jedi/inference/compiled/access.py index 53f22c70..99bd843c 100644 --- a/jedi/inference/compiled/access.py +++ b/jedi/inference/compiled/access.py @@ -507,22 +507,6 @@ class DirectObjectAccess(object): obj = self._obj if py_version < 33: raise ValueError("inspect.signature was introduced in 3.3") - if py_version == 34: - # In 3.4 inspect.signature are wrong for str and int. This has - # been fixed in 3.5. The signature of object is returned, - # because no signature was found for str. Here we imitate 3.5 - # logic and just ignore the signature if the magic methods - # don't match object. - # 3.3 doesn't even have the logic and returns nothing for str - # and classes that inherit from object. - user_def = inspect._signature_get_user_defined_method - if (inspect.isclass(obj) - and not user_def(type(obj), '__init__') - and not user_def(type(obj), '__new__') - and (obj.__init__ != object.__init__ - or obj.__new__ != object.__new__)): - raise ValueError - try: return inspect.signature(obj) except (RuntimeError, TypeError): diff --git a/jedi/inference/compiled/subprocess/functions.py b/jedi/inference/compiled/subprocess/functions.py index 71749346..39181993 100644 --- a/jedi/inference/compiled/subprocess/functions.py +++ b/jedi/inference/compiled/subprocess/functions.py @@ -1,10 +1,13 @@ from __future__ import print_function import sys import os +import re +import inspect from jedi._compatibility import find_module, cast_path, force_unicode, \ - iter_modules, all_suffixes + all_suffixes, scandir from jedi.inference.compiled import access +from jedi import debug from jedi import parser_utils @@ -40,13 +43,6 @@ def get_module_info(inference_state, sys_path=None, full_name=None, **kwargs): sys.path = temp -def list_module_names(inference_state, search_path): - return [ - force_unicode(name) - for module_loader, name, is_pkg in iter_modules(search_path) - ] - - def get_builtin_module_names(inference_state): return list(map(force_unicode, sys.builtin_module_names)) @@ -84,3 +80,37 @@ def _get_init_path(directory_path): def safe_literal_eval(inference_state, value): return parser_utils.safe_literal_eval(value) + + +def iter_module_names(*args, **kwargs): + return list(_iter_module_names(*args, **kwargs)) + return sorted(set(_iter_module_names(*args, **kwargs))) + + +def _iter_module_names(inference_state, paths): + # Python modules/packages + for path in paths: + try: + dirs = scandir(path) + except OSError: + # The file might not exist or reading it might lead to an error. + debug.warning("Not possible to list directory: %s", path) + continue + for dir_entry in dirs: + name = dir_entry.name + # First Namespaces then modules/stubs + if dir_entry.is_dir(): + # pycache is obviously not an interestin namespace. Also the + # name must be a valid identifier. + # TODO use str.isidentifier, once Python 2 is removed + if name != '__pycache__' and not re.search(r'\W|^\d', name): + yield name + else: + if name.endswith('.pyi'): # Stub files + modname = name[:-4] + else: + modname = inspect.getmodulename(name) + + if modname and '.' not in modname: + if modname != '__init__': + yield modname diff --git a/jedi/inference/compiled/value.py b/jedi/inference/compiled/value.py index bad5c73f..5b5ca0a3 100644 --- a/jedi/inference/compiled/value.py +++ b/jedi/inference/compiled/value.py @@ -21,7 +21,7 @@ from jedi.inference.context import CompiledContext, CompiledModuleContext class CheckAttribute(object): - """Raises an AttributeError if the attribute X isn't available.""" + """Raises :exc:`AttributeError` if the attribute X is not available.""" def __init__(self, check_name=None): # Remove the py in front of e.g. py__call__. self.check_name = check_name diff --git a/jedi/inference/context.py b/jedi/inference/context.py index 8ecf4b77..b4e9d9bf 100644 --- a/jedi/inference/context.py +++ b/jedi/inference/context.py @@ -129,6 +129,9 @@ class AbstractContext(object): def is_compiled(self): return False + def is_bound_method(self): + return False + @abstractmethod def py__name__(self): raise NotImplementedError @@ -190,6 +193,9 @@ class ValueContext(AbstractContext): def is_compiled(self): return self._value.is_compiled() + def is_bound_method(self): + return self._value.is_bound_method() + def py__name__(self): return self._value.py__name__() @@ -347,6 +353,10 @@ class NamespaceContext(TreeContextMixin, ValueContext): def get_value(self): return self._value + @property + def string_names(self): + return self._value.string_names + def py__file__(self): return self._value.py__file__() diff --git a/jedi/inference/docstrings.py b/jedi/inference/docstrings.py index 6ff12cf3..5f0941e8 100644 --- a/jedi/inference/docstrings.py +++ b/jedi/inference/docstrings.py @@ -1,7 +1,7 @@ """ Docstrings are another source of information for functions and classes. -:mod:`jedi.inference.dynamic` tries to find all executions of functions, while -the docstring parsing is much easier. There are three different types of +:mod:`jedi.inference.dynamic_params` tries to find all executions of functions, +while the docstring parsing is much easier. There are three different types of docstrings that |jedi| understands: - `Sphinx `_ @@ -23,7 +23,7 @@ from parso import parse, ParserSyntaxError from jedi._compatibility import u from jedi import debug -from jedi.inference.utils import indent_block +from jedi.common.utils import indent_block from jedi.inference.cache import inference_state_method_cache from jedi.inference.base_value import iterator_to_value_set, ValueSet, \ NO_VALUES diff --git a/jedi/inference/gradual/__init__.py b/jedi/inference/gradual/__init__.py index e69de29b..5c86b7b3 100644 --- a/jedi/inference/gradual/__init__.py +++ b/jedi/inference/gradual/__init__.py @@ -0,0 +1,4 @@ +""" +It is unfortunately not well documented how stubs and annotations work in Jedi. +If somebody needs an introduction, please let me know. +""" diff --git a/jedi/inference/gradual/base.py b/jedi/inference/gradual/base.py index f9ccddb2..3cd8e509 100644 --- a/jedi/inference/gradual/base.py +++ b/jedi/inference/gradual/base.py @@ -300,6 +300,9 @@ class _PseudoTreeNameClass(Value): def name(self): return ValueName(self, self._tree_name) + def get_qualified_names(self): + return (self._tree_name.value,) + def __repr__(self): return '%s(%s)' % (self.__class__.__name__, self._tree_name.value) diff --git a/jedi/inference/gradual/conversion.py b/jedi/inference/gradual/conversion.py index 541aa0d1..3c05bb14 100644 --- a/jedi/inference/gradual/conversion.py +++ b/jedi/inference/gradual/conversion.py @@ -3,6 +3,7 @@ from jedi.inference.base_value import ValueSet, \ NO_VALUES from jedi.inference.utils import to_list from jedi.inference.gradual.stub_value import StubModuleValue +from jedi.inference.gradual.typeshed import try_to_load_stub_cached def _stub_to_python_value_set(stub_value, ignore_compiled=False): @@ -87,8 +88,7 @@ def _try_stub_to_python_names(names, prefer_stub_to_compiled=False): def _load_stub_module(module): if module.is_stub(): return module - from jedi.inference.gradual.typeshed import _try_to_load_stub_cached - return _try_to_load_stub_cached( + return try_to_load_stub_cached( module.inference_state, import_names=module.string_names, python_value_set=ValueSet([module]), diff --git a/jedi/inference/gradual/typeshed.py b/jedi/inference/gradual/typeshed.py index d8604fed..48692aa0 100644 --- a/jedi/inference/gradual/typeshed.py +++ b/jedi/inference/gradual/typeshed.py @@ -2,6 +2,7 @@ import os import re from functools import wraps +from jedi import settings from jedi.file_io import FileIO from jedi._compatibility import FileNotFoundError, cast_path from jedi.parser_utils import get_cached_code_lines @@ -119,8 +120,8 @@ def import_module_decorator(func): if not prefer_stubs: return python_value_set - stub = _try_to_load_stub_cached(inference_state, import_names, python_value_set, - parent_module_value, sys_path) + stub = try_to_load_stub_cached(inference_state, import_names, python_value_set, + parent_module_value, sys_path) if stub is not None: return ValueSet([stub]) return python_value_set @@ -128,7 +129,7 @@ def import_module_decorator(func): return wrapper -def _try_to_load_stub_cached(inference_state, import_names, *args, **kwargs): +def try_to_load_stub_cached(inference_state, import_names, *args, **kwargs): if import_names is None: return None @@ -155,7 +156,7 @@ def _try_to_load_stub(inference_state, import_names, python_value_set, """ if parent_module_value is None and len(import_names) > 1: try: - parent_module_value = _try_to_load_stub_cached( + parent_module_value = try_to_load_stub_cached( inference_state, import_names[:-1], NO_VALUES, parent_module_value=None, sys_path=sys_path) except KeyError: @@ -258,11 +259,7 @@ def _load_from_typeshed(inference_state, python_value_set, parent_module_value, def _try_to_load_stub_from_file(inference_state, python_value_set, file_io, import_names): try: - stub_module_node = inference_state.parse( - file_io=file_io, - cache=True, - use_latest_grammar=True - ) + stub_module_node = parse_stub_module(inference_state, file_io) except (OSError, IOError): # IOError is Python 2 only # The file that you're looking for doesn't exist (anymore). return None @@ -273,6 +270,16 @@ def _try_to_load_stub_from_file(inference_state, python_value_set, file_io, impo ) +def parse_stub_module(inference_state, file_io): + return inference_state.parse( + file_io=file_io, + cache=True, + diff_cache=settings.fast_parser, + cache_path=settings.cache_directory, + use_latest_grammar=True + ) + + def create_stub_module(inference_state, python_value_set, stub_module_node, file_io, import_names): if import_names == ('typing',): module_cls = TypingModuleWrapper diff --git a/jedi/inference/helpers.py b/jedi/inference/helpers.py index 3b0b33dd..3e4d3952 100644 --- a/jedi/inference/helpers.py +++ b/jedi/inference/helpers.py @@ -8,7 +8,6 @@ from contextlib import contextmanager from parso.python import tree from jedi._compatibility import unicode -from jedi.parser_utils import get_parent_scope def is_stdlib_path(path): @@ -122,29 +121,6 @@ def get_names_of_node(node): return list(chain.from_iterable(get_names_of_node(c) for c in children)) -def get_module_names(module, all_scopes): - """ - Returns a dictionary with name parts as keys and their call paths as - values. - """ - names = list(chain.from_iterable(module.get_used_names().values())) - if not all_scopes: - # We have to filter all the names that don't have the module as a - # parent_scope. There's None as a parent, because nodes in the module - # node have the parent module and not suite as all the others. - # Therefore it's important to catch that case. - - def is_module_scope_name(name): - parent_scope = get_parent_scope(name) - # async functions have an extra wrapper. Strip it. - if parent_scope and parent_scope.type == 'async_stmt': - parent_scope = parent_scope.parent - return parent_scope in (module, None) - - names = [n for n in names if is_module_scope_name(n)] - return names - - def is_string(value): if value.inference_state.environment.version_info.major == 2: str_classes = (unicode, bytes) diff --git a/jedi/inference/imports.py b/jedi/inference/imports.py index a43e19aa..926d05ef 100644 --- a/jedi/inference/imports.py +++ b/jedi/inference/imports.py @@ -16,9 +16,10 @@ import os from parso.python import tree from parso.tree import search_ancestor -from jedi._compatibility import ImplicitNSInfo, force_unicode +from jedi._compatibility import ImplicitNSInfo, force_unicode, FileNotFoundError from jedi import debug from jedi import settings +from jedi.file_io import FolderIO from jedi.parser_utils import get_cached_code_lines from jedi.inference import sys_path from jedi.inference import helpers @@ -28,8 +29,8 @@ from jedi.inference.utils import unite from jedi.inference.cache import inference_state_method_cache from jedi.inference.names import ImportName, SubModuleName from jedi.inference.base_value import ValueSet, NO_VALUES -from jedi.inference.gradual.typeshed import import_module_decorator -from jedi.inference.value.module import iter_module_names +from jedi.inference.gradual.typeshed import import_module_decorator, \ + create_stub_module, parse_stub_module from jedi.plugins import plugin_manager @@ -265,24 +266,15 @@ class Importer(object): Get the names of all modules in the search_path. This means file names and not names defined in the files. """ - names = [] - # add builtin module names - if search_path is None and in_module is None: - names += [ - ImportName(self._module_context, name) - for name in self._inference_state.compiled_subprocess.get_builtin_module_names() - ] - if search_path is None: - search_path = self._sys_path_with_modifications(is_completion=True) - - for name in iter_module_names(self._inference_state, search_path): - if in_module is None: - n = ImportName(self._module_context, name) - else: - n = SubModuleName(in_module.as_context(), name) - names.append(n) - return names + sys_path = self._sys_path_with_modifications(is_completion=True) + else: + sys_path = search_path + return list(iter_module_names( + self._inference_state, self._module_context, sys_path, + module_cls=ImportName if in_module is None else SubModuleName, + add_builtin_modules=search_path is None and in_module is None, + )) def completion_names(self, inference_state, only_modules=False): """ @@ -441,7 +433,7 @@ def _load_python_module(inference_state, file_io, file_io=file_io, cache=True, diff_cache=settings.fast_parser, - cache_path=settings.cache_directory + cache_path=settings.cache_directory, ) from jedi.inference.value import ModuleValue @@ -472,32 +464,59 @@ def _load_builtin_module(inference_state, import_names=None, sys_path=None): return module -def load_module_from_path(inference_state, file_io, base_names=None): +def load_module_from_path(inference_state, file_io, import_names=None, is_package=None): """ This should pretty much only be used for get_modules_containing_name. It's here to ensure that a random path is still properly loaded into the Jedi module structure. """ path = file_io.path - if base_names: - module_name = os.path.basename(path) - module_name = sys_path.remove_python_path_suffix(module_name) - is_package = module_name == '__init__' - if is_package: - import_names = base_names - else: - import_names = base_names + (module_name,) - else: + if import_names is None: e_sys_path = inference_state.get_sys_path() import_names, is_package = sys_path.transform_path_to_dotted(e_sys_path, path) + else: + assert isinstance(is_package, bool) - module = _load_python_module( - inference_state, file_io, - import_names=import_names, - is_package=is_package, + is_stub = file_io.path.endswith('.pyi') + if is_stub: + folder_io = file_io.get_parent_folder() + if folder_io.path.endswith('-stubs'): + folder_io = FolderIO(folder_io.path[:-6]) + if file_io.path.endswith('__init__.pyi'): + python_file_io = folder_io.get_file_io('__init__.py') + else: + python_file_io = folder_io.get_file_io(import_names[-1] + '.py') + + try: + v = load_module_from_path( + inference_state, python_file_io, + import_names, is_package=is_package + ) + values = ValueSet([v]) + except FileNotFoundError: + values = NO_VALUES + + return create_stub_module( + inference_state, values, parse_stub_module(inference_state, file_io), + file_io, import_names + ) + else: + module = _load_python_module( + inference_state, file_io, + import_names=import_names, + is_package=is_package, + ) + inference_state.module_cache.add(import_names, ValueSet([module])) + return module + + +def load_namespace_from_path(inference_state, folder_io): + import_names, is_package = sys_path.transform_path_to_dotted( + inference_state.get_sys_path(), + folder_io.path ) - inference_state.module_cache.add(import_names, ValueSet([module])) - return module + from jedi.inference.value.namespace import ImplicitNamespaceValue + return ImplicitNamespaceValue(inference_state, import_names, [folder_io.path]) def follow_error_node_imports_if_possible(context, name): @@ -527,3 +546,18 @@ def follow_error_node_imports_if_possible(context, name): return Importer( context.inference_state, names, context.get_root_context(), level).follow() return None + + +def iter_module_names(inference_state, module_context, search_path, + module_cls=ImportName, add_builtin_modules=True): + """ + Get the names of all modules in the search_path. This means file names + and not names defined in the files. + """ + # add builtin module names + if add_builtin_modules: + for name in inference_state.compiled_subprocess.get_builtin_module_names(): + yield module_cls(module_context, name) + + for name in inference_state.compiled_subprocess.iter_module_names(search_path): + yield module_cls(module_context, name) diff --git a/jedi/inference/recursion.py b/jedi/inference/recursion.py index 8fdd3b10..a0897fa8 100644 --- a/jedi/inference/recursion.py +++ b/jedi/inference/recursion.py @@ -3,8 +3,8 @@ Recursions are the recipe of |jedi| to conquer Python code. However, someone must stop recursions going mad. Some settings are here to make |jedi| stop at the right time. You can read more about them :ref:`here `. -Next to :mod:`jedi.inference.cache` this module also makes |jedi| not -thread-safe. Why? ``execution_recursion_decorator`` uses class variables to +Next to the internal ``jedi.inference.cache`` this module also makes |jedi| not +thread-safe, because ``execution_recursion_decorator`` uses class variables to count the function calls. .. _settings-recursion: @@ -34,7 +34,7 @@ from jedi.inference.base_value import NO_VALUES recursion_limit = 15 """ -Like ``sys.getrecursionlimit()``, just for |jedi|. +Like :func:`sys.getrecursionlimit()`, just for |jedi|. """ total_function_execution_limit = 200 """ diff --git a/jedi/inference/references.py b/jedi/inference/references.py index 1d62595f..91cdcac8 100644 --- a/jedi/inference/references.py +++ b/jedi/inference/references.py @@ -3,6 +3,7 @@ import re from parso import python_bytes_to_unicode +from jedi.debug import dbg from jedi.file_io import KnownContentFileIO from jedi.inference.imports import SubModuleName, load_module_from_path from jedi.inference.filters import ParserTreeFilter @@ -192,14 +193,15 @@ def gitignored_lines(folder_io, file_io): return ignored_paths, ignored_names -def _recurse_find_python_files(folder_io, except_paths): +def recurse_find_python_folders_and_files(folder_io, except_paths=()): + except_paths = set(except_paths) for root_folder_io, folder_ios, file_ios in folder_io.walk(): # Delete folders that we don't want to iterate over. for file_io in file_ios: path = file_io.path if path.endswith('.py') or path.endswith('.pyi'): if path not in except_paths: - yield file_io + yield None, file_io if path.endswith('.gitignore'): ignored_paths, ignored_names = \ @@ -212,6 +214,14 @@ def _recurse_find_python_files(folder_io, except_paths): if folder_io.path not in except_paths and folder_io.get_base_name() not in _IGNORE_FOLDERS ] + for folder_io in folder_ios: + yield folder_io, None + + +def recurse_find_python_files(folder_io, except_paths=()): + for folder_io, file_io in recurse_find_python_folders_and_files(folder_io, except_paths): + if file_io is not None: + yield file_io def _find_python_files_in_sys_path(inference_state, module_contexts): @@ -228,7 +238,7 @@ def _find_python_files_in_sys_path(inference_state, module_contexts): path = folder_io.path if not any(path.startswith(p) for p in sys_path) or path in except_paths: break - for file_io in _recurse_find_python_files(folder_io, except_paths): + for file_io in recurse_find_python_files(folder_io, except_paths): if file_io.path not in yielded_paths: yield file_io except_paths.add(path) @@ -254,19 +264,28 @@ def get_module_contexts_containing_name(inference_state, module_contexts, name, if len(name) <= 2: return + file_io_iterator = _find_python_files_in_sys_path(inference_state, module_contexts) + for x in search_in_file_ios(inference_state, file_io_iterator, name, + limit_reduction=limit_reduction): + yield x # Python 2... + + +def search_in_file_ios(inference_state, file_io_iterator, name, limit_reduction=1): parse_limit = _PARSED_FILE_LIMIT / limit_reduction open_limit = _OPENED_FILE_LIMIT / limit_reduction file_io_count = 0 parsed_file_count = 0 regex = re.compile(r'\b' + re.escape(name) + r'\b') - for file_io in _find_python_files_in_sys_path(inference_state, module_contexts): + for file_io in file_io_iterator: file_io_count += 1 m = _check_fs(inference_state, file_io, regex) if m is not None: parsed_file_count += 1 yield m if parsed_file_count >= parse_limit: + dbg('Hit limit of parsed files: %s', parse_limit) break if file_io_count >= open_limit: + dbg('Hit limit of opened files: %s', open_limit) break diff --git a/jedi/inference/star_args.py b/jedi/inference/star_args.py index 176fd9d6..0cda3839 100644 --- a/jedi/inference/star_args.py +++ b/jedi/inference/star_args.py @@ -32,7 +32,7 @@ def _iter_nodes_for_param(param_name): argument = name.parent if argument.type == 'argument' \ and argument.children[0] == '*' * param_name.star_count: - # No support for Python <= 3.4 here, but they are end-of-life + # No support for Python 2.7 here, but they are end-of-life # anyway trailer = search_ancestor(argument, 'trailer') if trailer is not None: # Make sure we're in a function diff --git a/jedi/inference/sys_path.py b/jedi/inference/sys_path.py index 5234ac2c..32a3f6b0 100644 --- a/jedi/inference/sys_path.py +++ b/jedi/inference/sys_path.py @@ -1,4 +1,5 @@ import os +import re from jedi._compatibility import unicode, force_unicode, all_suffixes from jedi.inference.cache import inference_state_method_cache @@ -207,7 +208,7 @@ def _get_buildout_script_paths(search_path): def remove_python_path_suffix(path): - for suffix in all_suffixes(): + for suffix in all_suffixes() + ['.pyi']: if path.endswith(suffix): path = path[:-len(suffix)] break @@ -254,7 +255,9 @@ def transform_path_to_dotted(sys_path, module_path): # is very strange and is probably a file that is called # `.py`. return - yield tuple(split) + # Stub folders for foo can end with foo-stubs. Just remove + # it. + yield tuple(re.sub(r'-stubs$', '', s) for s in split) potential_solutions = tuple(iter_potential_solutions()) if not potential_solutions: diff --git a/jedi/inference/utils.py b/jedi/inference/utils.py index a1efc047..422e17e7 100644 --- a/jedi/inference/utils.py +++ b/jedi/inference/utils.py @@ -107,19 +107,9 @@ class PushBackIterator(object): def ignored(*exceptions): """ Value manager that ignores all of the specified exceptions. This will - be in the standard library starting with Python 3.4. + be in the standard library starting with Python 3.5. """ try: yield except exceptions: pass - - -def indent_block(text, indention=' '): - """This function indents a text block with a default of four spaces.""" - temp = '' - while text and text[-1] == '\n': - temp += text[-1] - text = text[:-1] - lines = text.split('\n') - return '\n'.join(map(lambda s: indention + s, lines)) + temp diff --git a/jedi/inference/value/function.py b/jedi/inference/value/function.py index 44cdfd6c..7513c6a9 100644 --- a/jedi/inference/value/function.py +++ b/jedi/inference/value/function.py @@ -202,9 +202,6 @@ class MethodValue(FunctionValue): class BaseFunctionExecutionContext(ValueContext, TreeContextMixin): - def is_function_execution(self): - return True - def _infer_annotations(self): raise NotImplementedError diff --git a/jedi/inference/value/module.py b/jedi/inference/value/module.py index 8f2ba9e1..51cc5863 100644 --- a/jedi/inference/value/module.py +++ b/jedi/inference/value/module.py @@ -1,7 +1,5 @@ -import re import os -from jedi import debug from jedi.inference.cache import inference_state_method_cache from jedi.inference.names import AbstractNameDefinition, ModuleName from jedi.inference.filters import GlobalNameFilter, ParserTreeFilter, DictFilter, MergedFilter @@ -37,32 +35,6 @@ class _ModuleAttributeName(AbstractNameDefinition): return compiled.get_string_value_set(self.parent_context.inference_state) -def iter_module_names(inference_state, paths): - # Python modules/packages - for n in inference_state.compiled_subprocess.list_module_names(paths): - yield n - - for path in paths: - try: - dirs = os.listdir(path) - except OSError: - # The file might not exist or reading it might lead to an error. - debug.warning("Not possible to list directory: %s", path) - continue - for name in dirs: - # Namespaces - if os.path.isdir(os.path.join(path, name)): - # pycache is obviously not an interestin namespace. Also the - # name must be a valid identifier. - # TODO use str.isidentifier, once Python 2 is removed - if name != '__pycache__' and not re.search(r'\W|^\d', name): - yield name - # Stub files - if name.endswith('.pyi'): - if name != '__init__.pyi': - yield name[:-4] - - class SubModuleDictMixin(object): @inference_state_method_cache() def sub_modules_dict(self): @@ -72,7 +44,9 @@ class SubModuleDictMixin(object): """ names = {} if self.is_package(): - mods = iter_module_names(self.inference_state, self.py__path__()) + mods = self.inference_state.compiled_subprocess.iter_module_names( + self.py__path__() + ) for name in mods: # It's obviously a relative import to the current module. names[name] = SubModuleName(self.as_context(), name) @@ -111,20 +85,7 @@ class ModuleMixin(SubModuleDictMixin): @property @inference_state_method_cache() def name(self): - return self._module_name_class(self, self._string_name) - - @property - def _string_name(self): - """ This is used for the goto functions. """ - # TODO It's ugly that we even use this, the name is usually well known - # ahead so just pass it when create a ModuleValue. - if self._path is None: - return '' # no path -> empty name - else: - sep = (re.escape(os.path.sep),) * 2 - r = re.search(r'([^%s]*?)(%s__init__)?(\.pyi?|\.so)?$' % sep, self._path) - # Remove PEP 3149 names - return re.sub(r'\.[a-z]+-\d{2}[mud]{0,3}$', '', r.group(1)) + return self._module_name_class(self, self.string_names[-1]) @inference_state_method_cache() def _module_attributes_dict(self): @@ -260,7 +221,7 @@ class ModuleValue(ModuleMixin, TreeValue): def __repr__(self): return "<%s: %s@%s-%s is_stub=%s>" % ( - self.__class__.__name__, self._string_name, + self.__class__.__name__, self.py__name__(), self.tree_node.start_pos[0], self.tree_node.end_pos[0], self.is_stub() ) diff --git a/jedi/inference/value/namespace.py b/jedi/inference/value/namespace.py index ec41e6f1..48a09e4a 100644 --- a/jedi/inference/value/namespace.py +++ b/jedi/inference/value/namespace.py @@ -35,6 +35,9 @@ class ImplicitNamespaceValue(Value, SubModuleDictMixin): def get_filters(self, origin_scope=None): yield DictFilter(self.sub_modules_dict()) + def get_qualified_names(self): + return () + @property @inference_state_method_cache() def name(self): diff --git a/jedi/refactoring.py b/jedi/refactoring.py deleted file mode 100644 index 6b5b4f66..00000000 --- a/jedi/refactoring.py +++ /dev/null @@ -1,203 +0,0 @@ -""" -THIS is not in active development, please check -https://github.com/davidhalter/jedi/issues/667 first before editing. - -Introduce some basic refactoring functions to |jedi|. This module is still in a -very early development stage and needs much testing and improvement. - -.. warning:: I won't do too much here, but if anyone wants to step in, please - do. Refactoring is none of my priorities - -It uses the |jedi| `API `_ and supports currently the -following functions (sometimes bug-prone): - -- rename -- extract variable -- inline variable -""" -import difflib - -from parso import python_bytes_to_unicode, split_lines -from jedi.inference import helpers - - -class Refactoring(object): - def __init__(self, change_dct): - """ - :param change_dct: dict(old_path=(new_path, old_lines, new_lines)) - """ - self.change_dct = change_dct - - def old_files(self): - dct = {} - for old_path, (new_path, old_l, new_l) in self.change_dct.items(): - dct[old_path] = '\n'.join(old_l) - return dct - - def new_files(self): - dct = {} - for old_path, (new_path, old_l, new_l) in self.change_dct.items(): - dct[new_path] = '\n'.join(new_l) - return dct - - def diff(self): - texts = [] - for old_path, (new_path, old_l, new_l) in self.change_dct.items(): - if old_path: - udiff = difflib.unified_diff(old_l, new_l) - else: - udiff = difflib.unified_diff(old_l, new_l, old_path, new_path) - texts.append('\n'.join(udiff)) - return '\n'.join(texts) - - -def rename(script, new_name): - """ The `args` / `kwargs` params are the same as in `api.Script`. - :param new_name: The new name of the script. - :param script: The source Script object. - :return: list of changed lines/changed files - """ - return Refactoring(_rename(script.get_references(), new_name)) - - -def _rename(names, replace_str): - """ For both rename and inline. """ - order = sorted(names, key=lambda x: (x.module_path, x.line, x.column), - reverse=True) - - def process(path, old_lines, new_lines): - if new_lines is not None: # goto next file, save last - dct[path] = path, old_lines, new_lines - - dct = {} - current_path = object() - new_lines = old_lines = None - for name in order: - if name.in_builtin_module(): - continue - if current_path != name.module_path: - current_path = name.module_path - - process(current_path, old_lines, new_lines) - if current_path is not None: - # None means take the source that is a normal param. - with open(current_path) as f: - source = f.read() - - new_lines = split_lines(python_bytes_to_unicode(source)) - old_lines = new_lines[:] - - nr, indent = name.line, name.column - line = new_lines[nr - 1] - new_lines[nr - 1] = line[:indent] + replace_str + \ - line[indent + len(name.name):] - process(current_path, old_lines, new_lines) - return dct - - -def extract(script, new_name): - """ The `args` / `kwargs` params are the same as in `api.Script`. - :param operation: The refactoring operation to execute. - :type operation: str - :type source: str - :return: list of changed lines/changed files - """ - new_lines = split_lines(python_bytes_to_unicode(script.source)) - old_lines = new_lines[:] - - user_stmt = script._parser.user_stmt() - - # TODO care for multi-line extracts - dct = {} - if user_stmt: - pos = script._pos - line_index = pos[0] - 1 - # Be careful here. 'array_for_pos' does not exist in 'helpers'. - arr, index = helpers.array_for_pos(user_stmt, pos) - if arr is not None: - start_pos = arr[index].start_pos - end_pos = arr[index].end_pos - - # take full line if the start line is different from end line - e = end_pos[1] if end_pos[0] == start_pos[0] else None - start_line = new_lines[start_pos[0] - 1] - text = start_line[start_pos[1]:e] - for l in range(start_pos[0], end_pos[0] - 1): - text += '\n' + str(l) - if e is None: - end_line = new_lines[end_pos[0] - 1] - text += '\n' + end_line[:end_pos[1]] - - # remove code from new lines - t = text.lstrip() - del_start = start_pos[1] + len(text) - len(t) - - text = t.rstrip() - del_end = len(t) - len(text) - if e is None: - new_lines[end_pos[0] - 1] = end_line[end_pos[1] - del_end:] - e = len(start_line) - else: - e = e - del_end - start_line = start_line[:del_start] + new_name + start_line[e:] - new_lines[start_pos[0] - 1] = start_line - new_lines[start_pos[0]:end_pos[0] - 1] = [] - - # add parentheses in multi-line case - open_brackets = ['(', '[', '{'] - close_brackets = [')', ']', '}'] - if '\n' in text and not (text[0] in open_brackets and text[-1] - == close_brackets[open_brackets.index(text[0])]): - text = '(%s)' % text - - # add new line before statement - indent = user_stmt.start_pos[1] - new = "%s%s = %s" % (' ' * indent, new_name, text) - new_lines.insert(line_index, new) - dct[script.path] = script.path, old_lines, new_lines - return Refactoring(dct) - - -def inline(script): - """ - :type script: api.Script - """ - new_lines = split_lines(python_bytes_to_unicode(script.source)) - - dct = {} - - definitions = script.goto() - assert len(definitions) == 1 - stmt = definitions[0]._definition - references = script.get_references() - inlines = [r for r in references - if not stmt.start_pos <= (r.line, r.column) <= stmt.end_pos] - inlines = sorted(inlines, key=lambda x: (x.module_path, x.line, x.column), - reverse=True) - expression_list = stmt.expression_list() - # don't allow multi-line refactorings for now. - assert stmt.start_pos[0] == stmt.end_pos[0] - index = stmt.start_pos[0] - 1 - - line = new_lines[index] - replace_str = line[expression_list[0].start_pos[1]:stmt.end_pos[1] + 1] - replace_str = replace_str.strip() - # tuples need parentheses - if expression_list and expression_list[0].type == 'TODO': - arr = expression_list[0] - if replace_str[0] not in ['(', '[', '{'] and len(arr) > 1: - replace_str = '(%s)' % replace_str - - # if it's the only assignment, remove the statement - if len(stmt.get_defined_names()) == 1: - line = line[:stmt.start_pos[1]] + line[stmt.end_pos[1]:] - - dct = _rename(inlines, replace_str) - # remove the empty line - new_lines = dct[script.path][2] - if line.strip(): - new_lines[index] = line - else: - new_lines.pop(index) - - return Refactoring(dct) diff --git a/jedi/settings.py b/jedi/settings.py index 9797f049..1573d59c 100644 --- a/jedi/settings.py +++ b/jedi/settings.py @@ -51,22 +51,21 @@ import os import platform # ---------------- -# completion output settings +# Completion Output Settings # ---------------- case_insensitive_completion = True """ -The completion is by default case insensitive. +Completions are by default case insensitive. """ add_bracket_after_function = False """ -Adds an opening bracket after a function, because that's normal behaviour. -Removed it again, because in VIM that is not very practical. +Adds an opening bracket after a function for completions. """ # ---------------- -# Filesystem cache +# Filesystem Cache # ---------------- if platform.system().lower() == 'windows': @@ -83,31 +82,32 @@ The path where the cache is stored. On Linux, this defaults to ``~/.cache/jedi/``, on OS X to ``~/Library/Caches/Jedi/`` and on Windows to ``%APPDATA%\\Jedi\\Jedi\\``. -On Linux, if environment variable ``$XDG_CACHE_HOME`` is set, +On Linux, if the environment variable ``$XDG_CACHE_HOME`` is set, ``$XDG_CACHE_HOME/jedi`` is used instead of the default one. """ # ---------------- -# parser +# Parser # ---------------- fast_parser = True """ -Use the fast parser. This means that reparsing is only being done if -something has been changed e.g. to a function. If this happens, only the -function is being reparsed. +Uses Parso's diff parser. If it is enabled, this might cause issues, please +read the warning on :class:`.Script`. This feature makes it possible to only +parse the parts again that have changed, while reusing the rest of the syntax +tree. """ _cropped_file_size = 10e6 # 1 Megabyte """ Jedi gets extremely slow if the file size exceed a few thousand lines. -To avoid getting stuck completely Jedi crops the file this point. +To avoid getting stuck completely Jedi crops the file at some point. One megabyte of typical Python code equals about 20'000 lines of code. """ # ---------------- -# dynamic stuff +# Dynamic Stuff # ---------------- dynamic_array_additions = True @@ -135,13 +135,13 @@ auto_import_modules = [ 'gi', # This third-party repository (GTK stuff) doesn't really work with jedi ] """ -Modules that are not analyzed but imported, although they contain Python code. +Modules that will not be analyzed but imported, if they contain Python code. This improves autocompletion for libraries that use ``setattr`` or ``globals()`` modifications a lot. """ # ---------------- -# caching validity (time) +# Caching Validity # ---------------- call_signatures_validity = 3.0 diff --git a/jedi/utils.py b/jedi/utils.py index 85fe4314..9d675174 100644 --- a/jedi/utils.py +++ b/jedi/utils.py @@ -19,15 +19,14 @@ READLINE_DEBUG = False def setup_readline(namespace_module=__main__, fuzzy=False): """ - Install Jedi completer to :mod:`readline`. + This function sets up :mod:`readline` to use Jedi in a Python interactive + shell. - This function setups :mod:`readline` to use Jedi in Python interactive - shell. If you want to use a custom ``PYTHONSTARTUP`` file (typically + If you want to use a custom ``PYTHONSTARTUP`` file (typically ``$HOME/.pythonrc.py``), you can add this piece of code:: try: from jedi.utils import setup_readline - setup_readline() except ImportError: # Fallback to the stdlib readline completer if it is installed. # Taken from http://docs.python.org/2/library/rlcompleter.html @@ -38,6 +37,8 @@ def setup_readline(namespace_module=__main__, fuzzy=False): readline.parse_and_bind("tab: complete") except ImportError: print("Readline is not installed either. No tab completion is enabled.") + else: + setup_readline() This will fallback to the readline completer if Jedi is not installed. The readline completer will only complete names in the global namespace, @@ -45,18 +46,18 @@ def setup_readline(namespace_module=__main__, fuzzy=False): ran - will complete to ``range`` + will complete to ``range``. - with both Jedi and readline, but:: + With Jedi the following code:: range(10).cou - will show complete to ``range(10).count`` only with Jedi. + will complete to ``range(10).count``, this does not work with the default + cPython :mod:`readline` completer. - You'll also need to add ``export PYTHONSTARTUP=$HOME/.pythonrc.py`` to + You will also need to add ``export PYTHONSTARTUP=$HOME/.pythonrc.py`` to your shell profile (usually ``.bash_profile`` or ``.profile`` if you use bash). - """ if READLINE_DEBUG: logging.basicConfig( diff --git a/setup.py b/setup.py index e3f037a9..b73fea79 100755 --- a/setup.py +++ b/setup.py @@ -33,16 +33,16 @@ setup(name='jedi', keywords='python completion refactoring vim', long_description=readme, packages=find_packages(exclude=['test', 'test.*']), - python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*', + python_requires='>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*', install_requires=install_requires, extras_require={ 'testing': [ - # Pytest 5 doesn't support Python 2 and Python 3.4 anymore. + # Pytest 5 doesn't support Python 2 anymore. 'pytest>=3.9.0,<5.0.0', # docopt for sith doctests 'docopt', # coloroma for colored debug output - 'colorama==0.4.1', # Pinned so it works for Python 3.4 + 'colorama', ], 'qa': [ 'flake8==3.7.9', @@ -60,7 +60,6 @@ setup(name='jedi', 'Programming Language :: Python :: 2', 'Programming Language :: Python :: 2.7', 'Programming Language :: Python :: 3', - 'Programming Language :: Python :: 3.4', 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', diff --git a/sith.py b/sith.py index 746bac4a..4718b8c1 100755 --- a/sith.py +++ b/sith.py @@ -168,7 +168,7 @@ class TestCase(object): def show_definitions(self): for completion in self.objects: - print(completion.desc_with_module) + print(completion.full_name) if completion.module_path is None: continue if os.path.abspath(completion.module_path) == os.path.abspath(self.path): diff --git a/test/completion/arrays.py b/test/completion/arrays.py index 9d3dcd2e..3d2e108f 100644 --- a/test/completion/arrays.py +++ b/test/completion/arrays.py @@ -473,7 +473,7 @@ def test_func(): #? int() tuple({1})[0] -# python >= 3.4 +# python > 2.7 # ----------------- # PEP 3132 Extended Iterable Unpacking (star unpacking) # ----------------- diff --git a/test/completion/generators.py b/test/completion/generators.py index e36f3928..566f0036 100644 --- a/test/completion/generators.py +++ b/test/completion/generators.py @@ -242,7 +242,7 @@ def x(): # yield from # ----------------- -# python >= 3.4 +# python > 2.7 def yield_from(): yield from iter([1]) diff --git a/test/completion/on_import.py b/test/completion/on_import.py index 76984b12..ddf04151 100644 --- a/test/completion/on_import.py +++ b/test/completion/on_import.py @@ -12,6 +12,8 @@ def from_names_goto(): def builtin_test(): #? ['math'] import math + #? ['mmap'] + import mmap # ----------------- # completions within imports diff --git a/test/completion/pep0484_basic.py b/test/completion/pep0484_basic.py index 6c89c738..a20475ca 100644 --- a/test/completion/pep0484_basic.py +++ b/test/completion/pep0484_basic.py @@ -1,6 +1,6 @@ """ Pep-0484 type hinting """ -# python >= 3.4 +# python > 2.7 class A(): diff --git a/test/completion/pep0484_typing.py b/test/completion/pep0484_typing.py index ad3fc51f..8060270b 100644 --- a/test/completion/pep0484_typing.py +++ b/test/completion/pep0484_typing.py @@ -283,7 +283,7 @@ def testnewtype2(y): y #? [] y. -# python >= 3.4 +# python > 2.7 class TestDefaultDict(typing.DefaultDict[str, int]): def setdud(self): @@ -311,7 +311,7 @@ for key in x.keys(): for value in x.values(): #? int() value -# python >= 3.4 +# python > 2.7 """ @@ -341,9 +341,8 @@ typing.Optional[0] TYPE_VARX = typing.TypeVar('TYPE_VARX') TYPE_VAR_CONSTRAINTSX = typing.TypeVar('TYPE_VAR_CONSTRAINTSX', str, int) -# TODO there should at least be some results. -#? [] -TYPE_VARX. +#? ['__class__'] +TYPE_VARX.__clas #! ["TYPE_VARX = typing.TypeVar('TYPE_VARX')"] TYPE_VARX diff --git a/test/completion/pytest.py b/test/completion/pytest.py index ba02e28d..c4030918 100644 --- a/test/completion/pytest.py +++ b/test/completion/pytest.py @@ -130,7 +130,7 @@ def test_p(monkeypatch): #? ['setattr'] monkeypatch.setatt -# python > 3.4 +# python > 2.7 #? ['capsysbinary'] def test_p(capsysbin diff --git a/test/completion/stdlib.py b/test/completion/stdlib.py index 3b2ca4ae..306aea11 100644 --- a/test/completion/stdlib.py +++ b/test/completion/stdlib.py @@ -359,7 +359,7 @@ class Test(metaclass=Meta): # Enum # ----------------- -# python >= 3.4 +# python > 2.7 import enum class X(enum.Enum): diff --git a/test/completion/stubs.py b/test/completion/stubs.py index 83013542..4e24eef1 100644 --- a/test/completion/stubs.py +++ b/test/completion/stubs.py @@ -1,4 +1,4 @@ -# python >= 3.4 +# python > 2.7 from stub_folder import with_stub, stub_only, with_stub_folder, stub_only_folder # ------------------------- diff --git a/test/conftest.py b/test/conftest.py index 750cbc1b..26254000 100644 --- a/test/conftest.py +++ b/test/conftest.py @@ -76,9 +76,11 @@ def pytest_generate_tests(metafunc): if 'refactor_case' in metafunc.fixturenames: base_dir = metafunc.config.option.refactor_case_dir + cases = list(refactor.collect_dir_tests(base_dir, test_files)) metafunc.parametrize( - 'refactor_case', - refactor.collect_dir_tests(base_dir, test_files)) + 'refactor_case', cases, + ids=[c.refactor_type + '-' + c.name for c in cases] + ) if 'static_analysis_case' in metafunc.fixturenames: base_dir = os.path.join(os.path.dirname(__file__), 'static_analysis') diff --git a/test/examples/buildout_project/bin/app b/test/examples/buildout_project/bin/app index e99613bd..e8df4eb6 100644 --- a/test/examples/buildout_project/bin/app +++ b/test/examples/buildout_project/bin/app @@ -2,7 +2,7 @@ import sys sys.path[0:0] = [ - '/usr/lib/python3.4/site-packages', + '/usr/lib/python3.8/site-packages', '/tmp/.buildout/eggs/important_package.egg' ] diff --git a/test/examples/init_extension_module/__init__.cpython-34m.so b/test/examples/init_extension_module/__init__.cpython-34m.so deleted file mode 100755 index abfadc39..00000000 Binary files a/test/examples/init_extension_module/__init__.cpython-34m.so and /dev/null differ diff --git a/test/examples/init_extension_module/__init__.cpython-38-x86_64-linux-gnu.so b/test/examples/init_extension_module/__init__.cpython-38-x86_64-linux-gnu.so new file mode 100755 index 00000000..19957841 Binary files /dev/null and b/test/examples/init_extension_module/__init__.cpython-38-x86_64-linux-gnu.so differ diff --git a/test/refactor.py b/test/refactor.py old mode 100755 new mode 100644 index 9cc10fda..2a997b19 --- a/test/refactor.py +++ b/test/refactor.py @@ -1,99 +1,103 @@ #!/usr/bin/env python """ -Refactoring tests work a little bit similar to Black Box tests. But the idea is -here to compare two versions of code. **Note: Refactoring is currently not in -active development (and was never stable), the tests are therefore not really -valuable - just ignore them.** +Refactoring tests work a little bit similar to integration tests. But the idea +is here to compare two versions of code. If you want to add a new test case, +just look at the existing ones in the ``test/refactor`` folder and copy them. """ from __future__ import with_statement import os +import platform import re +import sys + +from parso import split_lines from functools import reduce import jedi -from jedi import refactoring +from .helpers import test_dir class RefactoringCase(object): - def __init__(self, name, source, line_nr, index, path, - new_name, start_line_test, desired): + def __init__(self, name, code, line_nr, index, path, kwargs, type_, desired_result): self.name = name - self.source = source - self.line_nr = line_nr - self.index = index - self.path = path - self.new_name = new_name - self.start_line_test = start_line_test - self.desired = desired + self._code = code + self._line_nr = line_nr + self._index = index + self._path = path + self._kwargs = kwargs + self.type = type_ + self._desired_result = desired_result - def refactor(self): - script = jedi.Script(self.source, self.line_nr, self.index, self.path) - f_name = os.path.basename(self.path) - refactor_func = getattr(refactoring, f_name.replace('.py', '')) - args = (self.new_name,) if self.new_name else () - return refactor_func(script, *args) + def get_desired_result(self): - def run(self): - refactor_object = self.refactor() + if platform.system().lower() == 'windows' and self.type == 'diff': + # Windows uses backslashes to separate paths. + lines = split_lines(self._desired_result, keepends=True) + for i, line in enumerate(lines): + if re.search(' import_tree/', line): + lines[i] = line.replace('/', '\\') + return ''.join(lines) + return self._desired_result - # try to get the right excerpt of the newfile - f = refactor_object.new_files()[self.path] - lines = f.splitlines()[self.start_line_test:] + @property + def refactor_type(self): + f_name = os.path.basename(self._path) + return f_name.replace('.py', '') - end = self.start_line_test + len(lines) - pop_start = None - for i, l in enumerate(lines): - if l.startswith('# +++'): - end = i - break - elif '#? ' in l: - pop_start = i - lines.pop(pop_start) - self.result = '\n'.join(lines[:end - 1]).strip() - return self.result - - def check(self): - return self.run() == self.desired + def refactor(self, environment): + project = jedi.Project(os.path.join(test_dir, 'refactor')) + script = jedi.Script(self._code, path=self._path, project=project, environment=environment) + refactor_func = getattr(script, self.refactor_type) + return refactor_func(self._line_nr, self._index, **self._kwargs) def __repr__(self): return '<%s: %s:%s>' % (self.__class__.__name__, - self.name, self.line_nr - 1) + self.name, self._line_nr - 1) -def collect_file_tests(source, path, lines_to_execute): - r = r'^# --- ?([^\n]*)\n((?:(?!\n# \+\+\+).)*)' \ - r'\n# \+\+\+((?:(?!\n# ---).)*)' - for match in re.finditer(r, source, re.DOTALL | re.MULTILINE): +def _collect_file_tests(code, path, lines_to_execute): + r = r'^# -{5,} ?([^\n]*)\n((?:(?!\n# \+{5,}).)*\n)' \ + r'# \+{5,}\n((?:(?!\n# -{5,}).)*\n)' + match = None + for match in re.finditer(r, code, re.DOTALL | re.MULTILINE): name = match.group(1).strip() - first = match.group(2).strip() - second = match.group(3).strip() - start_line_test = source[:match.start()].count('\n') + 1 + first = match.group(2) + second = match.group(3) # get the line with the position of the operation - p = re.match(r'((?:(?!#\?).)*)#\? (\d*) ?([^\n]*)', first, re.DOTALL) + p = re.match(r'((?:(?!#\?).)*)#\? (\d*)( error| text|) ?([^\n]*)', first, re.DOTALL) if p is None: - print("Please add a test start.") + raise Exception("Please add a test start.") continue until = p.group(1) index = int(p.group(2)) - new_name = p.group(3) + type_ = p.group(3).strip() or 'diff' + if p.group(4): + kwargs = eval(p.group(4)) + else: + kwargs = {} - line_nr = start_line_test + until.count('\n') + 2 + line_nr = until.count('\n') + 2 if lines_to_execute and line_nr - 1 not in lines_to_execute: continue - yield RefactoringCase(name, source, line_nr, index, path, - new_name, start_line_test, second) + yield RefactoringCase(name, first, line_nr, index, path, kwargs, type_, second) + if match is None: + raise Exception("Didn't match any test") + if match.end() != len(code): + raise Exception("Didn't match until the end of the file in %s" % path) def collect_dir_tests(base_dir, test_files): + if sys.version_info[0] == 2: + return for f_name in os.listdir(base_dir): files_to_execute = [a for a in test_files.items() if a[0] in f_name] lines_to_execute = reduce(lambda x, y: x + y[1], files_to_execute, []) if f_name.endswith(".py") and (not test_files or files_to_execute): path = os.path.join(base_dir, f_name) - with open(path) as f: - source = f.read() - for case in collect_file_tests(source, path, lines_to_execute): + with open(path, newline='') as f: + code = f.read() + for case in _collect_file_tests(code, path, lines_to_execute): yield case diff --git a/test/refactor/extract.py b/test/refactor/extract.py deleted file mode 100644 index 312aced8..00000000 --- a/test/refactor/extract.py +++ /dev/null @@ -1,47 +0,0 @@ -# --- simple -def test(): - #? 35 a - return test(100, (30 + b, c) + 1) - -# +++ -def test(): - a = (30 + b, c) + 1 - return test(100, a) - - -# --- simple #2 -def test(): - #? 25 a - return test(100, (30 + b, c) + 1) - -# +++ -def test(): - a = 30 + b - return test(100, (a, c) + 1) - - -# --- multiline -def test(): - #? 30 x - return test(1, (30 + b, c) - + 1) -# +++ -def test(): - x = ((30 + b, c) - + 1) - return test(1, x -) - - -# --- multiline #2 -def test(): - #? 25 x - return test(1, (30 + b, c) - + 1) -# +++ -def test(): - x = 30 + b - return test(1, (x, c) - + 1) - - diff --git a/test/refactor/extract_function.py b/test/refactor/extract_function.py new file mode 100644 index 00000000..04bdbded --- /dev/null +++ b/test/refactor/extract_function.py @@ -0,0 +1,447 @@ +# -------------------------------------------------- in-module-1 +glob = 3 +#? 11 text {'new_name': 'a'} +test(100, (glob.a + b, c) + 1) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +glob = 3 +#? 11 text {'new_name': 'a'} +def a(b): + return glob.a + b + + +test(100, (a(b), c) + 1) +# -------------------------------------------------- in-module-2 +#? 0 text {'new_name': 'ab'} +100 + 1 * 2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 0 text {'new_name': 'ab'} +def ab(): + return 100 + 1 * 2 + + +ab() +# -------------------------------------------------- in-function-1 +def f(x): +#? 11 text {'new_name': 'ab'} + return x + 1 * 2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def ab(x): + return x + 1 * 2 + + +def f(x): +#? 11 text {'new_name': 'ab'} + return ab(x) +# -------------------------------------------------- in-function-with-dec +@classmethod +def f(x): +#? 11 text {'new_name': 'ab'} + return x + 1 * 2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def ab(x): + return x + 1 * 2 + + +@classmethod +def f(x): +#? 11 text {'new_name': 'ab'} + return ab(x) +# -------------------------------------------------- in-method-1 +class X: + def z(self): pass + + def f(x, b): + #? 11 text {'new_name': 'ab'} + return x + b * 2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +class X: + def z(self): pass + + def ab(x, b): + return x + b * 2 + + def f(x, b): + #? 11 text {'new_name': 'ab'} + return x.ab(b) +# -------------------------------------------------- in-method-2 +glob1 = 1 +class X: + def g(self): pass + + def f(self, b, c): + #? 11 text {'new_name': 'ab'} + return self.g() or self.f(b) ^ glob1 & b +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +glob1 = 1 +class X: + def g(self): pass + + def ab(self, b): + return self.g() or self.f(b) ^ glob1 & b + + def f(self, b, c): + #? 11 text {'new_name': 'ab'} + return self.ab(b) +# -------------------------------------------------- in-method-order +class X: + def f(self, b, c): + #? 18 text {'new_name': 'b'} + return b | self.a +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +class X: + def b(self, b): + return b | self.a + + def f(self, b, c): + #? 18 text {'new_name': 'b'} + return self.b(b) +# -------------------------------------------------- in-classmethod-1 +class X: + @classmethod + def f(x): + #? 16 text {'new_name': 'ab'} + return 25 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +class X: + @classmethod + def ab(x): + return 25 + + @classmethod + def f(x): + #? 16 text {'new_name': 'ab'} + return x.ab() +# -------------------------------------------------- in-staticmethod-1 +class X(int): + @staticmethod + def f(x): + #? 16 text {'new_name': 'ab'} + return 25 | 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def ab(): + return 25 | 3 + +class X(int): + @staticmethod + def f(x): + #? 16 text {'new_name': 'ab'} + return ab() +# -------------------------------------------------- in-class-1 +class Ya(): + a = 3 + #? 11 text {'new_name': 'f'} + c = a + 2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def f(a): + return a + 2 + + +class Ya(): + a = 3 + #? 11 text {'new_name': 'f'} + c = f(a) +# -------------------------------------------------- in-closure +def x(z): + def y(x): + #? 15 text {'new_name': 'f'} + return -x * z +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def f(x, z): + return -x * z + + +def x(z): + def y(x): + #? 15 text {'new_name': 'f'} + return f(x, z) +# -------------------------------------------------- with-range-1 +#? 0 text {'new_name': 'a', 'until_line': 4} +v1 = 3 +v2 = 2 +x = test(v1 + v2 * v3) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 0 text {'new_name': 'a', 'until_line': 4} +def a(test, v3): + v1 = 3 + v2 = 2 + x = test(v1 + v2 * v3) + return x + + +x = a(test, v3) +# -------------------------------------------------- with-range-2 +#? 2 text {'new_name': 'a', 'until_line': 6, 'until_column': 4} +#foo +v1 = 3 +v2 = 2 +x, y = test(v1 + v2 * v3) +#raaaa +y +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 2 text {'new_name': 'a', 'until_line': 6, 'until_column': 4} +def a(test, v3): + #foo + v1 = 3 + v2 = 2 + x, y = test(v1 + v2 * v3) + #raaaa + return y + + +y = a(test, v3) +y +# -------------------------------------------------- with-range-3 +#foo +#? 2 text {'new_name': 'a', 'until_line': 5, 'until_column': 4} +v1 = 3 +v2 = 2 +x, y = test(v1 + v2 * v3) +#raaaa +y +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#foo +#? 2 text {'new_name': 'a', 'until_line': 5, 'until_column': 4} +def a(test, v3): + v1 = 3 + v2 = 2 + x, y = test(v1 + v2 * v3) + return y + + +y = a(test, v3) +#raaaa +y +# -------------------------------------------------- with-range-func-1 +import os +# comment1 +@dec +# comment2 +def x(v1): + #foo + #? 2 text {'new_name': 'a', 'until_line': 9, 'until_column': 5} + v2 = 2 + if 1: + x, y = os.listdir(v1 + v2 * v3) + #bar + return x, y +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +import os +# comment1 +def a(v1, v3): + v2 = 2 + if 1: + x, y = os.listdir(v1 + v2 * v3) + return x, y + + +@dec +# comment2 +def x(v1): + #foo + #? 2 text {'new_name': 'a', 'until_line': 9, 'until_column': 5} + x, y = a(v1, v3) + #bar + return x, y +# -------------------------------------------------- with-range-func-2 +import os +# comment1 +# comment2 +def x(v1): + #? 2 text {'new_name': 'a', 'until_line': 10, 'until_column': 0} + #foo + v2 = 2 + if 1: + x, y = os.listdir(v1 + v2 * v3) + #bar + return y +x +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +import os +# comment1 +# comment2 +def a(v1, v3): + #foo + v2 = 2 + if 1: + x, y = os.listdir(v1 + v2 * v3) + #bar + return y + + +def x(v1): + #? 2 text {'new_name': 'a', 'until_line': 10, 'until_column': 0} + y = a(v1, v3) + return y +x +# -------------------------------------------------- with-range-func-3 +def x(v1): + #? 2 text {'new_name': 'func', 'until_line': 6, 'until_column': 4} + #foo + v2 = 2 + x = v1 * 2 + y = 3 + #bar + return x +x +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def func(v1): + #foo + v2 = 2 + x = v1 * 2 + return x + + +def x(v1): + #? 2 text {'new_name': 'func', 'until_line': 6, 'until_column': 4} + x = func(v1) + y = 3 + #bar + return x +x +# -------------------------------------------------- in-class-range-1 +class X1: + #? 11 text {'new_name': 'f', 'until_line': 4} + a = 3 + c = a + 2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def f(): + a = 3 + c = a + 2 + return c + + +class X1: + #? 11 text {'new_name': 'f', 'until_line': 4} + c = f() +# -------------------------------------------------- in-method-range-1 +glob1 = 1 +class X: + # ha + def g(self): pass + + # haha + def f(self, b, c): + #? 11 text {'new_name': 'ab', 'until_line': 12, 'until_column': 28} + #foo + local1 = 3 + local2 = 4 + x= self.g() or self.f(b) ^ glob1 & b is local1 + # bar +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +glob1 = 1 +class X: + # ha + def g(self): pass + + # haha + def ab(self, b): + #foo + local1 = 3 + local2 = 4 + x= self.g() or self.f(b) ^ glob1 & b is local1 + return x + + def f(self, b, c): + #? 11 text {'new_name': 'ab', 'until_line': 12, 'until_column': 28} + x = self.ab(b) + # bar +# -------------------------------------------------- in-method-range-2 +glob1 = 1 +class X: + # comment + + def f(self, b, c): + #? 11 text {'new_name': 'ab', 'until_line': 11, 'until_column': 10} + #foo + local1 = 3 + local2 = 4 + return local1 * glob1 * b + # bar +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +glob1 = 1 +class X: + # comment + + def ab(self, b): + #foo + local1 = 3 + local2 = 4 + return local1 * glob1 * b + # bar + + def f(self, b, c): + #? 11 text {'new_name': 'ab', 'until_line': 11, 'until_column': 10} + return self.ab(b) +# -------------------------------------------------- in-method-range-3 +glob1 = 1 +class X: + def f(self, b, c): + local1, local2 = 3, 4 + #foo + #? 11 text {'new_name': 'ab', 'until_line': 7, 'until_column': 29} + return local1 & glob1 & b + # bar + local2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +glob1 = 1 +class X: + def ab(self, local1, b): + return local1 & glob1 & b + + def f(self, b, c): + local1, local2 = 3, 4 + #foo + #? 11 text {'new_name': 'ab', 'until_line': 7, 'until_column': 29} + return self.ab(local1, b) + # bar + local2 +# -------------------------------------------------- in-method-no-param +glob1 = 1 +class X: + def f(): + #? 11 text {'new_name': 'ab', 'until_line': 5, 'until_column': 22} + return glob1 + 2 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +glob1 = 1 +class X: + def ab(): + return glob1 + 2 + + def f(): + #? 11 text {'new_name': 'ab', 'until_line': 5, 'until_column': 22} + return ab() +# -------------------------------------------------- random-return-1 +def x(): + #? 0 error {'new_name': 'ab', 'until_line': 5, 'until_column': 10} + if x: + return 1 + return 1 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Can only extract return statements if they are at the end. +# -------------------------------------------------- random-return-2 +def x(): + #? 0 error {'new_name': 'ab', 'until_line': 5, 'until_column': 10} + # + return + pass +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Can only extract return statements if they are at the end. +# -------------------------------------------------- random-yield-1 +def x(): + #? 0 error {'new_name': 'ab', 'until_line': 5, 'until_column': 10} + # + if (yield 1): + return + pass +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract yield statements. +# -------------------------------------------------- random-yield-2 +def x(): + #? 0 error {'new_name': 'ab', 'until_line': 4, 'until_column': 10} + # + try: + yield + finally: + pass +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract yield statements. diff --git a/test/refactor/extract_variable.py b/test/refactor/extract_variable.py new file mode 100644 index 00000000..772bee76 --- /dev/null +++ b/test/refactor/extract_variable.py @@ -0,0 +1,251 @@ +# -------------------------------------------------- simple-1 +def test(): + #? 35 text {'new_name': 'a'} + return test(100, (30 + b, c) + 1) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def test(): + #? 35 text {'new_name': 'a'} + a = (30 + b, c) + 1 + return test(100, a) +# -------------------------------------------------- simple-2 +def test(): + #? 25 text {'new_name': 'a'} + return test(100, (30 + b, c) + 1) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def test(): + #? 25 text {'new_name': 'a'} + a = 30 + b + return test(100, (a, c) + 1) +# -------------------------------------------------- simple-3 +#? 13 text {'new_name': 'zzx.x'} +test(100, {1 |1: 2 + 3}) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 13 text {'new_name': 'zzx.x'} +zzx.x = 1 |1 +test(100, {zzx.x: 2 + 3}) +# -------------------------------------------------- multiline-1 +def test(): + #? 30 text {'new_name': 'x'} + return test(1, (30 + b, c) + + 1) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def test(): + #? 30 text {'new_name': 'x'} + x = (30 + b, c) + + 1 + return test(1, x) +# -------------------------------------------------- multiline-2 +def test(): + #? 25 text {'new_name': 'x'} + return test(1, (30 + b, c) + + 1) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +def test(): + #? 25 text {'new_name': 'x'} + x = 30 + b + return test(1, (x, c) + + 1) +# -------------------------------------------------- for-param-error-1 +#? 10 error {'new_name': 'x'} +def test(p1): + return +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a name that defines something +# -------------------------------------------------- for-param-error-2 +#? 12 error {'new_name': 'x'} +def test(p1= 3): + return +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a "param" +# -------------------------------------------------- for-param-1 +#? 12 text {'new_name': 'x'} +def test(p1=20): + return +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 12 text {'new_name': 'x'} +x = 20 +def test(p1=x): + return +# -------------------------------------------------- for-something +#? 12 text {'new_name': 'x'} +def test(p1=20): + return +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 12 text {'new_name': 'x'} +x = 20 +def test(p1=x): + return +# -------------------------------------------------- class-inheritance-1 +#? 12 text {'new_name': 'x'} +class Foo(foo.Bar): + pass +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 12 text {'new_name': 'x'} +x = foo.Bar +class Foo(x): + pass +# -------------------------------------------------- class-inheritance-2 +#? 16 text {'new_name': 'x'} +class Foo(foo.Bar): + pass +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 16 text {'new_name': 'x'} +x = foo.Bar +class Foo(x): + pass +# -------------------------------------------------- keyword-pass +#? 12 error {'new_name': 'x'} +def x(): pass +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a "simple_stmt" +# -------------------------------------------------- keyword-continue +#? 5 error {'new_name': 'x'} +continue +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a "simple_stmt" +# -------------------------------------------------- keyword-None +if 1: + #? 4 text {'new_name': 'x'} + None +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +if 1: + #? 4 text {'new_name': 'x'} + x = None + x +# -------------------------------------------------- with-tuple +#? 4 text {'new_name': 'x'} +x + 1, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 4 text {'new_name': 'x'} +x = x + 1 +x, 3 +# -------------------------------------------------- range-1 +#? 4 text {'new_name': 'x', 'until_column': 9} +y + 1, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 4 text {'new_name': 'x', 'until_column': 9} +x = y + 1, 3 +x +# -------------------------------------------------- range-2 +#? 1 text {'new_name': 'x', 'until_column': 3} +y + 1, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 1 text {'new_name': 'x', 'until_column': 3} +x = y + 1 +x, 3 +# -------------------------------------------------- range-3 +#? 1 text {'new_name': 'x', 'until_column': 6} +y + 1, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 1 text {'new_name': 'x', 'until_column': 6} +x = y + 1 +x, 3 +# -------------------------------------------------- range-4 +#? 1 text {'new_name': 'x', 'until_column': 1} +y + 1, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 1 text {'new_name': 'x', 'until_column': 1} +x = y +x + 1, 3 +# -------------------------------------------------- addition-1 +#? 4 text {'new_name': 'x', 'until_column': 9} +z = y + 1 + 2+ 3, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 4 text {'new_name': 'x', 'until_column': 9} +x = y + 1 +z = x + 2+ 3, 3 +# -------------------------------------------------- addition-2 +#? 8 text {'new_name': 'x', 'until_column': 12} +z = y +1 + 2+ 3, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 8 text {'new_name': 'x', 'until_column': 12} +x = 1 + 2 +z = y +x+ 3, 3 +# -------------------------------------------------- addition-3 +#? 10 text {'new_name': 'x', 'until_column': 14} +z = y + 1 + 2+ 3, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 10 text {'new_name': 'x', 'until_column': 14} +x = 1 + 2+ 3 +z = y + x, 3 +# -------------------------------------------------- addition-4 +#? 13 text {'new_name': 'x', 'until_column': 17} +z = y + (1 + 2)+ 3, 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 13 text {'new_name': 'x', 'until_column': 17} +x = (1 + 2)+ 3 +z = y + x, 3 +# -------------------------------------------------- mult-add-1 +#? 8 text {'new_name': 'x', 'until_column': 11} +z = foo(y+1*2+3, 3) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 8 text {'new_name': 'x', 'until_column': 11} +x = y+1 +z = foo(x*2+3, 3) +# -------------------------------------------------- mult-add-2 +#? 12 text {'new_name': 'x', 'until_column': 15} +z = foo(y+1*2+3) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 12 text {'new_name': 'x', 'until_column': 15} +x = 2+3 +z = foo(y+1*x) +# -------------------------------------------------- mult-add-3 +#? 9 text {'new_name': 'x', 'until_column': 13} +z = (y+1*2+3) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 9 text {'new_name': 'x', 'until_column': 13} +x = (y+1*2+3) +z = x +# -------------------------------------------------- extract-weird-1 +#? 0 error {'new_name': 'x', 'until_column': 7} +foo = 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a "expr_stmt" +# -------------------------------------------------- extract-weird-2 +#? 0 error {'new_name': 'x', 'until_column': 5} +def x(): + foo = 3 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a "funcdef" +# -------------------------------------------------- extract-weird-3 +def x(): +#? 4 error {'new_name': 'x', 'until_column': 8} + if 1: + pass +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a "if_stmt" +# -------------------------------------------------- extract-weird-4 +#? 4 error {'new_name': 'x', 'until_column': 7} +x = foo = 4 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot extract a name that defines something +# -------------------------------------------------- keyword-None +#? 4 text {'new_name': 'x', 'until_column': 7} +yy = not foo or bar +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 4 text {'new_name': 'x', 'until_column': 7} +x = not foo +yy = x or bar +# -------------------------------------------------- augassign +yy = () +#? 6 text {'new_name': 'x', 'until_column': 10} +yy += 3, 4 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +yy = () +#? 6 text {'new_name': 'x', 'until_column': 10} +x = 3, 4 +yy += x +# -------------------------------------------------- if-else +#? 9 text {'new_name': 'x', 'until_column': 22} +yy = foo(a if y else b) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 9 text {'new_name': 'x', 'until_column': 22} +x = a if y else b +yy = foo(x) +# -------------------------------------------------- lambda +#? 8 text {'new_name': 'x', 'until_column': 17} +y = foo(lambda x: 3, 5) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +#? 8 text {'new_name': 'x', 'until_column': 17} +x = lambda x: 3 +y = foo(x, 5) diff --git a/test/refactor/import_tree/inline_mod.py b/test/refactor/import_tree/inline_mod.py new file mode 100644 index 00000000..0a1303e3 --- /dev/null +++ b/test/refactor/import_tree/inline_mod.py @@ -0,0 +1 @@ +inline_var = 5 + 3 diff --git a/test/refactor/import_tree/pkgx/__init__.py b/test/refactor/import_tree/pkgx/__init__.py new file mode 100644 index 00000000..85a0765a --- /dev/null +++ b/test/refactor/import_tree/pkgx/__init__.py @@ -0,0 +1,2 @@ +def pkgx(): + pass diff --git a/test/refactor/import_tree/pkgx/__init__.pyi b/test/refactor/import_tree/pkgx/__init__.pyi new file mode 100644 index 00000000..b47d393f --- /dev/null +++ b/test/refactor/import_tree/pkgx/__init__.pyi @@ -0,0 +1 @@ +def pkgx() -> int: ... diff --git a/test/refactor/import_tree/pkgx/mod.pyi b/test/refactor/import_tree/pkgx/mod.pyi new file mode 100644 index 00000000..76bc2c84 --- /dev/null +++ b/test/refactor/import_tree/pkgx/mod.pyi @@ -0,0 +1 @@ +from . import pkgx diff --git a/test/refactor/import_tree/pkgx/mod2.py b/test/refactor/import_tree/pkgx/mod2.py new file mode 100644 index 00000000..77968fc9 --- /dev/null +++ b/test/refactor/import_tree/pkgx/mod2.py @@ -0,0 +1 @@ +from .. import pkgx diff --git a/test/refactor/import_tree/some_mod.py b/test/refactor/import_tree/some_mod.py new file mode 100644 index 00000000..ccb0f5e6 --- /dev/null +++ b/test/refactor/import_tree/some_mod.py @@ -0,0 +1 @@ +foobar = 3 diff --git a/test/refactor/inline.py b/test/refactor/inline.py index c373be25..a506ec0e 100644 --- a/test/refactor/inline.py +++ b/test/refactor/inline.py @@ -1,18 +1,243 @@ -# --- simple +# -------------------------------------------------- no-name-error +#? 0 error +1 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +There is no name under the cursor +# -------------------------------------------------- multi-equal-error +def test(): + #? 4 error + a = b = 3 + return test(100, a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a statement with multiple definitions +# -------------------------------------------------- no-definition-error +#? 5 error +test(a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +No definition found to inline +# -------------------------------------------------- multi-names-error +#? 0 error +a, b[1] = 3 +test(a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a statement with multiple definitions +# -------------------------------------------------- addition-error +#? 0 error +a = 2 +a += 3 +test(a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a name with multiple definitions +# -------------------------------------------------- only-addition-error +#? 0 error +a += 3 +test(a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a statement with "+=" +# -------------------------------------------------- with-annotation +foobarb: int = 1 +#? 5 +test(foobarb) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,3 @@ +-foobarb: int = 1 + #? 5 +-test(foobarb) ++test(1) +# -------------------------------------------------- only-annotation-error +a: int +#? 5 error +test(a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a statement that is defined by an annotation +# -------------------------------------------------- builtin +import math +#? 7 error +math.cos +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline builtins/extensions +# -------------------------------------------------- module-error +from import_tree import inline_mod +#? 11 error +test(inline_mod) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline imports or modules +# -------------------------------------------------- module-works +from import_tree import inline_mod +#? 22 +test(x, inline_mod. inline_var.conjugate) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- import_tree/inline_mod.py ++++ import_tree/inline_mod.py +@@ -1,2 +1 @@ +-inline_var = 5 + 3 +--- inline.py ++++ inline.py +@@ -1,4 +1,4 @@ + from import_tree import inline_mod + #? 22 +-test(x, inline_mod. inline_var.conjugate) ++test(x, (5 + 3).conjugate) +# -------------------------------------------------- class +class A: pass +#? 5 error +test(A) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a class +# -------------------------------------------------- function +def foo(a): + return a + 1 +#? 5 error +test(foo(1)) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a function +# -------------------------------------------------- for-stmt +for x in []: + #? 9 error + test(x) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +Cannot inline a for_stmt +# -------------------------------------------------- simple def test(): #? 4 a = (30 + b, c) + 1 return test(100, a) -# +++ -def test(): - return test(100, (30 + b, c) + 1) - - -# --- simple +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,5 +1,4 @@ + def test(): + #? 4 +- a = (30 + b, c) + 1 +- return test(100, a) ++ return test(100, (30 + b, c) + 1) +# -------------------------------------------------- tuple if 1: #? 4 a = 1, 2 return test(100, a) -# +++ -if 1: - return test(100, (1, 2)) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,5 +1,4 @@ + if 1: + #? 4 +- a = 1, 2 +- return test(100, a) ++ return test(100, (1, 2)) +# -------------------------------------------------- multiplication-add-parens1 +a = 1+2 +#? 11 +test(100 * a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,3 @@ +-a = 1+2 + #? 11 +-test(100 * a) ++test(100 * (1+2)) +# -------------------------------------------------- multiplication-add-parens2 +a = 1+2 +#? 11 +(x, 100 * a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,3 @@ +-a = 1+2 + #? 11 +-(x, 100 * a) ++(x, 100 * (1+2)) +# -------------------------------------------------- multiplication-add-parens3 +x +a = 1+2 +#? 9 +(100 ** a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,5 +1,4 @@ + x +-a = 1+2 + #? 9 +-(100 ** a) ++(100 ** (1+2)) +# -------------------------------------------------- no-add-parens1 +x +a = 1+2 +#? 5 +test(a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,5 +1,4 @@ + x +-a = 1+2 + #? 5 +-test(a) ++test(1+2) +# -------------------------------------------------- no-add-parens2 +a = 1+2 +#? 9 +test(3, a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,3 @@ +-a = 1+2 + #? 9 +-test(3, a) ++test(3, 1+2) +# -------------------------------------------------- no-add-parens3 +a = 1|2 +#? 5 +(3, a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,3 @@ +-a = 1|2 + #? 5 +-(3, a) ++(3, 1|2) +# -------------------------------------------------- comment +a = 1 and 2 # foo +#? 9 +(3, 3 * a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,4 @@ +-a = 1 and 2 # foo ++ # foo + #? 9 +-(3, 3 * a) ++(3, 3 * (1 and 2)) +# -------------------------------------------------- semicolon +a = 1, 2 ; b = 3 +#? 9 +(3, 3 == a) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,4 @@ +-a = 1, 2 ; b = 3 ++ b = 3 + #? 9 +-(3, 3 == a) ++(3, 3 == (1, 2)) +# -------------------------------------------------- no-tree-name +a = 1 + 2 +#? 0 +a.conjugate +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- inline.py ++++ inline.py +@@ -1,4 +1,3 @@ +-a = 1 + 2 + #? 0 +-a.conjugate ++(1 + 2).conjugate diff --git a/test/refactor/rename.py b/test/refactor/rename.py index c717eb04..864032b8 100644 --- a/test/refactor/rename.py +++ b/test/refactor/rename.py @@ -3,15 +3,233 @@ Test coverage for renaming is mostly being done by testing `Script.get_references`. """ -# --- simple +# -------------------------------------------------- no-name +#? 0 error {'new_name': 'blabla'} +1 +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +There is no name under the cursor +# -------------------------------------------------- simple def test1(): - #? 7 blabla + #? 7 {'new_name': 'blabla'} test1() AssertionError return test1, test1.not_existing -# +++ -def blabla(): - blabla() - AssertionError - return blabla, blabla.not_existing - +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- rename.py ++++ rename.py +@@ -1,6 +1,6 @@ +-def test1(): ++def blabla(): + #? 7 {'new_name': 'blabla'} +- test1() ++ blabla() + AssertionError +- return test1, test1.not_existing ++ return blabla, blabla.not_existing +# -------------------------------------------------- var-not-found +undefined_var +#? 0 {'new_name': 'lala'} +undefined_var +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- rename.py ++++ rename.py +@@ -1,4 +1,4 @@ + undefined_var + #? 0 {'new_name': 'lala'} +-undefined_var ++lala +# -------------------------------------------------- different-scopes +def x(): + #? 7 {'new_name': 'v'} + some_var = 3 + some_var +def y(): + some_var = 3 + some_var +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- rename.py ++++ rename.py +@@ -1,7 +1,7 @@ + def x(): + #? 7 {'new_name': 'v'} +- some_var = 3 +- some_var ++ v = 3 ++ v + def y(): + some_var = 3 + some_var +# -------------------------------------------------- keyword-param1 +#? 22 {'new_name': 'lala'} +def mykeywordparam1(param1): + str(param1) +mykeywordparam1(1) +mykeywordparam1(param1=3) +mykeywordparam1(x, param1=2) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- rename.py ++++ rename.py +@@ -1,7 +1,7 @@ + #? 22 {'new_name': 'lala'} +-def mykeywordparam1(param1): +- str(param1) ++def mykeywordparam1(lala): ++ str(lala) + mykeywordparam1(1) +-mykeywordparam1(param1=3) +-mykeywordparam1(x, param1=2) ++mykeywordparam1(lala=3) ++mykeywordparam1(x, lala=2) +# -------------------------------------------------- keyword-param2 +def mykeywordparam2(param1): + str(param1) +mykeywordparam2(1) +mykeywordparam2(param1=3) +#? 22 {'new_name': 'lala'} +mykeywordparam2(x, param1=2) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- rename.py ++++ rename.py +@@ -1,7 +1,7 @@ +-def mykeywordparam2(param1): +- str(param1) ++def mykeywordparam2(lala): ++ str(lala) + mykeywordparam2(1) +-mykeywordparam2(param1=3) ++mykeywordparam2(lala=3) + #? 22 {'new_name': 'lala'} +-mykeywordparam2(x, param1=2) ++mykeywordparam2(x, lala=2) +# -------------------------------------------------- import +from import_tree.some_mod import foobar +#? 0 {'new_name': 'renamed'} +foobar +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- import_tree/some_mod.py ++++ import_tree/some_mod.py +@@ -1,2 +1,2 @@ +-foobar = 3 ++renamed = 3 +--- rename.py ++++ rename.py +@@ -1,4 +1,4 @@ +-from import_tree.some_mod import foobar ++from import_tree.some_mod import renamed + #? 0 {'new_name': 'renamed'} +-foobar ++renamed +# -------------------------------------------------- module +from import_tree import some_mod +#? 0 {'new_name': 'renamedm'} +some_mod +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +rename from import_tree/some_mod.py +rename to import_tree/renamedm.py +--- rename.py ++++ rename.py +@@ -1,4 +1,4 @@ +-from import_tree import some_mod ++from import_tree import renamedm + #? 0 {'new_name': 'renamedm'} +-some_mod ++renamedm +# -------------------------------------------------- import-not-found +#? 20 {'new_name': 'lala'} +import undefined_import +haha( undefined_import) +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- rename.py ++++ rename.py +@@ -1,4 +1,4 @@ + #? 20 {'new_name': 'lala'} +-import undefined_import +-haha( undefined_import) ++import lala ++haha( lala) +# -------------------------------------------------- in-package-with-stub +#? 31 {'new_name': 'renamedm'} +from import_tree.pkgx import pkgx +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +--- import_tree/pkgx/__init__.py ++++ import_tree/pkgx/__init__.py +@@ -1,3 +1,3 @@ +-def pkgx(): ++def renamedm(): + pass +--- import_tree/pkgx/__init__.pyi ++++ import_tree/pkgx/__init__.pyi +@@ -1,2 +1,2 @@ +-def pkgx() -> int: ... ++def renamedm() -> int: ... +--- import_tree/pkgx/mod.pyi ++++ import_tree/pkgx/mod.pyi +@@ -1,2 +1,2 @@ +-from . import pkgx ++from . import renamedm +--- rename.py ++++ rename.py +@@ -1,3 +1,3 @@ + #? 31 {'new_name': 'renamedm'} +-from import_tree.pkgx import pkgx ++from import_tree.pkgx import renamedm +# -------------------------------------------------- package-with-stub +#? 18 {'new_name': 'renamedp'} +from import_tree.pkgx +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +rename from import_tree/pkgx +rename to import_tree/renamedp +--- import_tree/pkgx/mod2.py ++++ import_tree/renamedp/mod2.py +@@ -1,2 +1,2 @@ +-from .. import pkgx ++from .. import renamedp +--- rename.py ++++ rename.py +@@ -1,3 +1,3 @@ + #? 18 {'new_name': 'renamedp'} +-from import_tree.pkgx ++from import_tree.renamedp +# -------------------------------------------------- weird-package-mix +if random_undefined_variable: + from import_tree.pkgx import pkgx +else: + from import_tree import pkgx +#? 4 {'new_name': 'rename'} +pkgx +# ++++++++++++++++++++++++++++++++++++++++++++++++++ +rename from import_tree/pkgx +rename to import_tree/rename +--- import_tree/pkgx/__init__.py ++++ import_tree/rename/__init__.py +@@ -1,3 +1,3 @@ +-def pkgx(): ++def rename(): + pass +--- import_tree/pkgx/__init__.pyi ++++ import_tree/rename/__init__.pyi +@@ -1,2 +1,2 @@ +-def pkgx() -> int: ... ++def rename() -> int: ... +--- import_tree/pkgx/mod.pyi ++++ import_tree/rename/mod.pyi +@@ -1,2 +1,2 @@ +-from . import pkgx ++from . import rename +--- import_tree/pkgx/mod2.py ++++ import_tree/rename/mod2.py +@@ -1,2 +1,2 @@ +-from .. import pkgx ++from .. import rename +--- rename.py ++++ rename.py +@@ -1,7 +1,7 @@ + if random_undefined_variable: +- from import_tree.pkgx import pkgx ++ from import_tree.rename import rename + else: +- from import_tree import pkgx ++ from import_tree import rename + #? 4 {'new_name': 'rename'} +-pkgx ++rename diff --git a/test/run.py b/test/run.py index 9fb3e3d8..731e9b4a 100755 --- a/test/run.py +++ b/test/run.py @@ -1,14 +1,11 @@ #!/usr/bin/env python """ -|jedi| is mostly being tested by what I would call "Blackbox Tests". These -tests are just testing the interface and do input/output testing. This makes a -lot of sense for |jedi|. Jedi supports so many different code structures, that -it is just stupid to write 200'000 unittests in the manner of -``regression.py``. Also, it is impossible to do doctests/unittests on most of -the internal data structures. That's why |jedi| uses mostly these kind of -tests. +|jedi| is mostly being tested by what I would call "integration tests". These +tests are testing type inference with the public API. This makes a +lot of sense for |jedi|. Also, it is hard to write doctests/unittests for +the internal data structures. -There are different kind of tests: +There are different kinds of tests: - completions / inference ``#?`` - goto: ``#!`` @@ -18,29 +15,31 @@ How to run tests? +++++++++++++++++ Jedi uses pytest_ to run unit and integration tests. To run tests, -simply run ``pytest``. You can also use tox_ to run tests for -multiple Python versions. +simply run ``pytest``. .. _pytest: http://pytest.org -.. _tox: http://testrun.org/tox -Integration test cases are located in ``test/completion`` directory -and each test case is indicated by either the comment ``#?`` (completions / -inference), ``#!`` (goto), or ``#<`` (references). +Most integration test cases are located in the ``test/completion`` directory +and each test case starts with one of these comments: + +- ``#?`` (completions / inference) +- ``#!`` (goto) +- ``#<`` (references) + There is also support for third party libraries. In a normal test run they are not being executed, you have to provide a ``--thirdparty`` option. -In addition to standard `-k` and `-m` options in pytest, you can use the -`-T` (`--test-files`) option to specify integration test cases to run. +In addition to pytest's ``-k`` and ``-m`` options, you can use the +``-T`` (``--test-files`) option to specify which test cases should run. It takes the format of ``FILE_NAME[:LINE[,LINE[,...]]]`` where ``FILE_NAME`` is a file in ``test/completion`` and ``LINE`` is a line -number of the test comment. Here is some recipes: +number of the test comment. Here are some examples: -Run tests only in ``basic.py`` and ``imports.py``:: +Run tests only in ``completion/basic.py`` and ``completion/imports.py``:: pytest test/test_integration.py -T basic.py -T imports.py -Run test at line 4, 6, and 8 in ``basic.py``:: +Run test at line 4, 6, and 8 in ``completion/basic.py``:: pytest test/test_integration.py -T basic.py:4,6,8 @@ -57,38 +56,30 @@ that you can start by running ``./run.py``. The above example could be run by:: ./run.py basic 4 6 8 50-80 The advantage of this runner is simplicity and more customized error reports. -Using both runners will help you to have a quicker overview of what's -happening. +Auto-Completion Tests ++++++++++++++++++++++ -Auto-Completion +Uses a comment to specify a test on the next line. The comment defines the +expected completions. The comment always begins with `#?`. The last row +symbolizes the cursor. For example:: + + #? ['upper'] + a = 'foo'; a.upp + +Inference Tests +++++++++++++++ -Uses comments to specify a test in the next line. The comment says which -results are expected. The comment always begins with `#?`. The last row -symbolizes the cursor. - -For example:: - - #? ['real'] - a = 3; a.rea - -Because it follows ``a.rea`` and a is an ``int``, which has a ``real`` -property. - -Inference -+++++++++ - -Inference tests use the same symbols like completion tests. This is -possible because the completion tests are defined with a list:: +Inference tests look very simliar. The difference is that inference tests don't +use brackets:: #? int() ab = 3; ab -Goto -++++ +Goto Tests +++++++++++ -Tests look like this:: +Goto Tests look like this:: abc = 1 #! ['abc=1'] @@ -100,13 +91,13 @@ describes the position of the test (otherwise it's just the end of line):: #! 2 ['abc=1'] abc -References -++++++++++ +Reference Tests ++++++++++++++++ Tests look like this:: abc = 1 - #< abc@1,0 abc@3,0 + #< (1,0), (3,0) abc """ import os @@ -124,7 +115,7 @@ import pytest import jedi from jedi import debug from jedi._compatibility import unicode, is_py3 -from jedi.api.classes import Definition +from jedi.api.classes import Name from jedi.api.completion import get_user_context from jedi import parser_utils from jedi.api.environment import get_default_environment, get_system_environment @@ -227,7 +218,7 @@ class IntegrationTestCase(BaseTestCase): def comparison(definition): suffix = '()' if definition.type == 'instance' else '' - return definition.desc_with_module + suffix + return definition.full_name + suffix def definition(correct, correct_start, path): should_be = set() @@ -244,7 +235,7 @@ class IntegrationTestCase(BaseTestCase): raise Exception('Could not resolve %s on line %s' % (match.string, self.line_nr - 1)) - should_be |= set(Definition(inference_state, r.name) for r in results) + should_be |= set(Name(inference_state, r.name) for r in results) debug.dbg('Finished getting types', color='YELLOW') # Because the objects have different ids, `repr`, then compare. @@ -411,7 +402,7 @@ def collect_dir_tests(base_dir, test_files, check_thirdparty=False): path = os.path.join(base_dir, f_name) if is_py3: - with open(path, encoding='utf-8') as f: + with open(path, encoding='utf-8', newline='') as f: source = f.read() else: with open(path) as f: @@ -440,7 +431,7 @@ Options: --pdb Enable pdb debugging on fail. -d, --debug Enable text output debugging (please install ``colorama``). --thirdparty Also run thirdparty tests (in ``completion/thirdparty``). - --env A Python version, like 2.7, 3.4, etc. + --env A Python version, like 2.7, 3.8, etc. """ if __name__ == '__main__': import docopt diff --git a/test/test_api/test_api.py b/test/test_api/test_api.py index 841151db..2172e189 100644 --- a/test/test_api/test_api.py +++ b/test/test_api/test_api.py @@ -141,7 +141,7 @@ def test_infer_on_generator(Script): def test_goto_definition_not_multiple(Script): """ - There should be only one Definition result if it leads back to the same + There should be only one result if it leads back to the same origin (e.g. instance method) """ diff --git a/test/test_api/test_call_signatures.py b/test/test_api/test_call_signatures.py index 9c27e517..d880dc4c 100644 --- a/test/test_api/test_call_signatures.py +++ b/test/test_api/test_call_signatures.py @@ -304,7 +304,7 @@ def test_builtins(Script): def test_signature_is_definition(Script): """ - Through inheritance, a signature is a sub class of Definition. + Through inheritance, a signature is a sub class of Name. Check if the attributes match. """ s = """class Spam(): pass\nSpam""" @@ -316,7 +316,8 @@ def test_signature_is_definition(Script): # Now compare all the attributes that a Signature must also have. for attr_name in dir(definition): dont_scan = ['defined_names', 'parent', 'goto_assignments', 'infer', - 'params', 'get_signatures', 'execute', 'goto'] + 'params', 'get_signatures', 'execute', 'goto', + 'desc_with_module'] if attr_name.startswith('_') or attr_name in dont_scan: continue diff --git a/test/test_api/test_classes.py b/test/test_api/test_classes.py index 9be2b07d..17265293 100644 --- a/test/test_api/test_classes.py +++ b/test/test_api/test_classes.py @@ -23,7 +23,7 @@ def test_basedefinition_type(Script, get_names): """ Return a list of definitions for parametrized tests. - :rtype: [jedi.api_classes.BaseDefinition] + :rtype: [jedi.api_classes.BaseName] """ source = dedent(""" import sys @@ -193,7 +193,7 @@ def test_hashlib_params(Script, environment): if environment.version_info < (3,): pytest.skip() - script = Script(source='from hashlib import sha256') + script = Script('from hashlib import sha256') c, = script.complete() sig, = c.get_signatures() assert [p.name for p in sig.params] == ['arg'] @@ -278,7 +278,7 @@ def test_parent_on_function(Script): code = 'def spam():\n pass' def_, = Script(code).goto(line=1, column=len('def spam')) parent = def_.parent() - assert parent.name == '' + assert parent.name == '__main__' assert parent.type == 'module' @@ -328,7 +328,7 @@ def test_parent_on_closure(Script): assert foo.parent().name == 'inner' assert foo.parent().parent().name == 'bar' assert foo.parent().parent().parent().name == 'Foo' - assert foo.parent().parent().parent().parent().name == '' + assert foo.parent().parent().parent().parent().name == '__main__' assert inner_func.parent().name == 'bar' assert inner_func.parent().parent().name == 'Foo' @@ -344,7 +344,7 @@ def test_parent_on_comprehension(Script): assert [name.name for name in ns] == ['spam', 'i'] - assert ns[0].parent().name == '' + assert ns[0].parent().name == '__main__' assert ns[0].parent().type == 'module' assert ns[1].parent().name == 'spam' assert ns[1].parent().type == 'function' @@ -375,7 +375,7 @@ def test_type_II(Script): """ -This tests the BaseDefinition.goto function, not the jedi +This tests the BaseName.goto function, not the jedi function. They are not really different in functionality, but really different as an implementation. """ diff --git a/test/test_api/test_completion.py b/test/test_api/test_completion.py index 545c0242..fed496ab 100644 --- a/test/test_api/test_completion.py +++ b/test/test_api/test_completion.py @@ -6,7 +6,7 @@ from textwrap import dedent import pytest from ..helpers import root_dir -from jedi.api.helpers import start_match, fuzzy_match +from jedi.api.helpers import _start_match, _fuzzy_match from jedi._compatibility import scandir @@ -92,11 +92,11 @@ def test_complete_expanduser(Script): non_dots = [p for p in possibilities if not p.name.startswith('.') and len(p.name) > 1] item = non_dots[0] line = "'~%s%s'" % (os.sep, item.name) - s = Script(line, line=1, column=len(line)-1) + s = Script(line) expected_name = item.name if item.is_dir(): expected_name += os.path.sep - assert expected_name in [c.name for c in s.completions()] + assert expected_name in [c.name for c in s.complete(column=len(line)-1)] def test_fake_subnodes(Script): @@ -312,62 +312,66 @@ def test_file_path_completions(Script, file, code, column, expected): assert [c.complete for c in comps] == expected +def test_file_path_should_have_completions(Script): + assert Script('r"').complete() # See GH #1503 + + _dict_keys_completion_tests = [ - ('ints[', 5, ['1', '50', Ellipsis]), - ('ints[]', 5, ['1', '50', Ellipsis]), - ('ints[1]', 5, ['1', '50', Ellipsis]), - ('ints[1]', 6, ['']), - ('ints[1', 5, ['1', '50', Ellipsis]), - ('ints[1', 6, ['']), + ('ints[', 5, ['1', '50', Ellipsis]), + ('ints[]', 5, ['1', '50', Ellipsis]), + ('ints[1]', 5, ['1', '50', Ellipsis]), + ('ints[1]', 6, ['']), + ('ints[1', 5, ['1', '50', Ellipsis]), + ('ints[1', 6, ['']), - ('ints[5]', 5, ['1', '50', Ellipsis]), - ('ints[5]', 6, ['0']), - ('ints[50', 5, ['1', '50', Ellipsis]), - ('ints[5', 6, ['0']), - ('ints[ 5', None, ['0']), - ('ints [ 5', None, ['0']), - ('ints[50', 6, ['0']), - ('ints[50', 7, ['']), + ('ints[5]', 5, ['1', '50', Ellipsis]), + ('ints[5]', 6, ['0']), + ('ints[50', 5, ['1', '50', Ellipsis]), + ('ints[5', 6, ['0']), + ('ints[ 5', None, ['0']), + ('ints [ 5', None, ['0']), + ('ints[50', 6, ['0']), + ('ints[50', 7, ['']), - ('strs[', 5, ["'asdf'", "'fbar'", "'foo'", Ellipsis]), - ('strs[]', 5, ["'asdf'", "'fbar'", "'foo'", Ellipsis]), - ("strs['", 6, ["asdf'", "fbar'", "foo'"]), - ("strs[']", 6, ["asdf'", "fbar'", "foo'"]), - ('strs["]', 6, ['asdf"', 'fbar"', 'foo"']), - ('strs["""]', 6, ['asdf', 'fbar', 'foo']), - ('strs["""]', 8, ['asdf"""', 'fbar"""', 'foo"""']), - ('strs[b"]', 8, []), - ('strs[r"asd', 10, ['f"']), - ('strs[r"asd"', 10, ['f']), - ('strs[R"asd', 10, ['f"']), - ('strs[ R"asd', None, ['f"']), - ('strs[\tR"asd', None, ['f"']), - ('strs[\nR"asd', None, ['f"']), - ('strs[f"asd', 10, []), - ('strs[br"""asd', 13, ['f"""']), - ('strs[br"""asd"""', 13, ['f']), - ('strs[ \t"""asd"""', 13, ['f']), + ('strs[', 5, ["'asdf'", "'fbar'", "'foo'", Ellipsis]), + ('strs[]', 5, ["'asdf'", "'fbar'", "'foo'", Ellipsis]), + ("strs['", 6, ["asdf'", "fbar'", "foo'"]), + ("strs[']", 6, ["asdf'", "fbar'", "foo'"]), + ('strs["]', 6, ['asdf"', 'fbar"', 'foo"']), + ('strs["""]', 6, ['asdf', 'fbar', 'foo']), + ('strs["""]', 8, ['asdf"""', 'fbar"""', 'foo"""']), + ('strs[b"]', 8, []), + ('strs[r"asd', 10, ['f"']), + ('strs[r"asd"', 10, ['f']), + ('strs[R"asd', 10, ['f"']), + ('strs[ R"asd', None, ['f"']), + ('strs[\tR"asd', None, ['f"']), + ('strs[\nR"asd', None, ['f"']), + ('strs[f"asd', 10, []), + ('strs[br"""asd', 13, ['f"""']), + ('strs[br"""asd"""', 13, ['f']), + ('strs[ \t"""asd"""', 13, ['f']), - ('strs["f', 7, ['bar"', 'oo"']), - ('strs["f"', 7, ['bar', 'oo']), - ('strs["f]', 7, ['bar"', 'oo"']), - ('strs["f"]', 7, ['bar', 'oo']), + ('strs["f', 7, ['bar"', 'oo"']), + ('strs["f"', 7, ['bar', 'oo']), + ('strs["f]', 7, ['bar"', 'oo"']), + ('strs["f"]', 7, ['bar', 'oo']), - ('mixed[', 6, [r"'a\\sdf'", '1', '1.1', "b'foo'", Ellipsis]), - ('mixed[1', 7, ['', '.1']), - ('mixed[Non', 9, ['e']), + ('mixed[', 6, [r"'a\\sdf'", '1', '1.1', "b'foo'", Ellipsis]), + ('mixed[1', 7, ['', '.1']), + ('mixed[Non', 9, ['e']), - ('casted["f', 9, ['3"', 'bar"', 'oo"']), - ('casted["f"', 9, ['3', 'bar', 'oo']), - ('casted["f3', 10, ['"']), - ('casted["f3"', 10, ['']), - ('casted_mod["f', 13, ['3"', 'bar"', 'oo"', 'ull"', 'uuu"']), + ('casted["f', 9, ['3"', 'bar"', 'oo"']), + ('casted["f"', 9, ['3', 'bar', 'oo']), + ('casted["f3', 10, ['"']), + ('casted["f3"', 10, ['']), + ('casted_mod["f', 13, ['3"', 'bar"', 'oo"', 'ull"', 'uuu"']), - ('keywords["', None, ['a"']), - ('keywords[Non', None, ['e']), - ('keywords[Fa', None, ['lse']), - ('keywords[Tr', None, ['ue']), - ('keywords[str', None, ['', 's']), + ('keywords["', None, ['a"']), + ('keywords[Non', None, ['e']), + ('keywords[Fa', None, ['lse']), + ('keywords[Tr', None, ['ue']), + ('keywords[str', None, ['', 's']), ] @@ -399,15 +403,15 @@ def test_dict_keys_completions(Script, added_code, column, expected, skip_pre_py def test_start_match(): - assert start_match('Condition', 'C') + assert _start_match('Condition', 'C') def test_fuzzy_match(): - assert fuzzy_match('Condition', 'i') - assert not fuzzy_match('Condition', 'p') - assert fuzzy_match('Condition', 'ii') - assert not fuzzy_match('Condition', 'Ciito') - assert fuzzy_match('Condition', 'Cdiio') + assert _fuzzy_match('Condition', 'i') + assert not _fuzzy_match('Condition', 'p') + assert _fuzzy_match('Condition', 'ii') + assert not _fuzzy_match('Condition', 'Ciito') + assert _fuzzy_match('Condition', 'Cdiio') def test_ellipsis_completion(Script): @@ -446,7 +450,7 @@ def test_completion_cache(Script, module_injector): @pytest.mark.parametrize('module', ['typing', 'os']) def test_module_completions(Script, module): - for c in Script('import {module}; {module}.'.format(module=module)).completions(): + for c in Script('import {module}; {module}.'.format(module=module)).complete(): # Just make sure that there are no errors c.type c.docstring() diff --git a/test/test_api/test_documentation.py b/test/test_api/test_documentation.py index fae16ddb..7f281b27 100644 --- a/test/test_api/test_documentation.py +++ b/test/test_api/test_documentation.py @@ -87,7 +87,7 @@ def test_version_info(Script): sys.version_info""")) - c, = s.completions() + c, = s.complete() assert c.docstring() == 'sys.version_info\n\nVersion information as a named tuple.' diff --git a/test/test_api/test_environment.py b/test/test_api/test_environment.py index 61f0d96b..ef3815a8 100644 --- a/test/test_api/test_environment.py +++ b/test/test_api/test_environment.py @@ -27,7 +27,7 @@ def test_find_system_environments(): @pytest.mark.parametrize( 'version', - ['2.7', '3.4', '3.5', '3.6', '3.7'] + ['2.7', '3.5', '3.6', '3.7'] ) def test_versions(version): try: @@ -118,9 +118,10 @@ def test_create_environment_executable(): assert environment.executable == sys.executable +@pytest.mark.skipif(sys.version_info[0] == 2, reason="Ignore Python 2, because EOL") def test_get_default_environment_from_env_does_not_use_safe(tmpdir, monkeypatch): fake_python = os.path.join(str(tmpdir), 'fake_python') - with open(fake_python, 'w') as f: + with open(fake_python, 'w', newline='') as f: f.write('') def _get_subprocess(self): diff --git a/test/test_api/test_full_name.py b/test/test_api/test_full_name.py index 5944ad72..1822a493 100644 --- a/test/test_api/test_full_name.py +++ b/test/test_api/test_full_name.py @@ -1,5 +1,5 @@ """ -Tests for :attr:`.BaseDefinition.full_name`. +Tests for :attr:`.BaseName.full_name`. There are three kinds of test: @@ -121,3 +121,10 @@ def test_param_name(Script): name, = Script('class X:\n def foo(bar): bar''').goto() assert name.type == 'param' assert name.full_name is None + + +def test_variable_in_func(Script): + names = Script('def f(): x = 3').get_names(all_scopes=True) + x = names[-1] + assert x.name == 'x' + assert x.full_name == '__main__.f.x' diff --git a/test/test_api/test_interpreter.py b/test/test_api/test_interpreter.py index e32bc72a..b5d601ce 100644 --- a/test/test_api/test_interpreter.py +++ b/test/test_api/test_interpreter.py @@ -123,23 +123,17 @@ def _assert_interpreter_complete(source, namespace, completions, def test_complete_raw_function(): from os.path import join - _assert_interpreter_complete('join("").up', - locals(), - ['upper']) + _assert_interpreter_complete('join("").up', locals(), ['upper']) def test_complete_raw_function_different_name(): from os.path import join as pjoin - _assert_interpreter_complete('pjoin("").up', - locals(), - ['upper']) + _assert_interpreter_complete('pjoin("").up', locals(), ['upper']) def test_complete_raw_module(): import os - _assert_interpreter_complete('os.path.join("a").up', - locals(), - ['upper']) + _assert_interpreter_complete('os.path.join("a").up', locals(), ['upper']) def test_complete_raw_instance(): @@ -148,31 +142,19 @@ def test_complete_raw_instance(): completions = ['time', 'timetz', 'timetuple'] if is_py3: completions += ['timestamp'] - _assert_interpreter_complete('(dt - dt).ti', - locals(), - completions) + _assert_interpreter_complete('(dt - dt).ti', locals(), completions) def test_list(): array = ['haha', 1] - _assert_interpreter_complete('array[0].uppe', - locals(), - ['upper']) - _assert_interpreter_complete('array[0].real', - locals(), - []) + _assert_interpreter_complete('array[0].uppe', locals(), ['upper']) + _assert_interpreter_complete('array[0].real', locals(), []) # something different, no index given, still just return the right - _assert_interpreter_complete('array[int].real', - locals(), - ['real']) - _assert_interpreter_complete('array[int()].real', - locals(), - ['real']) + _assert_interpreter_complete('array[int].real', locals(), ['real']) + _assert_interpreter_complete('array[int()].real', locals(), ['real']) # inexistent index - _assert_interpreter_complete('array[2].upper', - locals(), - ['upper']) + _assert_interpreter_complete('array[2].upper', locals(), ['upper']) def test_getattr(): @@ -186,9 +168,7 @@ def test_slice(): class Foo1: bar = [] baz = 'xbarx' - _assert_interpreter_complete('getattr(Foo1, baz[1:-1]).append', - locals(), - ['append']) + _assert_interpreter_complete('getattr(Foo1, baz[1:-1]).append', locals(), ['append']) def test_getitem_side_effects(): @@ -698,5 +678,5 @@ def bar(): def test_string_annotation(annotations, result, code): x = lambda foo: 1 x.__annotations__ = annotations - defs = jedi.Interpreter(code or 'x()', [locals()]).goto_definitions() + defs = jedi.Interpreter(code or 'x()', [locals()]).infer() assert [d.name for d in defs] == result diff --git a/test/test_api/test_project.py b/test/test_api/test_project.py index c3d016f3..a7d4846e 100644 --- a/test/test_api/test_project.py +++ b/test/test_api/test_project.py @@ -1,6 +1,9 @@ import os +import sys -from ..helpers import get_example_dir, set_cwd, root_dir +import pytest + +from ..helpers import get_example_dir, set_cwd, root_dir, test_dir from jedi import Interpreter from jedi.api import Project, get_default_project @@ -38,3 +41,113 @@ def test_load_save_project(tmpdir): loaded = Project.load(tmpdir.strpath) assert loaded.added_sys_path == ['/foo'] + + +@pytest.mark.parametrize( + 'string, full_names, kwargs', [ + ('test_load_save_project', ['test_api.test_project.test_load_save_project'], {}), + ('test_load_savep', [], dict(complete=True)), + ('test_load_save_p', ['test_api.test_project.test_load_save_project'], + dict(complete=True)), + ('test_load_save_p', ['test_api.test_project.test_load_save_project'], + dict(complete=True, all_scopes=True)), + + ('some_search_test_var', [], {}), + ('some_search_test_var', ['test_api.test_project.test_search.some_search_test_var'], + dict(all_scopes=True)), + ('some_search_test_var', ['test_api.test_project.test_search.some_search_test_var'], + dict(complete=True, all_scopes=True)), + + ('sample_int', ['helpers.sample_int'], {}), + ('sample_int', ['helpers.sample_int'], dict(all_scopes=True)), + ('sample_int.real', ['stub:builtins.int.real'], {}), + + ('class sample_int.real', [], {}), + ('foo sample_int.real', [], {}), + ('def sample_int.real', ['stub:builtins.int.real'], {}), + ('function sample_int.real', ['stub:builtins.int.real'], {}), + + # With modules + ('test_project.test_search', ['test_api.test_project.test_search'], {}), + ('test_project.test_searc', ['test_api.test_project.test_search'], dict(complete=True)), + ('test_api.test_project.test_search', ['test_api.test_project.test_search'], {}), + ('test_api.test_project.test_sear', ['test_api.test_project.test_search'], + dict(complete=True)), + + # With namespace + ('implicit_namespace_package.ns1.pkg', + ['examples.implicit_namespace_package.ns1.pkg'], {}), + ('implicit_namespace_package.ns1.pkg.ns1_file', + ['examples.implicit_namespace_package.ns1.pkg.ns1_file'], {}), + ('examples.implicit_namespace_package.ns1.pkg.ns1_file', + ['examples.implicit_namespace_package.ns1.pkg.ns1_file'], {}), + ('implicit_namespace_package.ns1.pkg.', + ['examples.implicit_namespace_package.ns1.pkg.ns1_file'], + dict(complete=True)), + ('implicit_namespace_package.', + ['examples.implicit_namespace_package.ns1', + 'examples.implicit_namespace_package.ns2'], + dict(complete=True)), + + # With stubs + ('with_python.module', ['examples.stub_packages.with_python.module'], {}), + ('with_python.modul', ['examples.stub_packages.with_python.module'], + dict(complete=True)), + ('no_python.foo', ['stub:examples.stub_packages.no_python.foo'], {}), + ('no_python.fo', ['stub:examples.stub_packages.no_python.foo'], + dict(complete=True)), + ('with_python-stubs.module', [], {}), + ('no_python-stubs.foo', [], {}), + # Both locations are given, because they live in separate folders (one + # suffixed with -stubs. + ('with_python', ['examples.stub_packages.with_python'], {}), + ('no_python', ['stub:examples.stub_packages.no_python'], {}), + # Completion stubs + ('stub_only', ['stub:completion.stub_folder.stub_only', + 'stub:examples.stub_packages.with_python.stub_only'], {}), + ('with_stub', ['completion.stub_folder.with_stub'], {}), + ('with_stub.in_with_stub_both', + ['completion.stub_folder.with_stub.in_with_stub_both'], {}), + ('with_stub.in_with_stub_python', + ['completion.stub_folder.with_stub.in_with_stub_python'], {}), + ('with_stub.in_with_stub_stub', + ['stub:completion.stub_folder.with_stub.in_with_stub_stub'], {}), + # Completion stubs: Folder + ('with_stub_folder', ['completion.stub_folder.with_stub_folder'], {}), + ('with_stub_folder.nested_with_stub', + ['completion.stub_folder.with_stub_folder.nested_with_stub'], {}), + ('nested_with_stub', + ['completion.stub_folder.stub_only_folder.nested_with_stub', + 'completion.stub_folder.with_stub_folder.nested_with_stub'], {}), + + # On sys path + ('sys.path', ['stub:sys.path'], {}), + ('json.dumps', ['json.dumps'], {}), # stdlib + stub + ('multiprocessing', ['multiprocessing'], {}), + ('multiprocessin', ['multiprocessing'], dict(complete=True)), + ] +) +@pytest.mark.skipif(sys.version_info < (3, 6), reason="Ignore Python 2, because EOL") +def test_search(string, full_names, kwargs, skip_pre_python36): + some_search_test_var = 1.0 + project = Project(test_dir) + if kwargs.pop('complete', False) is True: + defs = project.complete_search(string, **kwargs) + else: + defs = project.search(string, **kwargs) + assert [('stub:' if d.is_stub() else '') + d.full_name for d in defs] == full_names + + +@pytest.mark.parametrize( + 'string, completions, all_scopes', [ + ('SomeCl', ['ass'], False), + ('twic', [], False), + ('twic', ['e', 'e'], True), + ('test_load_save_p', ['roject'], False), + ] +) +@pytest.mark.skipif(sys.version_info < (3, 6), reason="Ignore Python 2, because EOL") +def test_complete_search(Script, string, completions, all_scopes, skip_pre_python36): + project = Project(test_dir) + defs = project.complete_search(string, all_scopes=all_scopes) + assert [d.complete for d in defs] == completions diff --git a/test/test_api/test_refactoring.py b/test/test_api/test_refactoring.py new file mode 100644 index 00000000..a40acd13 --- /dev/null +++ b/test/test_api/test_refactoring.py @@ -0,0 +1,64 @@ +import os +import sys +from textwrap import dedent + +import pytest + +import jedi + + +@pytest.fixture(autouse=True) +def skip_old_python(skip_pre_python36): + if sys.version_info < (3, 6): + pytest.skip() + + +@pytest.fixture() +def dir_with_content(tmpdir): + with open(os.path.join(tmpdir.strpath, 'modx.py'), 'w', newline='') as f: + f.write('import modx\nfoo\n') # self reference + return tmpdir.strpath + + +def test_rename_mod(Script, dir_with_content): + script = Script( + 'import modx; modx\n', + path=os.path.join(dir_with_content, 'some_script.py'), + project=jedi.Project(dir_with_content), + ) + refactoring = script.rename(line=1, new_name='modr') + refactoring.apply() + + p1 = os.path.join(dir_with_content, 'modx.py') + p2 = os.path.join(dir_with_content, 'modr.py') + expected_code = 'import modr\nfoo\n' + assert not os.path.exists(p1) + with open(p2, newline='') as f: + assert f.read() == expected_code + + assert refactoring.get_renames() == [(p1, p2)] + + assert refactoring.get_changed_files()[p1].get_new_code() == expected_code + + assert refactoring.get_diff() == dedent('''\ + rename from modx.py + rename to modr.py + --- modx.py + +++ modr.py + @@ -1,3 +1,3 @@ + -import modx + +import modr + foo + --- some_script.py + +++ some_script.py + @@ -1,2 +1,2 @@ + -import modx; modx + +import modr; modr + ''').format(dir=dir_with_content) + + +def test_rename_none_path(Script): + refactoring = Script('foo', path=None).rename(new_name='bar') + with pytest.raises(jedi.RefactoringError, match='on a Script with path=None'): + refactoring.apply() + assert refactoring diff --git a/test/test_api/test_search.py b/test/test_api/test_search.py new file mode 100644 index 00000000..0f38b560 --- /dev/null +++ b/test/test_api/test_search.py @@ -0,0 +1,92 @@ +import os +import sys + +import pytest + + +class SomeClass: + class SomeClass: + def twice(self, a): + something = os + return something + + def twice(self, b): + pass + + def some_function(): + pass + + +@pytest.mark.parametrize( + 'string, descriptions, kwargs', [ + # No completions + ('SomeClass', ['class SomeClass'], {}), + ('SomeClass', ['class SomeClass', 'class SomeClass.SomeClass'], dict(all_scopes=True)), + ('Some', [], dict(all_scopes=True)), + ('os', ['module os'], {}), + ('sys', ['module sys'], {}), + ('sys.path', ['statement sys.path'], {}), + ('sys.exit', ['function sys.exit'], {}), + ('something', [], {}), + ('something', ['statement SomeClass.SomeClass.twice.something'], dict(all_scopes=True)), + + # Completions + ('class Some', ['class SomeClass', 'class SomeClass.SomeClass'], + dict(all_scopes=True, complete=True)), + ('class Some', ['class SomeClass'], dict(complete=True)), + ('Some', ['class SomeClass', 'class SomeClass.SomeClass', + 'statement SomeClass.SomeClass.twice.something', + 'function SomeClass.some_function'], dict(all_scopes=True, complete=True)), + ('some', ['class SomeClass', 'class SomeClass.SomeClass', + 'statement SomeClass.SomeClass.twice.something', + 'function SomeClass.some_function'], dict(all_scopes=True, complete=True)), + + # Fuzzy + ('class Smelss', ['class SomeClass'], dict(complete=True, fuzzy=True)), + ('class Smelss', ['class SomeClass', 'class SomeClass.SomeClass'], + dict(complete=True, fuzzy=True, all_scopes=True)), + + # Nested + ('SomeClass.SomeClass', ['class SomeClass.SomeClass'], + dict(all_scopes=True)), + ('SomeClass.SomeClass.twice', ['function SomeClass.SomeClass.twice'], + dict(all_scopes=True)), + ('SomeClass.SomeClass.twice.__call__', ['function types.FunctionType.__call__'], + dict(all_scopes=True)), + ('SomeClass.SomeClass.twice.something', [], dict(all_scopes=True)), + ('SomeClass.twice', ['function SomeClass.twice', 'function SomeClass.SomeClass.twice'], + dict(all_scopes=True)), + + # Nested completions + ('SomeClass.twi', ['function SomeClass.twice', 'function SomeClass.SomeClass.twice'], + dict(all_scopes=True, complete=True)), + + # Fuzzy unfortunately doesn't work + ('SomeCl.twice', [], dict(all_scopes=True, complete=True, fuzzy=True)), + ] +) +def test_simple_search(Script, string, descriptions, kwargs, skip_pre_python36): + if sys.version_info < (3, 6): + pytest.skip() + + if kwargs.pop('complete', False) is True: + defs = Script(path=__file__).complete_search(string, **kwargs) + else: + defs = Script(path=__file__).search(string, **kwargs) + this_mod = 'test.test_api.test_search.' + assert [d.type + ' ' + d.full_name.replace(this_mod, '') for d in defs] == descriptions + + +@pytest.mark.parametrize( + 'string, completions, fuzzy, all_scopes', [ + ('SomeCl', ['ass'], False, False), + ('SomeCl', [None], True, False), + ('twic', [], False, False), + ('some_f', [], False, False), + ('twic', ['e', 'e'], False, True), + ('some_f', ['unction'], False, True), + ] +) +def test_complete_search(Script, string, completions, fuzzy, all_scopes): + defs = Script(path=__file__).complete_search(string, fuzzy=fuzzy, all_scopes=all_scopes) + assert [d.complete for d in defs] == completions diff --git a/test/test_api/test_unicode.py b/test/test_api/test_unicode.py index f7f7ec45..91090d1a 100644 --- a/test/test_api/test_unicode.py +++ b/test/test_api/test_unicode.py @@ -74,9 +74,3 @@ def test_wrong_encoding(Script, tmpdir): project = Project('.', sys_path=[tmpdir.strpath]) c, = Script('import x; x.foo', project=project).complete() assert c.name == 'foobar' - - -def test_encoding_parameter(Script): - name = u('hö') - s = Script(name.encode('latin-1'), encoding='latin-1') - assert s._module_node.get_code() == name diff --git a/test/test_compatibility.py b/test/test_compatibility.py index 01c9dae5..c3c5c0e5 100644 --- a/test/test_compatibility.py +++ b/test/test_compatibility.py @@ -5,22 +5,14 @@ from jedi._compatibility import highest_pickle_protocol def test_highest_pickle_protocol(): v = namedtuple('version', 'major, minor') assert highest_pickle_protocol([v(2, 7), v(2, 7)]) == 2 - assert highest_pickle_protocol([v(2, 7), v(3, 3)]) == 2 - assert highest_pickle_protocol([v(2, 7), v(3, 4)]) == 2 + assert highest_pickle_protocol([v(2, 7), v(3, 8)]) == 2 assert highest_pickle_protocol([v(2, 7), v(3, 5)]) == 2 assert highest_pickle_protocol([v(2, 7), v(3, 6)]) == 2 - assert highest_pickle_protocol([v(3, 3), v(2, 7)]) == 2 - assert highest_pickle_protocol([v(3, 3), v(3, 3)]) == 3 - assert highest_pickle_protocol([v(3, 3), v(3, 4)]) == 3 - assert highest_pickle_protocol([v(3, 3), v(3, 5)]) == 3 - assert highest_pickle_protocol([v(3, 3), v(3, 6)]) == 3 - assert highest_pickle_protocol([v(3, 4), v(2, 7)]) == 2 - assert highest_pickle_protocol([v(3, 4), v(3, 3)]) == 3 - assert highest_pickle_protocol([v(3, 4), v(3, 4)]) == 4 - assert highest_pickle_protocol([v(3, 4), v(3, 5)]) == 4 - assert highest_pickle_protocol([v(3, 4), v(3, 6)]) == 4 + assert highest_pickle_protocol([v(3, 8), v(2, 7)]) == 2 + assert highest_pickle_protocol([v(3, 8), v(3, 8)]) == 4 + assert highest_pickle_protocol([v(3, 8), v(3, 5)]) == 4 + assert highest_pickle_protocol([v(3, 8), v(3, 6)]) == 4 assert highest_pickle_protocol([v(3, 6), v(2, 7)]) == 2 - assert highest_pickle_protocol([v(3, 6), v(3, 3)]) == 3 - assert highest_pickle_protocol([v(3, 6), v(3, 4)]) == 4 + assert highest_pickle_protocol([v(3, 6), v(3, 8)]) == 4 assert highest_pickle_protocol([v(3, 6), v(3, 5)]) == 4 assert highest_pickle_protocol([v(3, 6), v(3, 6)]) == 4 diff --git a/test/test_deprecation.py b/test/test_deprecation.py index ed3603e6..5c4425c7 100644 --- a/test/test_deprecation.py +++ b/test/test_deprecation.py @@ -1,3 +1,18 @@ +# -*- coding: utf-8 -*- +import warnings + +import pytest + +from jedi._compatibility import u + + +@pytest.fixture(autouse=True) +def check_for_warning(recwarn): + warnings.simplefilter("always") + with pytest.warns(DeprecationWarning): + yield + + def test_goto_definitions(Script): int_, = Script('x = 1\nx, y\ny', line=2, column=0).goto_definitions() assert int_.name == 'int' @@ -25,3 +40,9 @@ def test_usages(Script): def test_call_signatures(Script): d1, = Script('abs(float(\nstr(', line=1, column=4).call_signatures() assert d1.name == 'abs' + + +def test_encoding_parameter(Script): + name = u('hö') + s = Script(name.encode('latin-1'), encoding='latin-1') + assert s._module_node.get_code() == name diff --git a/test/test_inference/test_buildout_detection.py b/test/test_inference/test_buildout_detection.py index aa70eca0..3d44c2d4 100644 --- a/test/test_inference/test_buildout_detection.py +++ b/test/test_inference/test_buildout_detection.py @@ -67,7 +67,7 @@ def test_path_from_sys_path_assignment(Script): import sys sys.path[0:0] = [ - '/usr/lib/python3.4/site-packages', + '/usr/lib/python3.8/site-packages', '/home/test/.buildout/eggs/important_package.egg' ] diff --git a/test/test_inference/test_extension.py b/test/test_inference/test_extension.py index a63af388..962a8f9c 100644 --- a/test/test_inference/test_extension.py +++ b/test/test_inference/test_extension.py @@ -33,9 +33,9 @@ def test_get_signatures_stdlib(Script): assert len(sigs[0].params) == 1 -# Check only on linux 64 bit platform and Python3.4. +# Check only on linux 64 bit platform and Python3.8. @pytest.mark.parametrize('load_unsafe_extensions', [False, True]) -@pytest.mark.skipif('sys.platform != "linux" or sys.maxsize <= 2**32 or sys.version_info[:2] != (3, 4)') +@pytest.mark.skipif('sys.platform != "linux" or sys.maxsize <= 2**32 or sys.version_info[:2] != (3, 8)') def test_init_extension_module(Script, load_unsafe_extensions): """ ``__init__`` extension modules are also packages and Jedi should understand @@ -45,10 +45,10 @@ def test_init_extension_module(Script, load_unsafe_extensions): This test was built by the module.c and setup.py combination you can find in the init_extension_module folder. You can easily build the - `__init__.cpython-34m.so` by compiling it (create a virtualenv and run + `__init__.cpython-38m.so` by compiling it (create a virtualenv and run `setup.py install`. - This is also why this test only runs on certain systems (and Python 3.4). + This is also why this test only runs on certain systems and Python 3.8. """ project = jedi.Project(get_example_dir(), load_unsafe_extensions=load_unsafe_extensions) diff --git a/test/test_inference/test_gradual/test_stubs.py b/test/test_inference/test_gradual/test_stubs.py index 1d82dd0b..6683474a 100644 --- a/test/test_inference/test_gradual/test_stubs.py +++ b/test/test_inference/test_gradual/test_stubs.py @@ -45,7 +45,7 @@ def test_infer_and_goto(Script, code, full_name, has_stub, has_python, way, kwargs, type_, options, environment): if environment.version_info < (3, 5): # We just don't care about much of the detailed Python 2 failures - # anymore, because its end-of-life soon. (same for 3.4) + # anymore, because its end-of-life soon. pytest.skip() if type_ == 'infer' and full_name == 'typing.Sequence' and environment.version_info >= (3, 7): diff --git a/test/test_inference/test_gradual/test_typeshed.py b/test/test_inference/test_gradual/test_typeshed.py index dd0e24dc..2c7bf9c6 100644 --- a/test/test_inference/test_gradual/test_typeshed.py +++ b/test/test_inference/test_gradual/test_typeshed.py @@ -24,9 +24,6 @@ def test_get_typeshed_directories(): dirs = get_dirs(PythonVersionInfo(2, 7)) assert dirs == transform({'stdlib/2and3', 'stdlib/2', 'third_party/2and3', 'third_party/2'}) - dirs = get_dirs(PythonVersionInfo(3, 4)) - assert dirs == transform({'stdlib/2and3', 'stdlib/3', 'third_party/2and3', 'third_party/3'}) - dirs = get_dirs(PythonVersionInfo(3, 5)) assert dirs == transform({'stdlib/2and3', 'stdlib/3', 'third_party/2and3', 'third_party/3'}) @@ -139,10 +136,13 @@ def test_math(Script): assert value -def test_type_var(Script): +def test_type_var(Script, environment): def_, = Script('import typing; T = typing.TypeVar("T1")').infer() assert def_.name == 'TypeVar' - assert def_.description == 'TypeVar = object()' + if environment.version_info.major == 2: + assert def_.description == 'TypeVar = object()' + else: + assert def_.description == 'class TypeVar' @pytest.mark.parametrize( diff --git a/test/test_inference/test_implicit_namespace_package.py b/test/test_inference/test_implicit_namespace_package.py index 0b3bee71..452375b3 100644 --- a/test/test_inference/test_implicit_namespace_package.py +++ b/test/test_inference/test_implicit_namespace_package.py @@ -8,7 +8,7 @@ from jedi import Project @pytest.fixture(autouse=True) def skip_not_supported_versions(environment): - if environment.version_info < (3, 4): + if environment.version_info < (3, 5): pytest.skip() @@ -58,9 +58,9 @@ def test_implicit_nested_namespace_package(Script): code = 'from implicit_nested_namespaces.namespace.pkg.module import CONST' project = Project('.', sys_path=[example_dir]) - script = Script(project=project, source=code, line=1, column=61) + script = Script(code, project=project) - result = script.infer() + result = script.infer(line=1, column=61) assert len(result) == 1 @@ -70,41 +70,41 @@ def test_implicit_nested_namespace_package(Script): def test_implicit_namespace_package_import_autocomplete(Script): - CODE = 'from implicit_name' + code = 'from implicit_name' project = Project('.', sys_path=[example_dir]) - script = Script(project=project, source=CODE) + script = Script(code, project=project) compl = script.complete() assert [c.name for c in compl] == ['implicit_namespace_package'] def test_namespace_package_in_multiple_directories_autocompletion(Script): - CODE = 'from pkg.' + code = 'from pkg.' sys_path = [get_example_dir('implicit_namespace_package', 'ns1'), get_example_dir('implicit_namespace_package', 'ns2')] project = Project('.', sys_path=sys_path) - script = Script(project=project, source=CODE) + script = Script(code, project=project) compl = script.complete() assert set(c.name for c in compl) == set(['ns1_file', 'ns2_file']) def test_namespace_package_in_multiple_directories_goto_definition(Script): - CODE = 'from pkg import ns1_file' + code = 'from pkg import ns1_file' sys_path = [get_example_dir('implicit_namespace_package', 'ns1'), get_example_dir('implicit_namespace_package', 'ns2')] project = Project('.', sys_path=sys_path) - script = Script(project=project, source=CODE) + script = Script(code, project=project) result = script.infer() assert len(result) == 1 def test_namespace_name_autocompletion_full_name(Script): - CODE = 'from pk' + code = 'from pk' sys_path = [get_example_dir('implicit_namespace_package', 'ns1'), get_example_dir('implicit_namespace_package', 'ns2')] project = Project('.', sys_path=sys_path) - script = Script(project=project, source=CODE) + script = Script(code, project=project) compl = script.complete() assert set(c.full_name for c in compl) == set(['pkg']) diff --git a/test/test_inference/test_imports.py b/test/test_inference/test_imports.py index 77b07be1..f5097f21 100644 --- a/test/test_inference/test_imports.py +++ b/test/test_inference/test_imports.py @@ -7,7 +7,7 @@ import os import pytest -from jedi.file_io import FileIO, KnownContentFileIO +from jedi.file_io import FileIO from jedi._compatibility import find_module_py33, find_module from jedi.inference import compiled from jedi.inference import imports @@ -259,7 +259,7 @@ def test_goto_following_on_imports(Script): def test_goto(Script): - sys, = Script("import sys", 1, 10).goto(follow_imports=True) + sys, = Script("import sys").goto(follow_imports=True) assert sys.type == 'module' @@ -344,23 +344,6 @@ def test_get_modules_containing_name(inference_state, path, goal, is_package): assert found_module.string_names == goal -@pytest.mark.parametrize( - ('path', 'base_names', 'is_package', 'names'), [ - ('/foo/bar.py', ('foo',), False, ('foo', 'bar')), - ('/foo/bar.py', ('foo', 'baz'), False, ('foo', 'baz', 'bar')), - ('/foo/__init__.py', ('foo',), True, ('foo',)), - ('/__init__.py', ('foo',), True, ('foo',)), - ('/foo/bar/__init__.py', ('foo',), True, ('foo',)), - ('/foo/bar/__init__.py', ('foo', 'bar'), True, ('foo', 'bar')), - ] -) -def test_load_module_from_path(inference_state, path, base_names, is_package, names): - file_io = KnownContentFileIO(path, '') - m = imports.load_module_from_path(inference_state, file_io, base_names) - assert m.is_package() == is_package - assert m.string_names == names - - @pytest.mark.parametrize( 'path', ('api/whatever/test_this.py', 'api/whatever/file')) @pytest.mark.parametrize('empty_sys_path', (False, True)) @@ -489,6 +472,6 @@ def test_relative_import_star(Script): from . import * furl.c """ - script = Script(source, 'export.py') + script = Script(source, path='export.py') assert script.complete(3, len("furl.c")) diff --git a/test/test_inference/test_namespace_package.py b/test/test_inference/test_namespace_package.py index 0867af85..67fa49b6 100644 --- a/test/test_inference/test_namespace_package.py +++ b/test/test_inference/test_namespace_package.py @@ -71,9 +71,7 @@ def test_nested_namespace_package(Script): sys_path = [example_dir] project = Project('.', sys_path=sys_path) - script = Script(project=project, source=code) - - result = script.infer(line=1, column=45) + result = Script(code, project=project).infer(line=1, column=45) assert len(result) == 1 @@ -82,7 +80,7 @@ def test_relative_import(Script, environment, tmpdir): """ Attempt a relative import in a very simple namespace package. """ - if environment.version_info < (3, 4): + if environment.version_info < (3, 5): pytest.skip() directory = get_example_dir('namespace_package_relative_import') diff --git a/test/test_inference/test_pyc.py b/test/test_inference/test_pyc.py index 35d51327..fec4c958 100644 --- a/test/test_inference/test_pyc.py +++ b/test/test_inference/test_pyc.py @@ -29,11 +29,11 @@ def pyc_project_path(tmpdir): path = tmpdir.strpath dummy_package_path = os.path.join(path, "dummy_package") os.mkdir(dummy_package_path) - with open(os.path.join(dummy_package_path, "__init__.py"), 'w'): + with open(os.path.join(dummy_package_path, "__init__.py"), 'w', newline=''): pass dummy_path = os.path.join(dummy_package_path, 'dummy.py') - with open(dummy_path, 'w') as f: + with open(dummy_path, 'w', newline='') as f: f.write(SRC) import compileall compileall.compile_file(dummy_path) @@ -56,6 +56,7 @@ def pyc_project_path(tmpdir): @pytest.mark.parametrize('load_unsafe_extensions', [False, True]) +@pytest.mark.skipif(sys.version_info[0] == 2, reason="Ignore Python 2, because EOL") def test_pyc(pyc_project_path, environment, load_unsafe_extensions): """ The list of completion must be greater than 2. diff --git a/test/test_integration.py b/test/test_integration.py index be447773..378fe893 100644 --- a/test/test_integration.py +++ b/test/test_integration.py @@ -4,6 +4,8 @@ import sys import pytest from . import helpers +from jedi.common.utils import indent_block +from jedi import RefactoringError def assert_case_equal(case, actual, desired): @@ -16,9 +18,11 @@ def assert_case_equal(case, actual, desired): """ assert actual == desired, """ Test %r failed. -actual = %s -desired = %s -""" % (case, actual, desired) +actual = +%s +desired = +%s +""" % (case, indent_block(str(actual)), indent_block(str(desired))) def assert_static_analysis(case, actual, desired): @@ -56,15 +60,25 @@ def test_static_analysis(static_analysis_case, environment): static_analysis_case.run(assert_static_analysis, environment) -def test_refactor(refactor_case): +def test_refactor(refactor_case, skip_pre_python36, environment): """ Run refactoring test case. :type refactor_case: :class:`.refactor.RefactoringCase` """ - if 0: - # TODO Refactoring is not relevant at the moment, it will be changed - # significantly in the future, but maybe we can use these tests: - refactor_case.run() - assert_case_equal(refactor_case, - refactor_case.result, refactor_case.desired) + if sys.version_info < (3, 6): + pytest.skip() + + desired_result = refactor_case.get_desired_result() + if refactor_case.type == 'error': + with pytest.raises(RefactoringError) as e: + refactor_case.refactor(environment) + assert e.value.args[0] == desired_result.strip() + elif refactor_case.type == 'text': + refactoring = refactor_case.refactor(environment) + assert not refactoring.get_renames() + text = ''.join(f.get_new_code() for f in refactoring.get_changed_files().values()) + assert_case_equal(refactor_case, text, desired_result) + else: + diff = refactor_case.refactor(environment).get_diff() + assert_case_equal(refactor_case, diff, desired_result) diff --git a/test/test_parso_integration/test_basic.py b/test/test_parso_integration/test_basic.py index efedc827..f2831dc5 100644 --- a/test/test_parso_integration/test_basic.py +++ b/test/test_parso_integration/test_basic.py @@ -70,7 +70,7 @@ def test_add_to_end(Script): " self." def complete(code, line=None, column=None): - script = Script(code, 'example.py') + script = Script(code, path='example.py') assert script.complete(line, column) complete(a, 7, 12) diff --git a/tox.ini b/tox.ini index d990d712..dfa023c2 100644 --- a/tox.ini +++ b/tox.ini @@ -5,7 +5,6 @@ extras = testing deps = # for testing the typing module py27: typing - py34: typing # numpydoc for typing scipy stack numpydoc # sphinx, a dependency of numpydoc, dropped Python 2 support in version 2.0 @@ -23,7 +22,6 @@ setenv = # To test Jedi in different versions than the same Python version, set a # different test environment. env27: JEDI_TEST_ENVIRONMENT=27 - env34: JEDI_TEST_ENVIRONMENT=34 env35: JEDI_TEST_ENVIRONMENT=35 env36: JEDI_TEST_ENVIRONMENT=36 env37: JEDI_TEST_ENVIRONMENT=37