VimPlug/jedi-fork
1
0
Fork 0
forked from VimPlug/jedi
Code Pull Requests Activity

new parsing docstring

Browse Source
This commit is contained in:
David Halter
2013-02-05 00:02:29 +01:00
parent 057ed6bda2
commit 3678f235c0
1 changed files with 18 additions and 30 deletions
Download Patch File Download Diff File Expand all files Collapse all files

48
jedi/parsing.py
View File

@@ -4,45 +4,33 @@ This module is split in two parts:
- the Python parser itself - the Python parser itself
- classes for the parser to save the information - classes for the parser to save the information
The parser tries to represent the available Python code in an easy to read The ``Parser`` tries to represent the available Python code in an easy to read
format. The Python module ``tokenize`` is a very important part in the parser, format. The Python module ``tokenize`` is a very important part in the
because it splits the code into different words. ``Parser``, because it splits the code into different words. Sometimes it looks
a bit messy. Sorry for that! You might ask now: "Why didn't you use the ``ast``
module for this? Well, ``ast`` does a very good job understanding proper Python
code, but fails to work if there's only one single line of broken code.
The classes are not very hard to understand. They are being named like you The classes are not very hard to understand. They are being named like you
would call them. would call them: ``Import``, ``Class``, etc.
A very central class is ``Scope``. It is not used direcdtly by the parser, but A very central class is ``Scope``. It is not used directly by the parser, but
inherited. It's used by ``Function``, ``Class``, ``Flow``, etc. A ``Scope`` may inherited. It's used by ``Function``, ``Class``, ``Flow``, etc. A ``Scope`` may
have ``subscopes``, ``imports`` and ``statements``. The entire parser is based have ``subscopes``, ``imports`` and ``statements``. The entire parser is based
on scopes, because they also stand for indentation. on scopes, because they also stand for indentation.
There's one important optimization that needs to be known: Statements are not
being parsed completely. ``Statement`` is just a representation of the tokens
within the statement. This lowers memory usage and cpu time and reduces the
complexity of the ``Parser`` (there's another parser sitting inside
``Statement``, which produces ``Array`` and ``Call``).
Another strange thing about the parser is that ``Array`` is two dimensional.
This has been caused by the fact that each array element can be defined by
operations: ``[1, 2+3]``. So I chose to use a second dimension for ``2+3``. In
the future it might be useful to use Statements there, too. This is also how
``Param`` works. Every single ``Param`` is a ``Statement``.
In my opinion there's only one strange thing about the parser:
- The Array class is two dimensional. This is mainly strange because
speed optimization array/call
parses python code, with the goal of a good representation of
the code within a tree structure. Variables, Classes and Functions are defined
within this tree structure, containing their exact locations in the code.
It is also a primary goal to work with code which contains syntax errors.
This behaviour may be used to refactor, modify, search and complete code.
**The structure of the following script:**
All these objects have ``Name``. ``Call`` and ``Array`` are used as detail
objects of a statement.
All those classes are being generated by PyFuzzyParser, which takes python text
as input and ignores just all the non-python stuff. Basically you could feed it
a perl script, and it should still work (which means throw no error).
.. todo:: remove docstr params from Scope.__init__() .. todo:: remove docstr params from Scope.__init__()
""" """