Rework the parser so we can use arbitrary start nodes of the syntax.

This also includes a rework for error recovery in the parser. This is now just possible for file_input parsing, which means for full files.
Includes also a refactoring of the tokenizer. No more do we have to add an additional newline, because it now works correctly (removes certain confusion.
This commit is contained in:
Dave Halter
2015-12-20 22:21:47 +01:00
parent 9a93d599da
commit c4906e0e3f
22 changed files with 246 additions and 198 deletions

View File

@@ -5,14 +5,14 @@ import sys
import pytest
from jedi._compatibility import unicode
from jedi.parser import Parser, load_grammar
from jedi.parser import ParserWithRecovery, load_grammar
from jedi.evaluate import sys_path, Evaluator
def test_paths_from_assignment():
def paths(src):
grammar = load_grammar()
stmt = Parser(grammar, unicode(src)).module.statements[0]
stmt = ParserWithRecovery(grammar, unicode(src)).module.statements[0]
return set(sys_path._paths_from_assignment(Evaluator(grammar), stmt))
assert paths('sys.path[0:0] = ["a"]') == set(['a'])