Dave Halter
e371b670f5
Remove a comprehension hack in the parser that would have made the diff parsers world hell.
2016-09-22 18:26:09 +02:00
Dave Halter
959f7b5e00
We don't need to reset the last failed start_pos anymore, because this is something that was necessary with the fucked up old parser.
2016-09-18 00:52:22 +02:00
Dave Halter
885cf62a12
Remove the position_modifier from the parser.
2016-09-18 00:50:31 +02:00
Dave Halter
c764976ef2
Merge branch 'remove_names_dicts' into diff
2016-09-11 13:24:11 +02:00
Dave Halter
7667cba17e
Remove old indent/dedent usages. Now they are not needed anymore.
2016-09-11 13:20:24 +02:00
Dave Halter
1226962922
Remove dedents from the parser tree. No need for them.
2016-09-11 13:03:29 +02:00
Dave Halter
42e5777620
Some progress and bugfixes.
2016-08-26 12:47:02 +02:00
Dave Halter
1e5ad467d3
Start debugging the beast.
2016-08-23 18:24:58 +02:00
Dave Halter
b9040870c0
Some ideas for a diff parser.
2016-08-14 00:23:40 +02:00
Dave Halter
5f064a2a0a
Add a way to get the line in a BaseDefinition. Fixes #518 .
2016-07-31 20:37:48 +02:00
Dave Halter
142f6652b5
Move toward ParserWithRecovery for the completion context.
...
It was simply not possible to do it with the normal parser, because of dedents.
2016-07-28 23:14:24 +02:00
Dave Halter
ff47fab62a
Remove Whitespace class and replace it with Newline and Endmarker.
2016-07-24 17:16:36 +02:00
Dave Halter
e5f09e1c7d
Fix an issue with end_pos of a module.
2016-07-08 00:03:52 +02:00
Dave Halter
adcc1c2b51
Don't delete ErrorNode names. They are part of the parser now.
...
Fixes #594 and possibly also #590 and #579 .
2016-07-07 18:33:45 +02:00
Dave Halter
9245181a8c
Some python 2.7 (and 3.3) compatibility improvements.
2016-07-03 02:54:21 +02:00
Dave Halter
80aa9ad079
A small python 2 bugfix.
2016-06-22 00:45:14 +02:00
Dave Halter
6853bd70f4
Adding a token in pgen should have the same signature that the tokenizer uses.
2016-06-14 18:09:31 +02:00
Dave Halter
01ddacfec4
Generalize the tuple order of tokens.
2016-06-14 08:57:38 +02:00
Dave Halter
118ba7d833
A lot of stuff is not needed anymore, because of the recent refactorings.
2016-06-14 08:31:36 +02:00
Dave Halter
27f05de3b7
Fix a few more issues, mostly with the fast parser.
2016-06-13 18:21:17 +02:00
Dave Halter
82c76fa689
Merge with the linter branch (especially the changes of pep484.
2016-06-07 13:51:25 +02:00
Dave Halter
87a75fdfe4
Remove old unused code.
2016-06-06 09:27:43 +02:00
Dave Halter
dd85fc6ffd
Add error token in a normal way to the syntax tree as ErrorLeaf.
2016-06-05 14:49:57 +02:00
Dave Halter
5edcf47512
Break Interpreter completion even more in favor of a better solution in the future.
2016-06-03 19:31:42 +02:00
Dave Halter
0c7894b3e6
Fix a few fast parser tests.
2016-06-02 08:24:52 +02:00
Dave Halter
ad8d730a57
More test fixes.
2016-05-31 01:12:07 +02:00
Dave Halter
4f6368e7eb
Now ErrorLeaf and ErrorNode are part of the syntax tree. This makes probably sense. The documentation will follow once it's clear how they will shape out.
2016-05-30 00:34:58 +02:00
Dave Halter
daa68b66ad
Fix a few issues caused by the refactoring.
2016-05-29 19:49:35 +02:00
Dave Halter
feef45f4bb
Fixed all on_import tests.
2016-05-29 12:08:53 +02:00
Dave Halter
4714b464a6
Further import completion improvements.
2016-05-28 02:08:43 +02:00
Dave Halter
cbba314286
Progress and actually passing a few tests.
2016-05-26 00:10:54 +02:00
Dave Halter
d4a10929e2
Starting to create a way of how context sensitive completions can be made.
...
This involves playing heavily with the parser pgen2. We use its stack to check for all possible tokens/keywords.
2016-05-23 18:11:44 +02:00
Dave Halter
c09a916ab5
Didn't load grammar for Python 2.6 correctly.
2016-01-27 17:52:42 -02:00
Dave Halter
2a691eefff
Including pgen2 tests from the cpython repo.
2015-12-26 03:15:09 +01:00
Dave Halter
ab5d0ed72b
Starting with Python 3.4 from is not a token that always is a "new" statement.
2015-12-26 02:47:22 +01:00
Dave Halter
eb2e41f771
Grammar versioning has now a smoother interface.
2015-12-25 19:30:25 +01:00
Dave Halter
a373e34229
The parser without error recovery raises an error now if he's not able to parse something.
2015-12-25 18:53:05 +01:00
Dave Halter
c15551ccc1
Errortokens should also make the parser fail in the normal parser.
2015-12-20 23:11:52 +01:00
Dave Halter
c4906e0e3f
Rework the parser so we can use arbitrary start nodes of the syntax.
...
This also includes a rework for error recovery in the parser. This is now just possible for file_input parsing, which means for full files.
Includes also a refactoring of the tokenizer. No more do we have to add an additional newline, because it now works correctly (removes certain confusion.
2015-12-20 22:25:41 +01:00
Dave Halter
ab91cfa3b5
Fix: print_stmt was not actually cared for in Python 2.7, #662 .
2015-12-15 13:08:37 +01:00
Dave Halter
10df0f933f
Remove the strange check in the parser to always create expr_stmt and file_input.
2015-03-05 15:30:07 +01:00
Dave Halter
9c2e73d460
Add syntax errors to the parser.
2015-03-04 17:12:51 +01:00
Dave Halter
3a5b2d396e
Failed statements should not lead to parser fails.
2015-02-22 20:29:22 +01:00
Dave Halter
0c1bbf78e2
Rename SubModule to Module, because that's a more fitting description. There were reasons for the name before the new fast parser, but those don't exist anymore.
2015-02-05 14:16:43 +01:00
Dave Halter
109fdc53e0
Fix the remaining fast parser issues.
2015-02-05 01:13:00 +01:00
Dave Halter
fdfe17ada5
Import the token IDs directly, this way we minimize lookups.
2015-02-05 00:44:01 +01:00
Dave Halter
dce952aec6
Fix an issue with omited dedents in the parser.
2015-02-05 00:11:12 +01:00
Dave Halter
e23e354fe8
Simplified the line splitting and with that a few other things in the fast parser.
2015-02-03 22:22:57 +01:00
Dave Halter
66dfa59286
Fix some endmarker prefix issues in the fast parser.
2015-02-03 22:09:55 +01:00
Dave Halter
f9fe6b47eb
Fix error statement stacks positions.
2015-02-02 10:43:47 +01:00
