Dave Halter
d4a10929e2
Starting to create a way of how context sensitive completions can be made.
...
This involves playing heavily with the parser pgen2. We use its stack to check for all possible tokens/keywords.
2016-05-23 18:11:44 +02:00
Dave Halter
c4906e0e3f
Rework the parser so we can use arbitrary start nodes of the syntax.
...
This also includes a rework for error recovery in the parser. This is now just possible for file_input parsing, which means for full files.
Includes also a refactoring of the tokenizer. No more do we have to add an additional newline, because it now works correctly (removes certain confusion.
2015-12-20 22:25:41 +01:00
Dave Halter
265e6b2c35
Change parser and api to use tree instead of pr.
2015-04-27 23:38:48 +02:00
Dave Halter
48392a7dac
Fix some issues in call signatures.
2015-02-24 16:55:33 +01:00
Dave Halter
4f2d4992da
Fix an mro resolution issue.
2015-02-23 19:04:35 +01:00
Dave Halter
aebeafccc4
Rewrite last newlines in the fast parser to get correct get_code outputs even with the fast parser.
2015-02-23 13:36:43 +01:00
Dave Halter
489ea8fc83
Replace set_parser with direct ParserNode instance calls.
2015-02-23 13:10:40 +01:00
Dave Halter
2fcb1b9b65
Fast parser fix.
2015-02-23 01:00:17 +01:00
Dave Halter
3ec96b25cc
Issue with backslashes again in the fast parser.
2015-02-21 18:07:21 +01:00
Dave Halter
0b5a509e83
Small correction: mixed up a re.match and re.search.
2015-02-20 00:48:05 +01:00
Dave Halter
ce96af5e04
Fix an issue with open parentheses and function definitions right after. The fast parser should behave like the normal one and just ignore the open brackets.
2015-02-19 11:02:11 +01:00
Dave Halter
0e73bf7d80
Account for code parts that were not parsed in the fast parser.
2015-02-19 01:42:13 +01:00
Dave Halter
39bf9f426b
Handle backslash escaping.
2015-02-18 17:32:34 +01:00
Dave Halter
595da50ab8
The fast parser splitting now also checks for parentheses levels, because without that, sometimes we split in very strange positions, while ignoring others.
2015-02-18 13:49:03 +01:00
Dave Halter
506d602795
Fix multi line param issues in the fast parser.
2015-02-17 15:24:49 +01:00
Dave Halter
7663703989
Fix issues with multi line for loops in the fast parser.
2015-02-17 14:57:00 +01:00
Dave Halter
4d9608ea6f
Check more precisely for flow keywords.
2015-02-16 16:04:48 +01:00
Dave Halter
e1c28d2c3f
variables starting with 'class' and 'def' should not slow down the parser, changed the check to 'class ' and 'def '.
2015-02-16 10:08:22 +01:00
Dave Halter
db31e0e37d
The fast parser works now faster in case of for flows with a simple_stmt after.
2015-02-14 18:57:04 +01:00
Dave Halter
2dd08594fc
Simplify the indent calculation in the fast parser.
2015-02-05 14:37:24 +01:00
Dave Halter
0c1bbf78e2
Rename SubModule to Module, because that's a more fitting description. There were reasons for the name before the new fast parser, but those don't exist anymore.
2015-02-05 14:16:43 +01:00
Dave Halter
c689573b0b
Removed the line_offset from tokenize, we have better ways to modify positions, now.
2015-02-05 14:00:58 +01:00
Dave Halter
59cf1bce5d
Delete legacy code from the fast parser.
2015-02-05 13:47:35 +01:00
Dave Halter
dce952aec6
Fix an issue with omited dedents in the parser.
2015-02-05 00:11:12 +01:00
Dave Halter
e1c623d3f3
Python 2 compatibility.
2015-02-04 17:09:18 +01:00
Dave Halter
e23e354fe8
Simplified the line splitting and with that a few other things in the fast parser.
2015-02-03 22:22:57 +01:00
Dave Halter
66dfa59286
Fix some endmarker prefix issues in the fast parser.
2015-02-03 22:09:55 +01:00
Dave Halter
6cdfecb541
Fix a number of issues in the fast parser around functions with only one statement (no suite) and wrong indentations).
2015-02-02 15:03:57 +01:00
Dave Halter
f9fe6b47eb
Fix error statement stacks positions.
2015-02-02 10:43:47 +01:00
Dave Halter
c58cdbbf9b
Fix an issue that comes from a combination of property/__slots__/pickle
2015-02-02 00:45:17 +01:00
Dave Halter
9a0f1363e3
Start removing the print statements that were used for debugging.
2015-02-01 02:32:52 +01:00
Dave Halter
bc118e8047
Simplify the fast parser tokenizer more. Now it is more readable and less buggy (+bugfixes).
2015-01-31 20:09:44 +01:00
Dave Halter
1826f432c8
Fix an issue in the fast parser splitting.
2015-01-30 15:17:38 +01:00
Dave Halter
413da3b790
Remove the line_offset calculation. We can now also remove it from tokenize. With the position_modifier we have enough tools to change a position, we don't need to do that in tokenize.py.
2015-01-29 17:57:01 +01:00
Dave Halter
cf1b2ff54b
Function tests now pass with the fast parser.
2015-01-29 15:47:38 +01:00
Dave Halter
a221eee02c
Fix more issues in the fast parser.
2015-01-29 15:38:38 +01:00
Dave Halter
0a537c05c4
Fix an issue with Function/Flow combination in the fast parser.
2015-01-29 02:24:11 +01:00
Dave Halter
dde0e9c7c6
Fix for loop issues in the fast parser.
2015-01-29 01:36:16 +01:00
Dave Halter
e412694fa2
Fix issues with flows in the fast parser.
2015-01-28 17:06:18 +01:00
Dave Halter
b8c63f366c
FastModule seems to be compatible now with the normal Module, because it inherits from it and makes some minor modifications in some cases.
2015-01-28 15:11:53 +01:00
Dave Halter
d0589430bb
FastModule should inherit from SubModule, because it has almost all the same properties.
2015-01-28 14:59:00 +01:00
Dave Halter
6ec89e6785
Fix issues with flows.
2015-01-28 13:03:57 +01:00
Dave Halter
62e45aa42b
Fix issues with the new newline end_pos positions.
2015-01-27 02:21:05 +01:00
Dave Halter
cdae250b36
code -> source and also care for added newlines in the fast parser.
2015-01-26 22:01:39 +01:00
Dave Halter
a8943b8a80
Get the position modifiers right.
2015-01-24 20:42:28 +01:00
Dave Halter
446f5b9018
Fix issues with the right count of parsers used.
2015-01-24 20:19:03 +01:00
Dave Halter
4d6afd3c99
Fix fast parser tests.
2015-01-24 00:06:16 +01:00
Dave Halter
8569651bf4
Fast parser simplifications and bug fixes.
2015-01-21 18:34:22 +01:00
Dave Halter
91ab1d0ecd
Fix an issue in the fast parser that caused stuff to be parsed always.
2015-01-21 02:03:06 +01:00
Dave Halter
7188105dc7
The fast parser is now in a more readable shape.
2015-01-19 16:21:25 +01:00
