Dave Halter
c7c464e5e9
Avoid nasty side effects in creation of Node
...
This issue led to bugs in Jedi, because Jedi used the nodes in a wrong way.
2019-01-01 23:35:20 +01:00
Dave Halter
29325d3052
Make parso errors even more informative
2018-12-31 11:47:02 +01:00
Dave Halter
750b8af37b
Fix diff parser get_last_line calculation
2018-12-31 01:25:11 +01:00
Dave Halter
0126a38bd1
Fix graph asserting for error indents
2018-12-30 18:20:55 +01:00
Dave Halter
c2985c111e
Better checks for checking valid graphs
2018-12-30 16:34:11 +01:00
Dave Halter
45f9d4b204
Create better ways for debugging the diff parser
2018-12-30 16:03:54 +01:00
Dave Halter
f99fe6ad21
Fix diff-parser: Copying parts of if else should not lead to the whole thing being copied
2018-12-30 15:25:17 +01:00
Dave Halter
a64c32bb2a
Reenable diff parser parser counting in all tests
2018-12-30 02:46:44 +01:00
Dave Halter
e5fb1927bb
Fix: Make the NodesStack to a NodesTree
...
This fixes an issue with positions that were doubled if the stack was closed too early.
2018-12-30 01:27:37 +01:00
Dave Halter
0ef4809377
Fix for diff parser : Make sure that start_pos are growing always
...
The problem was that functions/classes were sometimes not well positioned. Now
all diff tests are ensuring that leaves always grow.
2018-12-28 21:49:49 +01:00
Dave Halter
29456a6c0a
Add a check to see if leaves have the right start positions
2018-12-28 02:24:22 +01:00
Dave Halter
ada84ed063
Add parso version to an exception
2018-12-27 13:33:10 +01:00
Dave Halter
7064ecf3fb
Don't use invalid escape sequences in regex, see https://github.com/davidhalter/jedi-vim/issues/843
2018-07-12 08:53:48 +02:00
Dave Halter
e6bc924fba
Use a setuptools that still supports py33
2018-07-09 20:53:06 +02:00
Dave Halter
59605438e9
3.1 release notes
2018-07-09 20:27:10 +02:00
Dave Halter
e7f71a3eba
Use one simple functions to check for funcdefs in diff parser
2018-07-08 20:30:31 +02:00
Dave Halter
3f7aad84f9
Make sure to treat async funcdefs the same way as normal funcdefs
2018-07-08 20:18:15 +02:00
Dave Halter
52e3db4834
Fix an issue in the diff parser
...
Forgot to check for functions/classes that were part of a decorator/async func.
Fixes https://github.com/davidhalter/jedi/issues/1132
2018-07-06 01:25:06 +02:00
Dave Halter
0daf4d9068
Asterisks in function definitions may be at the end of a func without a comma, fixes #44
2018-07-04 09:51:34 +02:00
Dave Halter
29b6232541
Remove some TODOs that were fixed
2018-07-03 19:28:05 +02:00
Dave Halter
e05d7fd59f
Error recovery should not match the whole line in case of an invalid token, fixes #40
2018-07-03 01:31:32 +02:00
Dave Halter
ff67de248f
Merge branch 'pgen'
2018-06-29 18:14:03 +02:00
Dave Halter
1af5d9d46b
Add a changelog for 0.3.0
2018-06-29 18:13:53 +02:00
Dave Halter
fce3ead829
Bump version to 0.3.0
2018-06-29 18:04:55 +02:00
Dave Halter
55d5d39c53
Add a private API for jedi to work with the parser stack
2018-06-29 10:04:54 +02:00
Dave Halter
c8bf23b787
Remove get_tos_nodes and get_tos_first_tokens, because they are not used (not even in Jedi)
2018-06-29 00:00:09 +02:00
Dave Halter
98c9a1ec7f
Better documentation for _add_token
2018-06-28 10:11:44 +02:00
Dave Halter
ecdb90d9bc
Way better documentation for the DFA generator
2018-06-28 10:08:09 +02:00
Dave Halter
375ebf2181
Better documentation of the parser generator
2018-06-28 09:49:35 +02:00
Dave Halter
badb2fe010
Move transition_to_generator to transitions
2018-06-28 09:42:37 +02:00
Dave Halter
8e118c913c
Remove note about print as absolute import. This is probably not going to happen anymore, Python 2 is pretty much end-of-life
2018-06-28 01:01:46 +02:00
Dave Halter
52fc8fc569
Finish the stack in a way we want to.
2018-06-28 00:59:55 +02:00
Dave Halter
97cdb448d4
Pass tokens around and not all the different token values
2018-06-28 00:33:22 +02:00
Dave Halter
603b67ee6d
Just always pass token objects to the tokenizer
2018-06-28 00:18:44 +02:00
Dave Halter
7686273287
Use the stack from the parser itself
2018-06-28 00:12:18 +02:00
Dave Halter
692436ba12
Don't use grammar as an argument anymore, because it's already there
2018-06-28 00:01:47 +02:00
Dave Halter
f7d3d4e82f
Merge the PgenParser and our own parser
2018-06-27 23:45:04 +02:00
Dave Halter
edce279dee
Remove a function that was no longer used
2018-06-27 23:19:57 +02:00
Dave Halter
a9e40eb578
Simplify error recovery for suites
2018-06-27 22:21:17 +02:00
Dave Halter
b14f518306
Rename the last usage of ilabel to transition
2018-06-27 00:18:27 +02:00
Dave Halter
8407894b25
Fix python 2 tests
2018-06-27 00:15:00 +02:00
Dave Halter
e4efebc9f3
s/ilabel/transition/g
2018-06-26 23:05:04 +02:00
Dave Halter
f66e47c540
Check better for more transitions
2018-06-26 22:53:02 +02:00
Dave Halter
706a92ee0d
Merge branch 'master' of github.com:davidhalter/parso
2018-06-26 10:23:17 +02:00
Dave Halter
91d864b23d
Make it clearer which things are public in pgen
2018-06-26 10:22:38 +02:00
Dave Halter
e20f2069ba
Move the grammar to a fitting file.
2018-06-26 10:20:05 +02:00
Dave Halter
4cf198285a
Move things out of the grammar class
2018-06-26 10:15:31 +02:00
Dave Halter
30cf491b4f
Move the Grammar to the pgen module
2018-06-26 10:08:44 +02:00
Dave Halter
c1675da0cb
Make nonterminal_to_dfas public
2018-06-26 09:56:49 +02:00
Dave Halter
7b7b66eb3c
Get rid of the first_terminal variable in the grammar generator
2018-06-26 09:48:13 +02:00
Dave Halter
5d46c3e18b
Trying to reduce the amount of variables used in first sets
2018-06-26 01:04:22 +02:00
Dave Halter
e9fde82512
Remove the overlapcheck, it's probably not needed anymore
2018-06-26 01:00:06 +02:00
Dave Halter
a46ecbb499
Fix an ambiguity issue
...
Unfortunately had to refactor most of the transition generation
2018-06-26 00:58:19 +02:00
Dave Halter
da5aa8a2ab
Better detection of ambiguities
2018-06-25 01:56:02 +02:00
Dave Halter
43d4a8a834
Don't use a function that doesn't work
2018-06-24 23:45:18 +02:00
Dave Halter
309033ae2d
Work with token types whenever possible
2018-06-24 17:59:32 +02:00
Dave Halter
2a9d8632fe
Remove label caching
2018-06-24 17:56:13 +02:00
Dave Halter
530a324643
Remove labels
2018-06-24 17:55:22 +02:00
Dave Halter
71003bc20e
Just use caching instead of strange transitions
2018-06-24 17:29:03 +02:00
Dave Halter
c5d141bf60
Make some more things faster
2018-06-24 16:43:28 +02:00
Dave Halter
e958b241c7
Use some tokenize names directly
2018-06-24 16:39:48 +02:00
Dave Halter
34ab35558f
Remove a lot of the old token code
2018-06-24 16:31:58 +02:00
Dave Halter
03de9cebb8
Introduce TokenTypes
2018-06-24 16:24:09 +02:00
Dave Halter
6098d89150
Add PythonTokens to get rid of a lot of the token module eventually
2018-06-24 13:03:07 +02:00
Dave Halter
ff4358cd97
Merge remote-tracking branch 'origin/master' into pgen
2018-06-24 11:49:34 +02:00
Dave Halter
b5378e4602
Use token.OP and use reserved words
...
This change breaks the tokenizer backwards compatibility a bit. Details of
operators is now part of the parser and not the tokenizer anymore. The parser
does this anyway, so we don't need the complexity in the tokenizer.
2018-06-24 11:28:23 +02:00
Dave Halter
33e321a539
Don't set the root node before it's not actually defined
2018-06-22 13:04:00 +02:00
Dave Halter
a890ddd6cc
Remove make_first
2018-06-22 12:54:27 +02:00
Dave Halter
1362d4f05d
Remove more unused grammar stuff
2018-06-22 12:53:38 +02:00
Dave Halter
532aef2342
Remove nonterminal2number and number2nonterminal, they are no longer used
2018-06-22 12:52:44 +02:00
Dave Halter
878b4b2d3b
Use nonterminals instead of numbers if possible
2018-06-22 12:47:02 +02:00
Dave Halter
87299335c4
Remove more unused code
2018-06-22 12:36:48 +02:00
Dave Halter
4f0e9c0fd7
Remove old dfas and states from the parser generator
2018-06-22 11:47:18 +02:00
Dave Halter
67ca091631
delete a lot of the old parser code
2018-06-22 11:44:42 +02:00
Dave Halter
4e5ba02dbb
Fix the final issues of the new parser
2018-06-22 11:38:34 +02:00
Dave Halter
a85f544901
Fix all tests except diff tests. Mostly error recovery fixes
2018-06-22 11:12:10 +02:00
Dave Halter
9e8066c6fd
Fix a lot of the old error recovery
2018-06-22 09:56:58 +02:00
Dave Halter
68eab72229
Some slight changes to error recovery
2018-06-22 01:59:39 +02:00
Dave Halter
d9264609f2
Get quite a bit of the error recovery working
2018-06-22 01:56:29 +02:00
Dave Halter
79c7e0b59d
Use the new parser. Error recovery is not yet working
2018-06-22 00:38:18 +02:00
Dave Halter
f03a87b876
Actually parse some first things with the new approach
2018-06-21 23:56:34 +02:00
Dave Halter
2a082d69df
Add better reprs
2018-06-21 22:13:27 +02:00
Dave Halter
e6fc739670
Get most things ready for plans
2018-06-21 21:32:05 +02:00
Dave Halter
12e11b3d16
Remove a while loop that is not necessary
2018-06-21 21:14:14 +02:00
Dave Halter
cc8038966b
isfinal -> is_final
2018-06-21 21:11:46 +02:00
Dave Halter
31aecf2d35
Calculate the first plans in a very messy way
2018-06-21 21:10:28 +02:00
Dave Halter
d8554d86d1
A lot of new code to hopefully transition to a better parsing mechanism in the future
2018-06-21 18:17:32 +02:00
Dave Halter
d691bf0fd1
Some minor changes
2018-06-20 09:42:21 +02:00
Dave Halter
5712ffb5ca
Introduce a label cache that is currently not used
2018-06-18 20:14:29 +02:00
Dave Halter
55d6a69aad
Some more renames
2018-06-18 01:14:09 +02:00
Dave Halter
453471eeb6
Move some ParserGenerator stuff into the Grammar class
2018-06-18 00:15:31 +02:00
Dave Halter
a06c3a3129
name -> nonterminal
2018-06-17 23:10:27 +02:00
Dave Halter
73ce57428b
Try to completely remove the word symbol and use nonterminal
...
The ones that we could not remove are in grammar.py, because that's the public documented API.
2018-06-17 18:30:20 +02:00
Dave Halter
640f544af9
One instance of symbol -> terminal
2018-06-17 18:12:13 +02:00
Dave Halter
b6cbf306d7
Use the name first_terminals instead of first
2018-06-17 18:09:12 +02:00
Dave Halter
95e4ecf592
next -> next_
2018-06-17 17:49:43 +02:00
Dave Halter
fbed1ecfe0
More dict to set
2018-06-17 17:45:40 +02:00
Dave Halter
1f27fa9320
Use more nonterminal/terminal terminology
2018-06-17 17:37:15 +02:00
Dave Halter
23362ec2d3
Start using the term nonterminal
2018-06-17 16:40:11 +02:00
Dave Halter
6b391af071
Use sets instead of dicts if possible
2018-06-17 16:36:27 +02:00
Dave Halter