Dave Halter
d1d866f6c6
Use the right diff order in debug output
2019-01-05 18:36:48 +01:00
Dave Halter
a8ec75fedd
Fix diff parser: The prefix was wrong in some copy cases
2019-01-05 18:33:38 +01:00
Dave Halter
deaf1f310b
Make fuzz parser compatible with Python 2
2019-01-05 14:57:58 +01:00
Dave Halter
2a881bf875
Make it possible to print all diffs in fuzzer
2019-01-05 14:50:59 +01:00
Dave Halter
4d713f56e9
Introduce a redo flag 'only_last' to narrow down issues
2019-01-05 14:20:30 +01:00
Dave Halter
d202fdea49
Add docopt to testing dependencies
2019-01-05 14:09:14 +01:00
Dave Halter
5e6d5dec59
Rewrite the fuzz diff parser to cache errors (so we can re-run those)
2019-01-05 14:05:19 +01:00
Dave Halter
c1846dd082
Fix diff parser: Decorators were sometimes parsed without their functions
2019-01-05 09:29:00 +01:00
Dave Halter
5da51720cd
Fix tokenizer: Dedents should only happen after newlines
2019-01-03 11:44:17 +01:00
Dave Halter
fde64d0eae
Usability for diff parser fuzzing
2019-01-02 17:31:07 +01:00
Dave Halter
430f13af5e
Fix for diff parser: Rewrite prefix logic and don't mutate prematurely
2019-01-02 17:28:01 +01:00
Dave Halter
96ae6a078b
Fix diff parser: positioning of functions if decorators were removed
2019-01-02 13:16:22 +01:00
Dave Halter
a9f58b7c45
Ignore ERROR_DEDENT in graph validation
2019-01-02 12:15:05 +01:00
Dave Halter
e0d0e57bd0
Add a small diff parser fuzzer
...
It should help us find the rest of the issues that the diff parser has
2019-01-02 11:26:31 +01:00
Dave Halter
d2542983e9
Fix diff parser: get_last_line was sometimes wrong
...
Now the calculation is way simpler. Still annoying that it even happened.
2019-01-02 01:39:53 +01:00
Dave Halter
64cf24d9da
Fix error reporting order for diff issues
2019-01-02 00:33:43 +01:00
Dave Halter
02f48a68f2
Clean up the test diff parser file
2019-01-01 23:37:31 +01:00
Dave Halter
c7c464e5e9
Avoid nasty side effects in creation of Node
...
This issue led to bugs in Jedi, because Jedi used the nodes in a wrong way.
2019-01-01 23:35:20 +01:00
Dave Halter
29325d3052
Make parso errors even more informative
2018-12-31 11:47:02 +01:00
Dave Halter
750b8af37b
Fix diff parser get_last_line calculation
2018-12-31 01:25:11 +01:00
Dave Halter
0126a38bd1
Fix graph asserting for error indents
2018-12-30 18:20:55 +01:00
Dave Halter
c2985c111e
Better checks for checking valid graphs
2018-12-30 16:34:11 +01:00
Dave Halter
45f9d4b204
Create better ways for debugging the diff parser
2018-12-30 16:03:54 +01:00
Dave Halter
f99fe6ad21
Fix diff-parser: Copying parts of if else should not lead to the whole thing being copied
2018-12-30 15:25:17 +01:00
Dave Halter
a64c32bb2a
Reenable diff parser parser counting in all tests
2018-12-30 02:46:44 +01:00
Dave Halter
e5fb1927bb
Fix: Make the NodesStack to a NodesTree
...
This fixes an issue with positions that were doubled if the stack was closed too early.
2018-12-30 01:27:37 +01:00
Dave Halter
0ef4809377
Fix for diff parser : Make sure that start_pos are growing always
...
The problem was that functions/classes were sometimes not well positioned. Now
all diff tests are ensuring that leaves always grow.
2018-12-28 21:49:49 +01:00
Dave Halter
29456a6c0a
Add a check to see if leaves have the right start positions
2018-12-28 02:24:22 +01:00
Dave Halter
ada84ed063
Add parso version to an exception
2018-12-27 13:33:10 +01:00
Dave Halter
7064ecf3fb
Don't use invalid escape sequences in regex, see https://github.com/davidhalter/jedi-vim/issues/843
2018-07-12 08:53:48 +02:00
Dave Halter
e6bc924fba
Use a setuptools that still supports py33
2018-07-09 20:53:06 +02:00
Dave Halter
59605438e9
3.1 release notes
2018-07-09 20:27:10 +02:00
Dave Halter
e7f71a3eba
Use one simple functions to check for funcdefs in diff parser
2018-07-08 20:30:31 +02:00
Dave Halter
3f7aad84f9
Make sure to treat async funcdefs the same way as normal funcdefs
2018-07-08 20:18:15 +02:00
Dave Halter
52e3db4834
Fix an issue in the diff parser
...
Forgot to check for functions/classes that were part of a decorator/async func.
Fixes https://github.com/davidhalter/jedi/issues/1132
2018-07-06 01:25:06 +02:00
Dave Halter
0daf4d9068
Asterisks in function definitions may be at the end of a func without a comma, fixes #44
2018-07-04 09:51:34 +02:00
Dave Halter
29b6232541
Remove some TODOs that were fixed
2018-07-03 19:28:05 +02:00
Dave Halter
e05d7fd59f
Error recovery should not match the whole line in case of an invalid token, fixes #40
2018-07-03 01:31:32 +02:00
Dave Halter
ff67de248f
Merge branch 'pgen'
2018-06-29 18:14:03 +02:00
Dave Halter
1af5d9d46b
Add a changelog for 0.3.0
2018-06-29 18:13:53 +02:00
Dave Halter
fce3ead829
Bump version to 0.3.0
2018-06-29 18:04:55 +02:00
Dave Halter
55d5d39c53
Add a private API for jedi to work with the parser stack
2018-06-29 10:04:54 +02:00
Dave Halter
c8bf23b787
Remove get_tos_nodes and get_tos_first_tokens, because they are not used (not even in Jedi)
2018-06-29 00:00:09 +02:00
Dave Halter
98c9a1ec7f
Better documentation for _add_token
2018-06-28 10:11:44 +02:00
Dave Halter
ecdb90d9bc
Way better documentation for the DFA generator
2018-06-28 10:08:09 +02:00
Dave Halter
375ebf2181
Better documentation of the parser generator
2018-06-28 09:49:35 +02:00
Dave Halter
badb2fe010
Move transition_to_generator to transitions
2018-06-28 09:42:37 +02:00
Dave Halter
8e118c913c
Remove note about print as absolute import. This is probably not going to happen anymore, Python 2 is pretty much end-of-life
2018-06-28 01:01:46 +02:00
Dave Halter
52fc8fc569
Finish the stack in a way we want to.
2018-06-28 00:59:55 +02:00
Dave Halter
97cdb448d4
Pass tokens around and not all the different token values
2018-06-28 00:33:22 +02:00
Dave Halter
603b67ee6d
Just always pass token objects to the tokenizer
2018-06-28 00:18:44 +02:00
Dave Halter
7686273287
Use the stack from the parser itself
2018-06-28 00:12:18 +02:00
Dave Halter
692436ba12
Don't use grammar as an argument anymore, because it's already there
2018-06-28 00:01:47 +02:00
Dave Halter
f7d3d4e82f
Merge the PgenParser and our own parser
2018-06-27 23:45:04 +02:00
Dave Halter
edce279dee
Remove a function that was no longer used
2018-06-27 23:19:57 +02:00
Dave Halter
a9e40eb578
Simplify error recovery for suites
2018-06-27 22:21:17 +02:00
Dave Halter
b14f518306
Rename the last usage of ilabel to transition
2018-06-27 00:18:27 +02:00
Dave Halter
8407894b25
Fix python 2 tests
2018-06-27 00:15:00 +02:00
Dave Halter
e4efebc9f3
s/ilabel/transition/g
2018-06-26 23:05:04 +02:00
Dave Halter
f66e47c540
Check better for more transitions
2018-06-26 22:53:02 +02:00
Dave Halter
706a92ee0d
Merge branch 'master' of github.com:davidhalter/parso
2018-06-26 10:23:17 +02:00
Dave Halter
91d864b23d
Make it clearer which things are public in pgen
2018-06-26 10:22:38 +02:00
Dave Halter
e20f2069ba
Move the grammar to a fitting file.
2018-06-26 10:20:05 +02:00
Dave Halter
4cf198285a
Move things out of the grammar class
2018-06-26 10:15:31 +02:00
Dave Halter
30cf491b4f
Move the Grammar to the pgen module
2018-06-26 10:08:44 +02:00
Dave Halter
c1675da0cb
Make nonterminal_to_dfas public
2018-06-26 09:56:49 +02:00
Dave Halter
7b7b66eb3c
Get rid of the first_terminal variable in the grammar generator
2018-06-26 09:48:13 +02:00
Dave Halter
5d46c3e18b
Trying to reduce the amount of variables used in first sets
2018-06-26 01:04:22 +02:00
Dave Halter
e9fde82512
Remove the overlapcheck, it's probably not needed anymore
2018-06-26 01:00:06 +02:00
Dave Halter
a46ecbb499
Fix an ambiguity issue
...
Unfortunately had to refactor most of the transition generation
2018-06-26 00:58:19 +02:00
Dave Halter
da5aa8a2ab
Better detection of ambiguities
2018-06-25 01:56:02 +02:00
Dave Halter
43d4a8a834
Don't use a function that doesn't work
2018-06-24 23:45:18 +02:00
Dave Halter
309033ae2d
Work with token types whenever possible
2018-06-24 17:59:32 +02:00
Dave Halter
2a9d8632fe
Remove label caching
2018-06-24 17:56:13 +02:00
Dave Halter
530a324643
Remove labels
2018-06-24 17:55:22 +02:00
Dave Halter
71003bc20e
Just use caching instead of strange transitions
2018-06-24 17:29:03 +02:00
Dave Halter
c5d141bf60
Make some more things faster
2018-06-24 16:43:28 +02:00
Dave Halter
e958b241c7
Use some tokenize names directly
2018-06-24 16:39:48 +02:00
Dave Halter
34ab35558f
Remove a lot of the old token code
2018-06-24 16:31:58 +02:00
Dave Halter
03de9cebb8
Introduce TokenTypes
2018-06-24 16:24:09 +02:00
Dave Halter
6098d89150
Add PythonTokens to get rid of a lot of the token module eventually
2018-06-24 13:03:07 +02:00
Dave Halter
ff4358cd97
Merge remote-tracking branch 'origin/master' into pgen
2018-06-24 11:49:34 +02:00
Dave Halter
b5378e4602
Use token.OP and use reserved words
...
This change breaks the tokenizer backwards compatibility a bit. Details of
operators is now part of the parser and not the tokenizer anymore. The parser
does this anyway, so we don't need the complexity in the tokenizer.
2018-06-24 11:28:23 +02:00
Dave Halter
33e321a539
Don't set the root node before it's not actually defined
2018-06-22 13:04:00 +02:00
Dave Halter
a890ddd6cc
Remove make_first
2018-06-22 12:54:27 +02:00
Dave Halter
1362d4f05d
Remove more unused grammar stuff
2018-06-22 12:53:38 +02:00
Dave Halter
532aef2342
Remove nonterminal2number and number2nonterminal, they are no longer used
2018-06-22 12:52:44 +02:00
Dave Halter
878b4b2d3b
Use nonterminals instead of numbers if possible
2018-06-22 12:47:02 +02:00
Dave Halter
87299335c4
Remove more unused code
2018-06-22 12:36:48 +02:00
Dave Halter
4f0e9c0fd7
Remove old dfas and states from the parser generator
2018-06-22 11:47:18 +02:00
Dave Halter
67ca091631
delete a lot of the old parser code
2018-06-22 11:44:42 +02:00
Dave Halter
4e5ba02dbb
Fix the final issues of the new parser
2018-06-22 11:38:34 +02:00
Dave Halter
a85f544901
Fix all tests except diff tests. Mostly error recovery fixes
2018-06-22 11:12:10 +02:00
Dave Halter
9e8066c6fd
Fix a lot of the old error recovery
2018-06-22 09:56:58 +02:00
Dave Halter
68eab72229
Some slight changes to error recovery
2018-06-22 01:59:39 +02:00
Dave Halter
d9264609f2
Get quite a bit of the error recovery working
2018-06-22 01:56:29 +02:00
Dave Halter
79c7e0b59d
Use the new parser. Error recovery is not yet working
2018-06-22 00:38:18 +02:00
Dave Halter
f03a87b876
Actually parse some first things with the new approach
2018-06-21 23:56:34 +02:00
Dave Halter
2a082d69df
Add better reprs
2018-06-21 22:13:27 +02:00
Dave Halter
e6fc739670
Get most things ready for plans
2018-06-21 21:32:05 +02:00
Dave Halter