Dave Halter
5d46c3e18b
Trying to reduce the amount of variables used in first sets
2018-06-26 01:04:22 +02:00
Dave Halter
e9fde82512
Remove the overlapcheck, it's probably not needed anymore
2018-06-26 01:00:06 +02:00
Dave Halter
a46ecbb499
Fix an ambiguity issue
...
Unfortunately had to refactor most of the transition generation
2018-06-26 00:58:19 +02:00
Dave Halter
da5aa8a2ab
Better detection of ambiguities
2018-06-25 01:56:02 +02:00
Dave Halter
43d4a8a834
Don't use a function that doesn't work
2018-06-24 23:45:18 +02:00
Dave Halter
309033ae2d
Work with token types whenever possible
2018-06-24 17:59:32 +02:00
Dave Halter
2a9d8632fe
Remove label caching
2018-06-24 17:56:13 +02:00
Dave Halter
530a324643
Remove labels
2018-06-24 17:55:22 +02:00
Dave Halter
71003bc20e
Just use caching instead of strange transitions
2018-06-24 17:29:03 +02:00
Dave Halter
c5d141bf60
Make some more things faster
2018-06-24 16:43:28 +02:00
Dave Halter
e958b241c7
Use some tokenize names directly
2018-06-24 16:39:48 +02:00
Dave Halter
34ab35558f
Remove a lot of the old token code
2018-06-24 16:31:58 +02:00
Dave Halter
03de9cebb8
Introduce TokenTypes
2018-06-24 16:24:09 +02:00
Dave Halter
6098d89150
Add PythonTokens to get rid of a lot of the token module eventually
2018-06-24 13:03:07 +02:00
Dave Halter
ff4358cd97
Merge remote-tracking branch 'origin/master' into pgen
2018-06-24 11:49:34 +02:00
Dave Halter
b5378e4602
Use token.OP and use reserved words
...
This change breaks the tokenizer backwards compatibility a bit. Details of
operators is now part of the parser and not the tokenizer anymore. The parser
does this anyway, so we don't need the complexity in the tokenizer.
2018-06-24 11:28:23 +02:00
Dave Halter
33e321a539
Don't set the root node before it's not actually defined
2018-06-22 13:04:00 +02:00
Dave Halter
a890ddd6cc
Remove make_first
2018-06-22 12:54:27 +02:00
Dave Halter
1362d4f05d
Remove more unused grammar stuff
2018-06-22 12:53:38 +02:00
Dave Halter
532aef2342
Remove nonterminal2number and number2nonterminal, they are no longer used
2018-06-22 12:52:44 +02:00
Dave Halter
878b4b2d3b
Use nonterminals instead of numbers if possible
2018-06-22 12:47:02 +02:00
Dave Halter
87299335c4
Remove more unused code
2018-06-22 12:36:48 +02:00
Dave Halter
4f0e9c0fd7
Remove old dfas and states from the parser generator
2018-06-22 11:47:18 +02:00
Dave Halter
67ca091631
delete a lot of the old parser code
2018-06-22 11:44:42 +02:00
Dave Halter
4e5ba02dbb
Fix the final issues of the new parser
2018-06-22 11:38:34 +02:00
Dave Halter
a85f544901
Fix all tests except diff tests. Mostly error recovery fixes
2018-06-22 11:12:10 +02:00
Dave Halter
9e8066c6fd
Fix a lot of the old error recovery
2018-06-22 09:56:58 +02:00
Dave Halter
68eab72229
Some slight changes to error recovery
2018-06-22 01:59:39 +02:00
Dave Halter
d9264609f2
Get quite a bit of the error recovery working
2018-06-22 01:56:29 +02:00
Dave Halter
79c7e0b59d
Use the new parser. Error recovery is not yet working
2018-06-22 00:38:18 +02:00
Dave Halter
f03a87b876
Actually parse some first things with the new approach
2018-06-21 23:56:34 +02:00
Dave Halter
2a082d69df
Add better reprs
2018-06-21 22:13:27 +02:00
Dave Halter
e6fc739670
Get most things ready for plans
2018-06-21 21:32:05 +02:00
Dave Halter
12e11b3d16
Remove a while loop that is not necessary
2018-06-21 21:14:14 +02:00
Dave Halter
cc8038966b
isfinal -> is_final
2018-06-21 21:11:46 +02:00
Dave Halter
31aecf2d35
Calculate the first plans in a very messy way
2018-06-21 21:10:28 +02:00
Dave Halter
d8554d86d1
A lot of new code to hopefully transition to a better parsing mechanism in the future
2018-06-21 18:17:32 +02:00
Dave Halter
d691bf0fd1
Some minor changes
2018-06-20 09:42:21 +02:00
Dave Halter
5712ffb5ca
Introduce a label cache that is currently not used
2018-06-18 20:14:29 +02:00
Dave Halter
55d6a69aad
Some more renames
2018-06-18 01:14:09 +02:00
Dave Halter
453471eeb6
Move some ParserGenerator stuff into the Grammar class
2018-06-18 00:15:31 +02:00
Dave Halter
a06c3a3129
name -> nonterminal
2018-06-17 23:10:27 +02:00
Dave Halter
73ce57428b
Try to completely remove the word symbol and use nonterminal
...
The ones that we could not remove are in grammar.py, because that's the public documented API.
2018-06-17 18:30:20 +02:00
Dave Halter
640f544af9
One instance of symbol -> terminal
2018-06-17 18:12:13 +02:00
Dave Halter
b6cbf306d7
Use the name first_terminals instead of first
2018-06-17 18:09:12 +02:00
Dave Halter
95e4ecf592
next -> next_
2018-06-17 17:49:43 +02:00
Dave Halter
fbed1ecfe0
More dict to set
2018-06-17 17:45:40 +02:00
Dave Halter
1f27fa9320
Use more nonterminal/terminal terminology
2018-06-17 17:37:15 +02:00
Dave Halter
23362ec2d3
Start using the term nonterminal
2018-06-17 16:40:11 +02:00
Dave Halter
6b391af071
Use sets instead of dicts if possible
2018-06-17 16:36:27 +02:00
Dave Halter
c43cb21a0e
Add docstring
2018-06-15 17:57:59 +02:00
Dave Halter
24346a0d32
Add some comments
2018-06-15 10:21:00 +02:00
Dave Halter
9d452ec66a
Use a set instead of dict if it's not necessary
2018-06-15 00:35:39 +02:00
Dave Halter
567e0d7aed
Simplify some code
2018-06-15 00:06:56 +02:00
Dave Halter
1f02327cff
Some refactorings for simplicity
2018-06-14 18:18:57 +02:00
Dave Halter
8c348aee6f
Use NFAArc as a class
2018-06-14 10:29:19 +02:00
Dave Halter
a277ccf288
Some more renames
2018-06-14 01:10:23 +02:00
Dave Halter
a5ce2caab6
Another rename
2018-06-13 23:19:52 +02:00
Dave Halter
da4df9c0f1
Rename
2018-06-13 21:12:48 +02:00
Dave Halter
bd444df417
Move the grammar parsing to a separate module
2018-06-13 21:01:06 +02:00
Dave Halter
275dbca1b9
More renames
2018-06-13 20:54:18 +02:00
Dave Halter
9a0b6f4928
Rename
2018-06-13 20:47:16 +02:00
Dave Halter
fc5560874b
Separate generating dfas from parsing
2018-06-13 20:46:36 +02:00
Dave Halter
6e5a520e7b
Cleanup some names
2018-06-13 10:18:05 +02:00
Dave Halter
dcabf3d415
Remove some code that is not used anymore
2018-06-13 10:15:21 +02:00
Dave Halter
3bc82d112d
Some cleanups and documentation
2018-06-13 10:10:17 +02:00
Dave Halter
ec186a78f8
Move out some more functions out of classes
2018-06-13 01:50:25 +02:00
Dave Halter
3818fb2b22
Some more refactorings for clarification
2018-06-13 01:44:44 +02:00
Dave Halter
95ddeb4012
Factor out start_symbol into a better position
2018-06-13 00:54:18 +02:00
Dave Halter
f638abb08e
Refactor out dfas
2018-06-13 00:31:45 +02:00
Dave Halter
f8558df27a
Document pgen grammars a bit better
2018-06-13 00:27:51 +02:00
Dave Halter
bae56e72e1
addarc -> add_arc
2018-06-12 22:13:08 +02:00
Dave Halter
41c38311f7
Refactor some things in pgen
2018-06-12 22:08:19 +02:00
Dave Halter
eeb456a6d4
Move some initializations
2018-06-12 22:00:49 +02:00
Dave Halter
1c0956d9e0
Change license again. The year shouldn't matter
2018-06-12 21:58:27 +02:00
Dave Halter
c17156bd36
Make first private
2018-06-12 21:56:26 +02:00
Dave Halter
8865aa452c
Change copyright years
2018-06-12 21:55:37 +02:00
Dave Halter
e0c79a9fcc
Remove some code from the grammar
2018-06-12 21:49:18 +02:00
Dave Halter
3c08b1b058
Separate the grammar generation from the grammar parsing
2018-06-12 21:30:09 +02:00
Dave Halter
0f32673092
In pgen now everything is named grammar and not c
2018-06-12 18:17:02 +02:00
Dave Halter
1e18163402
Better recovery for online classes and functions
2018-06-12 13:23:49 +02:00
Dave Halter
cef9f1bdbd
Fix one-line error recovery for all things that are using a suite
...
Fixes https://github.com/davidhalter/jedi/issues/1138 .
2018-06-12 12:56:27 +02:00
Dave Halter
23db71a5f7
Add a debug function for first tokens
2018-06-12 01:50:37 +02:00
Dave Halter
6f385bdba1
Not testing Python 3.3 anymore on travis. It seems to be broken
2018-05-21 12:55:49 +02:00
Dave Halter
4fc31c58b3
Add a changelog for 0.2.1
2018-05-21 12:48:31 +02:00
Dave Halter
689decc66c
Push the version
2018-05-21 12:44:10 +02:00
Dave Halter
c2eacdb81c
The diff parser was slighly off with prefixes, fixes #1121
2018-05-20 19:13:50 +02:00
Dave Halter
ac0bf4fcdd
A better repr for the endmarker
2018-05-17 09:56:16 +02:00
Dave Halter
948f9ccecc
Merge branch 'master' of github.com:davidhalter/parso
2018-04-23 23:42:11 +02:00
Dave Halter
f20106d88e
Fix a prefix issue with error leafs.
2018-04-22 19:28:30 +02:00
Dave Halter
579146b501
Don't test python 2.6 in tox by default, because the newer pip versions don't support it anymore
2018-04-15 14:46:35 +02:00
Dave Halter
deb4dbce1c
Set a release date
2018-04-15 13:54:10 +02:00
Dave Halter
8eda8decea
Fix whitespace issues with prefixes
2018-04-07 15:34:58 +02:00
Dave Halter
f6935935c0
Use proper leafs for fstring start/end
2018-04-07 12:32:33 +02:00
Dave Halter
d3fa7e1cad
Fix a Python 2 related issue.
2018-04-07 11:14:41 +02:00
Dave Halter
83d9abd036
Forgot to delete another print. WTF I'm tired
2018-04-07 02:22:45 +02:00
Dave Halter
222e9117b4
Unfortunately forgot to delete a print
2018-04-07 02:21:59 +02:00
Dave Halter
eda2207e6c
Start to write a changelog for 0.2.0
2018-04-07 02:20:00 +02:00
Dave Halter
a91e5f2775
Merge branch 'fstrings'
...
f-strings are now parsed as part of the Python grammar and not in separate
steps.
Note that this is not the way that CPython does it. CPython still uses multiple
parse steps in ast.c.
2018-04-07 02:17:02 +02:00
Dave Halter
cba4f2ccc1
Fix the syntax errors from f-strings
2018-04-07 02:14:35 +02:00
Dave Halter