Dave Halter
e591b929eb
Fix diff parser: Skip last leaves for last line offset leaves
2019-01-17 00:15:38 +01:00
Dave Halter
dac4c445a7
Fix indentation error tokens
2019-01-16 23:21:31 +01:00
Dave Halter
20fd32b45d
Fix diff parser: Avoid side effects for prefix
2019-01-14 21:37:19 +01:00
Dave Halter
9cc8178998
Fix tokenizer: backslashes sometimes led to newline token generation
2019-01-14 09:59:16 +01:00
Dave Halter
1e25445176
Make lines easier copyable in the fuzzer
2019-01-14 01:50:39 +01:00
Dave Halter
d7171ae927
Fix tokenizer: Carriage returns after backslashes were not properly handled
2019-01-14 01:49:09 +01:00
Dave Halter
d3d28480ed
Fix in diff parser: prefix calculation was wrong when copying nodes
2019-01-14 01:00:17 +01:00
Dave Halter
564be7882e
Replace --print-diff with --print-code
2019-01-14 00:20:49 +01:00
Dave Halter
76c5754b76
Fix diff parser generation for empty files
2019-01-13 23:38:35 +01:00
Dave Halter
55247a5a2c
Docopt should not be needed for tests
2019-01-13 23:24:17 +01:00
Dave Halter
7ae1efe5c7
Fix tokenizer: Form feeds and multiline docstrings didn't work together
2019-01-13 23:16:09 +01:00
Dave Halter
01dba7f8ce
Fix diff parser: Need to calculate the prefix for the diff tokenizer better
2019-01-13 22:38:53 +01:00
Dave Halter
ea8a758051
Remove copied nodes stuff, to simplify some things
2019-01-13 19:57:23 +01:00
Dave Halter
a7e24a37e7
Fix newline endings and a few parser/copy counts
2019-01-13 19:55:18 +01:00
Dave Halter
f80d9de7a0
Feature: The diff parser fuzzer is now able to use random Python fragments
...
This hopefully leads to the fuzzer finding more and faster issues in the diff
parser.
2019-01-13 16:00:36 +01:00
Dave Halter
eaee2b9ca0
Fix: The Python 3.8 grammar did not include f-string support
2019-01-13 15:51:46 +01:00
Dave Halter
dd1761da96
Fix tokenizer: Closing parentheses in the wrong place should not lead to strange behavior
2019-01-13 14:52:33 +01:00
Dave Halter
e10802ab09
Fix end positions with error dedents
2019-01-13 14:14:16 +01:00
Dave Halter
3d402d0a77
Fix diff parser tests for Python 2
2019-01-10 09:26:42 +01:00
Dave Halter
f6a8b997f2
Randomize the fuzzer a bit more with inserting characters
2019-01-10 01:22:24 +01:00
Dave Halter
94c2681c8e
Simplify the regexes
2019-01-10 01:21:56 +01:00
Dave Halter
610a820799
Fix a regex clause that was totally wrong
2019-01-10 01:00:08 +01:00
Dave Halter
57320af6eb
Fix another tokenizer issue
2019-01-09 00:55:54 +01:00
Dave Halter
574e1c63e8
Apply \r changes in syntax trees
2019-01-09 00:34:19 +01:00
Dave Halter
fbaad7883f
Actually make \r usable
2019-01-08 20:03:08 +01:00
Dave Halter
b1f613fe16
Fix split lines for Python code
...
Some characters like Vertical Tab or File Separator were used as line separators.
This is not legal. Line Separators in Python are only Carriage Return \r and Line Feed \n.
2019-01-08 08:42:30 +01:00
Dave Halter
f4696a6245
Add \r as a valid linebreak for splitlines
2019-01-07 18:46:16 +01:00
Dave Halter
48c1a0e590
Move split_lines tests around
2019-01-07 18:40:34 +01:00
Dave Halter
6f63147f69
Start generating really random strings with the fuzzer
2019-01-06 20:51:49 +01:00
Dave Halter
94bd48bae1
Fix tokenizer: Dedents before error tokens are properly done, now.
2019-01-06 19:26:49 +01:00
Dave Halter
edbceba4f8
Fix diff parser: Also check async with
2019-01-06 16:25:28 +01:00
Dave Halter
b33c2b3ae1
Make the diff parser use a lot of different files by default
2019-01-06 15:43:37 +01:00
Dave Halter
65a0748f4f
Fix diff parser: Forgot that with statments are also flows
2019-01-06 15:41:16 +01:00
Dave Halter
c442cf98be
Fix valid graph asserting for some dedents that are errors
2019-01-06 12:39:04 +01:00
Dave Halter
65b15b05e3
Fix diff parser: If funcs are not copied, errors shouldn't either
2019-01-06 11:39:51 +01:00
Dave Halter
26aee1c6a9
Better documentation for the fuzz diff parser script
2019-01-06 01:10:15 +01:00
Dave Halter
c88a862bae
Rename a test
2019-01-06 01:08:15 +01:00
Dave Halter
d6b0585933
More verbose output for the diff fuzzer
2019-01-06 01:05:07 +01:00
Dave Halter
6eba40b4c5
Fix diff parser: error dedent issues
2019-01-06 01:00:34 +01:00
Dave Halter
428bde0573
Fix diff parser: Avoid indentation issues
2019-01-05 22:40:31 +01:00
Dave Halter
d1d866f6c6
Use the right diff order in debug output
2019-01-05 18:36:48 +01:00
Dave Halter
a8ec75fedd
Fix diff parser: The prefix was wrong in some copy cases
2019-01-05 18:33:38 +01:00
Dave Halter
deaf1f310b
Make fuzz parser compatible with Python 2
2019-01-05 14:57:58 +01:00
Dave Halter
2a881bf875
Make it possible to print all diffs in fuzzer
2019-01-05 14:50:59 +01:00
Dave Halter
4d713f56e9
Introduce a redo flag 'only_last' to narrow down issues
2019-01-05 14:20:30 +01:00
Dave Halter
d202fdea49
Add docopt to testing dependencies
2019-01-05 14:09:14 +01:00
Dave Halter
5e6d5dec59
Rewrite the fuzz diff parser to cache errors (so we can re-run those)
2019-01-05 14:05:19 +01:00
Dave Halter
c1846dd082
Fix diff parser: Decorators were sometimes parsed without their functions
2019-01-05 09:29:00 +01:00
Dave Halter
5da51720cd
Fix tokenizer: Dedents should only happen after newlines
2019-01-03 11:44:17 +01:00
Dave Halter
fde64d0eae
Usability for diff parser fuzzing
2019-01-02 17:31:07 +01:00
Dave Halter
430f13af5e
Fix for diff parser: Rewrite prefix logic and don't mutate prematurely
2019-01-02 17:28:01 +01:00
Dave Halter
96ae6a078b
Fix diff parser: positioning of functions if decorators were removed
2019-01-02 13:16:22 +01:00
Dave Halter
a9f58b7c45
Ignore ERROR_DEDENT in graph validation
2019-01-02 12:15:05 +01:00
Dave Halter
e0d0e57bd0
Add a small diff parser fuzzer
...
It should help us find the rest of the issues that the diff parser has
2019-01-02 11:26:31 +01:00
Dave Halter
d2542983e9
Fix diff parser: get_last_line was sometimes wrong
...
Now the calculation is way simpler. Still annoying that it even happened.
2019-01-02 01:39:53 +01:00
Dave Halter
64cf24d9da
Fix error reporting order for diff issues
2019-01-02 00:33:43 +01:00
Dave Halter
02f48a68f2
Clean up the test diff parser file
2019-01-01 23:37:31 +01:00
Dave Halter
c7c464e5e9
Avoid nasty side effects in creation of Node
...
This issue led to bugs in Jedi, because Jedi used the nodes in a wrong way.
2019-01-01 23:35:20 +01:00
Dave Halter
29325d3052
Make parso errors even more informative
2018-12-31 11:47:02 +01:00
Dave Halter
750b8af37b
Fix diff parser get_last_line calculation
2018-12-31 01:25:11 +01:00
Dave Halter
0126a38bd1
Fix graph asserting for error indents
2018-12-30 18:20:55 +01:00
Dave Halter
c2985c111e
Better checks for checking valid graphs
2018-12-30 16:34:11 +01:00
Dave Halter
45f9d4b204
Create better ways for debugging the diff parser
2018-12-30 16:03:54 +01:00
Dave Halter
f99fe6ad21
Fix diff-parser: Copying parts of if else should not lead to the whole thing being copied
2018-12-30 15:25:17 +01:00
Dave Halter
a64c32bb2a
Reenable diff parser parser counting in all tests
2018-12-30 02:46:44 +01:00
Dave Halter
e5fb1927bb
Fix: Make the NodesStack to a NodesTree
...
This fixes an issue with positions that were doubled if the stack was closed too early.
2018-12-30 01:27:37 +01:00
Dave Halter
0ef4809377
Fix for diff parser : Make sure that start_pos are growing always
...
The problem was that functions/classes were sometimes not well positioned. Now
all diff tests are ensuring that leaves always grow.
2018-12-28 21:49:49 +01:00
Dave Halter
29456a6c0a
Add a check to see if leaves have the right start positions
2018-12-28 02:24:22 +01:00
Dave Halter
ada84ed063
Add parso version to an exception
2018-12-27 13:33:10 +01:00
Dave Halter
7064ecf3fb
Don't use invalid escape sequences in regex, see https://github.com/davidhalter/jedi-vim/issues/843
2018-07-12 08:53:48 +02:00
Dave Halter
e6bc924fba
Use a setuptools that still supports py33
2018-07-09 20:53:06 +02:00
Dave Halter
59605438e9
3.1 release notes
2018-07-09 20:27:10 +02:00
Dave Halter
e7f71a3eba
Use one simple functions to check for funcdefs in diff parser
2018-07-08 20:30:31 +02:00
Dave Halter
3f7aad84f9
Make sure to treat async funcdefs the same way as normal funcdefs
2018-07-08 20:18:15 +02:00
Dave Halter
52e3db4834
Fix an issue in the diff parser
...
Forgot to check for functions/classes that were part of a decorator/async func.
Fixes https://github.com/davidhalter/jedi/issues/1132
2018-07-06 01:25:06 +02:00
Dave Halter
0daf4d9068
Asterisks in function definitions may be at the end of a func without a comma, fixes #44
2018-07-04 09:51:34 +02:00
Dave Halter
29b6232541
Remove some TODOs that were fixed
2018-07-03 19:28:05 +02:00
Dave Halter
e05d7fd59f
Error recovery should not match the whole line in case of an invalid token, fixes #40
2018-07-03 01:31:32 +02:00
Dave Halter
ff67de248f
Merge branch 'pgen'
2018-06-29 18:14:03 +02:00
Dave Halter
1af5d9d46b
Add a changelog for 0.3.0
2018-06-29 18:13:53 +02:00
Dave Halter
fce3ead829
Bump version to 0.3.0
2018-06-29 18:04:55 +02:00
Dave Halter
55d5d39c53
Add a private API for jedi to work with the parser stack
2018-06-29 10:04:54 +02:00
Dave Halter
c8bf23b787
Remove get_tos_nodes and get_tos_first_tokens, because they are not used (not even in Jedi)
2018-06-29 00:00:09 +02:00
Dave Halter
98c9a1ec7f
Better documentation for _add_token
2018-06-28 10:11:44 +02:00
Dave Halter
ecdb90d9bc
Way better documentation for the DFA generator
2018-06-28 10:08:09 +02:00
Dave Halter
375ebf2181
Better documentation of the parser generator
2018-06-28 09:49:35 +02:00
Dave Halter
badb2fe010
Move transition_to_generator to transitions
2018-06-28 09:42:37 +02:00
Dave Halter
8e118c913c
Remove note about print as absolute import. This is probably not going to happen anymore, Python 2 is pretty much end-of-life
2018-06-28 01:01:46 +02:00
Dave Halter
52fc8fc569
Finish the stack in a way we want to.
2018-06-28 00:59:55 +02:00
Dave Halter
97cdb448d4
Pass tokens around and not all the different token values
2018-06-28 00:33:22 +02:00
Dave Halter
603b67ee6d
Just always pass token objects to the tokenizer
2018-06-28 00:18:44 +02:00
Dave Halter
7686273287
Use the stack from the parser itself
2018-06-28 00:12:18 +02:00
Dave Halter
692436ba12
Don't use grammar as an argument anymore, because it's already there
2018-06-28 00:01:47 +02:00
Dave Halter
f7d3d4e82f
Merge the PgenParser and our own parser
2018-06-27 23:45:04 +02:00
Dave Halter
edce279dee
Remove a function that was no longer used
2018-06-27 23:19:57 +02:00
Dave Halter
a9e40eb578
Simplify error recovery for suites
2018-06-27 22:21:17 +02:00
Dave Halter
b14f518306
Rename the last usage of ilabel to transition
2018-06-27 00:18:27 +02:00
Dave Halter
8407894b25
Fix python 2 tests
2018-06-27 00:15:00 +02:00
Dave Halter
e4efebc9f3
s/ilabel/transition/g
2018-06-26 23:05:04 +02:00
Dave Halter
f66e47c540
Check better for more transitions
2018-06-26 22:53:02 +02:00
Dave Halter