Dave Halter
01ae01a382
Remove dead code
2019-01-23 23:28:18 +01:00
Dave Halter
5fbc207892
Refactor f-string support
2019-01-23 10:58:36 +01:00
Dave Halter
60e4591837
Fix: End detection for strings was mostly wrong, fixes #51
2019-01-23 10:13:25 +01:00
Dave Halter
ef56debb78
Fix f-string escapes, fixes #48
...
The tokenizer was not detecting backslash escapes for f-string endings properly
2019-01-22 22:20:32 +01:00
Dave Halter
dc2582f488
Tokenizer: Simplify end of string regexes
2019-01-22 09:33:57 +01:00
Dave Halter
fe69989fbc
Add a comment from the Python3.7 code base
2019-01-21 00:33:02 +01:00
Dave Halter
ce8b531175
Fix diff parser: The previous fix was a bit off
2019-01-20 19:03:45 +01:00
Dave Halter
069c08883a
Change fuzzer: Add ways to not always use correct parse input
2019-01-20 18:18:13 +01:00
Dave Halter
0da0a8655a
Fix diff parser: issue with opening brackets
2019-01-20 00:41:11 +01:00
Dave Halter
3d890c3a00
Async doesn't work in 3.4
2019-01-19 12:59:35 +01:00
Dave Halter
956ea55048
Skip some tests for Python2.6 and Python3.3
2019-01-19 12:08:39 +01:00
Dave Halter
0bd17bee2c
Fix diff parser: DEDENT as error leaves should also be ignored and reparsed
2019-01-18 18:41:08 +01:00
Dave Halter
f3015efb2d
Fix diff parser: error dedents in between nodes should be ignored for now when copying
2019-01-18 02:43:12 +01:00
Dave Halter
197391dc53
Fix diff parser: Don't copy error nodes/leaves in the beginning, leads to strange issues in some cases
2019-01-17 23:48:00 +01:00
Dave Halter
32321a74b1
Diff fuzzer: Create a check to see if the errors make sense.
2019-01-17 22:07:59 +01:00
Dave Halter
52d01685ba
Fix diff parser: Don't copy DEDENT tokens at the beginning
2019-01-17 21:31:13 +01:00
Dave Halter
e591b929eb
Fix diff parser: Skip last leaves for last line offset leaves
2019-01-17 00:15:38 +01:00
Dave Halter
dac4c445a7
Fix indentation error tokens
2019-01-16 23:21:31 +01:00
Dave Halter
20fd32b45d
Fix diff parser: Avoid side effects for prefix
2019-01-14 21:37:19 +01:00
Dave Halter
9cc8178998
Fix tokenizer: backslashes sometimes led to newline token generation
2019-01-14 09:59:16 +01:00
Dave Halter
1e25445176
Make lines easier copyable in the fuzzer
2019-01-14 01:50:39 +01:00
Dave Halter
d7171ae927
Fix tokenizer: Carriage returns after backslashes were not properly handled
2019-01-14 01:49:09 +01:00
Dave Halter
d3d28480ed
Fix in diff parser: prefix calculation was wrong when copying nodes
2019-01-14 01:00:17 +01:00
Dave Halter
564be7882e
Replace --print-diff with --print-code
2019-01-14 00:20:49 +01:00
Dave Halter
76c5754b76
Fix diff parser generation for empty files
2019-01-13 23:38:35 +01:00
Dave Halter
55247a5a2c
Docopt should not be needed for tests
2019-01-13 23:24:17 +01:00
Dave Halter
7ae1efe5c7
Fix tokenizer: Form feeds and multiline docstrings didn't work together
2019-01-13 23:16:09 +01:00
Dave Halter
01dba7f8ce
Fix diff parser: Need to calculate the prefix for the diff tokenizer better
2019-01-13 22:38:53 +01:00
Dave Halter
ea8a758051
Remove copied nodes stuff, to simplify some things
2019-01-13 19:57:23 +01:00
Dave Halter
a7e24a37e7
Fix newline endings and a few parser/copy counts
2019-01-13 19:55:18 +01:00
Dave Halter
f80d9de7a0
Feature: The diff parser fuzzer is now able to use random Python fragments
...
This hopefully leads to the fuzzer finding more and faster issues in the diff
parser.
2019-01-13 16:00:36 +01:00
Dave Halter
eaee2b9ca0
Fix: The Python 3.8 grammar did not include f-string support
2019-01-13 15:51:46 +01:00
Dave Halter
dd1761da96
Fix tokenizer: Closing parentheses in the wrong place should not lead to strange behavior
2019-01-13 14:52:33 +01:00
Dave Halter
e10802ab09
Fix end positions with error dedents
2019-01-13 14:14:16 +01:00
Dave Halter
3d402d0a77
Fix diff parser tests for Python 2
2019-01-10 09:26:42 +01:00
Dave Halter
f6a8b997f2
Randomize the fuzzer a bit more with inserting characters
2019-01-10 01:22:24 +01:00
Dave Halter
94c2681c8e
Simplify the regexes
2019-01-10 01:21:56 +01:00
Dave Halter
610a820799
Fix a regex clause that was totally wrong
2019-01-10 01:00:08 +01:00
Dave Halter
57320af6eb
Fix another tokenizer issue
2019-01-09 00:55:54 +01:00
Dave Halter
574e1c63e8
Apply \r changes in syntax trees
2019-01-09 00:34:19 +01:00
Dave Halter
fbaad7883f
Actually make \r usable
2019-01-08 20:03:08 +01:00
Dave Halter
b1f613fe16
Fix split lines for Python code
...
Some characters like Vertical Tab or File Separator were used as line separators.
This is not legal. Line Separators in Python are only Carriage Return \r and Line Feed \n.
2019-01-08 08:42:30 +01:00
Dave Halter
f4696a6245
Add \r as a valid linebreak for splitlines
2019-01-07 18:46:16 +01:00
Dave Halter
48c1a0e590
Move split_lines tests around
2019-01-07 18:40:34 +01:00
Dave Halter
6f63147f69
Start generating really random strings with the fuzzer
2019-01-06 20:51:49 +01:00
Dave Halter
94bd48bae1
Fix tokenizer: Dedents before error tokens are properly done, now.
2019-01-06 19:26:49 +01:00
Dave Halter
edbceba4f8
Fix diff parser: Also check async with
2019-01-06 16:25:28 +01:00
Dave Halter
b33c2b3ae1
Make the diff parser use a lot of different files by default
2019-01-06 15:43:37 +01:00
Dave Halter
65a0748f4f
Fix diff parser: Forgot that with statments are also flows
2019-01-06 15:41:16 +01:00
Dave Halter
c442cf98be
Fix valid graph asserting for some dedents that are errors
2019-01-06 12:39:04 +01:00
