Dave Halter
99dd4a84d4
Merge branch 'master' of github.com:davidhalter/parso
2019-07-12 21:35:06 +02:00
Dave Halter
9501b0bde0
Fixed name tokenizing issues for tamil characters, fixes davidhalter/jedi#1368
2019-07-12 21:31:49 +02:00
Benjamin Woodruff
ad57a51800
Fix line continuation characters inside f-strings
...
Line continuation characters are valid inside of strings, but weren't
handled correctly in certain cases with f-strings, due to some small
tokenizer bugs.
This pull request to address those issues, and adds tests to validate
the new logic.
2019-07-12 21:20:00 +02:00
Dave Halter
ecbe2b9926
Add positional only arguments to grammar
2019-06-09 21:15:03 +02:00
Dave Halter
a7aa23a7f0
Parse named expressions
2019-06-02 23:34:37 +02:00
Dave Halter
5430415d44
Change a test, because it doesn't really matter
...
The test had changed behavior for Python 3.8, a syntax error of:
SyntaxError: unexpected EOF while parsing
instead of
SyntaxError: invalid syntax
2019-06-02 22:54:45 +02:00
Dave Halter
6cdd47fe2b
f-string syntax in Python 3.8 was enhanced
...
See e.g. https://twitter.com/raymondh/status/1135253771846471680
2019-06-02 22:48:47 +02:00
Dave Halter
917b4421f3
Fix fstring format spec parsing, fixes #74
2019-06-02 15:18:42 +02:00
prim
93ddf5322a
parse long number notation ( #72 )
...
* parse long number notation
* parse long number notation
2019-06-02 11:14:15 +02:00
Dave Halter
b2ab64d8f9
Fix Python 3.8 error issues
2019-04-05 18:30:48 +02:00
Dave Halter
dfe7fba08e
continue in finally is no longer an error
2019-04-05 16:17:30 +02:00
Dave Halter
3e2956264c
Add FileIO to make it possible to cache e.g. files from zip files
2019-03-25 00:48:59 +01:00
Dave Halter
3f6fc8a5ad
Fix an f-string tokenizer issue
2019-02-13 00:17:37 +01:00
Dave Halter
58850f8bfa
Rename a test
2019-02-06 09:51:46 +01:00
Dave Halter
d38a60278e
Remove some unused code
2019-02-06 09:50:27 +01:00
Dave Halter
6c65aea47d
Fix working with async functions in the diff parser, fixes #56
2019-02-06 09:31:46 +01:00
Dave Halter
0d37ff865c
Fix bytes/fstring mixing when using iter_errors, fixes #57 .
2019-02-06 01:28:47 +01:00
Dave Halter
bb2855897b
Escape a backslash properly
2019-01-24 00:31:26 +01:00
Dave Halter
32a83b932a
Fix get_start_pos_of_prefix
2019-01-24 00:00:06 +01:00
Dave Halter
60e4591837
Fix: End detection for strings was mostly wrong, fixes #51
2019-01-23 10:13:25 +01:00
Dave Halter
ef56debb78
Fix f-string escapes, fixes #48
...
The tokenizer was not detecting backslash escapes for f-string endings properly
2019-01-22 22:20:32 +01:00
Dave Halter
ce8b531175
Fix diff parser: The previous fix was a bit off
2019-01-20 19:03:45 +01:00
Dave Halter
069c08883a
Change fuzzer: Add ways to not always use correct parse input
2019-01-20 18:18:13 +01:00
Dave Halter
0da0a8655a
Fix diff parser: issue with opening brackets
2019-01-20 00:41:11 +01:00
Dave Halter
3d890c3a00
Async doesn't work in 3.4
2019-01-19 12:59:35 +01:00
Dave Halter
956ea55048
Skip some tests for Python2.6 and Python3.3
2019-01-19 12:08:39 +01:00
Dave Halter
0bd17bee2c
Fix diff parser: DEDENT as error leaves should also be ignored and reparsed
2019-01-18 18:41:08 +01:00
Dave Halter
f3015efb2d
Fix diff parser: error dedents in between nodes should be ignored for now when copying
2019-01-18 02:43:12 +01:00
Dave Halter
197391dc53
Fix diff parser: Don't copy error nodes/leaves in the beginning, leads to strange issues in some cases
2019-01-17 23:48:00 +01:00
Dave Halter
32321a74b1
Diff fuzzer: Create a check to see if the errors make sense.
2019-01-17 22:07:59 +01:00
Dave Halter
52d01685ba
Fix diff parser: Don't copy DEDENT tokens at the beginning
2019-01-17 21:31:13 +01:00
Dave Halter
e591b929eb
Fix diff parser: Skip last leaves for last line offset leaves
2019-01-17 00:15:38 +01:00
Dave Halter
dac4c445a7
Fix indentation error tokens
2019-01-16 23:21:31 +01:00
Dave Halter
20fd32b45d
Fix diff parser: Avoid side effects for prefix
2019-01-14 21:37:19 +01:00
Dave Halter
9cc8178998
Fix tokenizer: backslashes sometimes led to newline token generation
2019-01-14 09:59:16 +01:00
Dave Halter
1e25445176
Make lines easier copyable in the fuzzer
2019-01-14 01:50:39 +01:00
Dave Halter
d7171ae927
Fix tokenizer: Carriage returns after backslashes were not properly handled
2019-01-14 01:49:09 +01:00
Dave Halter
d3d28480ed
Fix in diff parser: prefix calculation was wrong when copying nodes
2019-01-14 01:00:17 +01:00
Dave Halter
564be7882e
Replace --print-diff with --print-code
2019-01-14 00:20:49 +01:00
Dave Halter
76c5754b76
Fix diff parser generation for empty files
2019-01-13 23:38:35 +01:00
Dave Halter
55247a5a2c
Docopt should not be needed for tests
2019-01-13 23:24:17 +01:00
Dave Halter
7ae1efe5c7
Fix tokenizer: Form feeds and multiline docstrings didn't work together
2019-01-13 23:16:09 +01:00
Dave Halter
01dba7f8ce
Fix diff parser: Need to calculate the prefix for the diff tokenizer better
2019-01-13 22:38:53 +01:00
Dave Halter
a7e24a37e7
Fix newline endings and a few parser/copy counts
2019-01-13 19:55:18 +01:00
Dave Halter
f80d9de7a0
Feature: The diff parser fuzzer is now able to use random Python fragments
...
This hopefully leads to the fuzzer finding more and faster issues in the diff
parser.
2019-01-13 16:00:36 +01:00
Dave Halter
dd1761da96
Fix tokenizer: Closing parentheses in the wrong place should not lead to strange behavior
2019-01-13 14:52:33 +01:00
Dave Halter
e10802ab09
Fix end positions with error dedents
2019-01-13 14:14:16 +01:00
Dave Halter
3d402d0a77
Fix diff parser tests for Python 2
2019-01-10 09:26:42 +01:00
Dave Halter
f6a8b997f2
Randomize the fuzzer a bit more with inserting characters
2019-01-10 01:22:24 +01:00
Dave Halter
94c2681c8e
Simplify the regexes
2019-01-10 01:21:56 +01:00
