Dave Halter
b9725364ab
Add a lot of comment to the diff parser
2020-04-13 11:46:36 +02:00
Dave Halter
66ecc264f9
Write 0.7.0 release notes
2020-04-13 11:15:05 +02:00
Dave Halter
63b73a05e6
Diff parser: Take care of one line function error recovery with decorator
2020-04-13 11:07:37 +02:00
Dave Halter
baec4ac58f
Diff parser: Take care of one line function error recovery
2020-04-12 02:47:46 +02:00
Dave Halter
b5f58ac33c
Ignore some slow files for the fuzzer
2020-04-12 01:14:24 +02:00
Dave Halter
83cb71f7a1
The fuzzer now tries to reuse previous modfiications as well sometimes
2020-04-11 23:29:00 +02:00
Dave Halter
30a2b2f40d
Fix an error case with prefixes
2020-04-11 22:51:17 +02:00
Dave Halter
d81e393c0c
Fix indentation issues with backslashes and def error recovery
2020-04-10 21:48:28 +02:00
Dave Halter
7822f8be84
Python 2 compatibility
2020-04-09 22:47:50 +02:00
Dave Halter
93788a3e09
Add a test for the diff parser that xfails
2020-04-09 00:03:39 +02:00
Dave Halter
085f666ca1
Add more tokens that can break parens to tokenizer
2020-04-08 23:24:30 +02:00
Dave Halter
9e546e42de
Diff parser: Fix another byte order mark issue
2020-04-07 22:58:47 +02:00
Dave Halter
7b14a86e0a
Fix tokenizer error tokens
2020-04-07 09:55:28 +02:00
Dave Halter
f45941226f
Diff parser: Fix other BOM issues
2020-04-07 01:06:03 +02:00
Dave Halter
e04552b14a
Fix tests for Python 2
2020-04-06 23:52:29 +02:00
Dave Halter
cd9c213a62
Fix fstring issues when error leaves are involved
2020-04-06 23:34:27 +02:00
Dave Halter
561e81df00
Replace non utf8 errors properly in diff fuzzer
2020-04-06 02:04:48 +02:00
Dave Halter
556ce86cde
Tokenizer: It should not be possible to break out of backslashes on the next line, even if it was an error
2020-04-06 01:25:06 +02:00
Dave Halter
b12dd498bb
Diff parser: Fix BOM with indentation issues
2020-04-05 20:47:49 +02:00
Dave Halter
db10b4fa72
Diff parser: Need to care for eror dedents in some open parentheses/always break contexts
2020-04-05 14:39:56 +02:00
Dave Halter
ed38518052
Diff parser: Make sure that nested suites get properly copied
2020-04-05 02:48:41 +02:00
Dave Halter
ebc69545c7
Fix error recovery for multi line strings at the end of the file
2020-04-05 00:13:55 +02:00
Dave Halter
67ebb6acac
async is actually a token that cannot appear in brackets
2020-04-04 23:14:10 +02:00
Dave Halter
bcf76949b6
Diff parser: Remove error statements before caring about nested functions
2020-04-04 22:43:33 +02:00
Dave Halter
6c7b397cc7
Diff parser: Check indentation for copies correctly
2020-04-04 20:36:19 +02:00
Dave Halter
1927ba7254
Start using the parser count/copy count again
2020-04-04 17:49:35 +02:00
Dave Halter
a6c33411d4
Remove all the error dedent/indent additions in the diff parser
...
The parser should just reparse stuff that is strangely indented
2020-04-04 16:15:17 +02:00
Dave Halter
f8dce76ef7
Make sure to only copy nodes that have the same indentation in diff parser
2020-04-04 16:07:54 +02:00
Dave Halter
3242e36859
Python 2 compatibility
2020-04-04 15:45:03 +02:00
Dave Halter
734a4b0e67
Remove support for specialized treatment of form feeds
...
This is a very intentional change. Previously form feeds were handled very
poorly and sometimes where not counted as indentation. This obviously makes
sense. But at the same time indentation is very tricky to deal with (both for
editors and parso).
Especially in the diff parser this led to a lot of very weird issues. The
decision probably makes sense since:
1. Almost nobody uses form feeds in the first place.
2. People that use form feeds like Barry Warsaw often put a newline ater them.
(e.g Python's email.__init__)
3. If you write an editor you want to be able to identify a unicode character
with a clear line/column. This would not be the case if form feeds were just
ignored when counting.
Form feeds will still work in Jedi, will not cause parse errors and in general
you should be fine using them. It might just cause Jedi to count them as
indentation **if** you use it like '\f foo()'. This is however confusing for
most editors anyway. It leads to a weird display e.g. in VIM, even if it's
perfectly valid code in Python.
Since parso is a code analysis parser and not the languages parser I think it's
fine to ignore this edge case.
2020-04-04 15:38:10 +02:00
Dave Halter
1047204654
Small tokenizer refactoring
2020-04-04 13:13:00 +02:00
Dave Halter
ae6af7849e
Diff parser: All indent checks should use _get_indent
2020-04-04 13:08:47 +02:00
Dave Halter
e1632cdadc
Fix some issues with async funcs
2020-04-04 04:01:15 +02:00
Dave Halter
7f0dd35c37
Remove the piece of shit _get_insertion_node function
2020-04-04 03:51:28 +02:00
Dave Halter
ad88783ac9
Remove get_first_indentation
2020-04-03 16:47:00 +02:00
Dave Halter
8550a52e48
Remove indents from _NodesTreeNode
2020-04-03 16:26:01 +02:00
Dave Halter
c88a736e35
Fix indent issues
2020-04-03 16:24:26 +02:00
Dave Halter
a07146f8a5
Deal with indents in diff parser more explicitly
2020-04-03 12:41:28 +02:00
Dave Halter
0c0aa31a91
Don't use max as a variable
2020-04-03 03:35:21 +02:00
Dave Halter
77327a4cea
Make node insertion a bit easier
2020-04-03 03:28:14 +02:00
Dave Halter
8bbd304eb9
Define token types a bit different in diff parser
2020-04-03 01:05:11 +02:00
Dave Halter
62fd03edda
Pass tokens in diff tokenizer
2020-04-03 01:01:37 +02:00
Dave Halter
12063d42fc
When debugging print
2020-04-03 00:56:59 +02:00
Dave Halter
c86af743df
Initialize start pos properly in diff parser
2020-04-03 00:54:13 +02:00
Dave Halter
fb2ea551d5
Move the tokenizer/diff parser closer together
2020-04-03 00:18:35 +02:00
Dave Halter
ce170e8aae
WIP: Try to use the tokenizer in a more native way
2020-04-02 02:00:35 +02:00
Dave Halter
d674bc9895
Fix a backslash issue
2020-03-29 23:59:53 +02:00
Dave Halter
0d9886c22a
Diff parser: Rewrite tokenizer modifications a bit
2020-03-29 22:41:59 +02:00
Dave Halter
9f8a68677d
Tokenizer: It's now clearer when an error dedent appears
2020-03-29 13:50:36 +02:00
Dave Halter
a950b82066
Fix tokenizer for random invalid unicode points
2020-03-28 21:02:04 +01:00
