Dave Halter
db10b4fa72
Diff parser: Need to care for eror dedents in some open parentheses/always break contexts
2020-04-05 14:39:56 +02:00
Dave Halter
ed38518052
Diff parser: Make sure that nested suites get properly copied
2020-04-05 02:48:41 +02:00
Dave Halter
ebc69545c7
Fix error recovery for multi line strings at the end of the file
2020-04-05 00:13:55 +02:00
Dave Halter
67ebb6acac
async is actually a token that cannot appear in brackets
2020-04-04 23:14:10 +02:00
Dave Halter
bcf76949b6
Diff parser: Remove error statements before caring about nested functions
2020-04-04 22:43:33 +02:00
Dave Halter
6c7b397cc7
Diff parser: Check indentation for copies correctly
2020-04-04 20:36:19 +02:00
Dave Halter
1927ba7254
Start using the parser count/copy count again
2020-04-04 17:49:35 +02:00
Dave Halter
a6c33411d4
Remove all the error dedent/indent additions in the diff parser
...
The parser should just reparse stuff that is strangely indented
2020-04-04 16:15:17 +02:00
Dave Halter
f8dce76ef7
Make sure to only copy nodes that have the same indentation in diff parser
2020-04-04 16:07:54 +02:00
Dave Halter
3242e36859
Python 2 compatibility
2020-04-04 15:45:03 +02:00
Dave Halter
734a4b0e67
Remove support for specialized treatment of form feeds
...
This is a very intentional change. Previously form feeds were handled very
poorly and sometimes where not counted as indentation. This obviously makes
sense. But at the same time indentation is very tricky to deal with (both for
editors and parso).
Especially in the diff parser this led to a lot of very weird issues. The
decision probably makes sense since:
1. Almost nobody uses form feeds in the first place.
2. People that use form feeds like Barry Warsaw often put a newline ater them.
(e.g Python's email.__init__)
3. If you write an editor you want to be able to identify a unicode character
with a clear line/column. This would not be the case if form feeds were just
ignored when counting.
Form feeds will still work in Jedi, will not cause parse errors and in general
you should be fine using them. It might just cause Jedi to count them as
indentation **if** you use it like '\f foo()'. This is however confusing for
most editors anyway. It leads to a weird display e.g. in VIM, even if it's
perfectly valid code in Python.
Since parso is a code analysis parser and not the languages parser I think it's
fine to ignore this edge case.
2020-04-04 15:38:10 +02:00
Dave Halter
1047204654
Small tokenizer refactoring
2020-04-04 13:13:00 +02:00
Dave Halter
ae6af7849e
Diff parser: All indent checks should use _get_indent
2020-04-04 13:08:47 +02:00
Dave Halter
e1632cdadc
Fix some issues with async funcs
2020-04-04 04:01:15 +02:00
Dave Halter
7f0dd35c37
Remove the piece of shit _get_insertion_node function
2020-04-04 03:51:28 +02:00
Dave Halter
ad88783ac9
Remove get_first_indentation
2020-04-03 16:47:00 +02:00
Dave Halter
8550a52e48
Remove indents from _NodesTreeNode
2020-04-03 16:26:01 +02:00
Dave Halter
c88a736e35
Fix indent issues
2020-04-03 16:24:26 +02:00
Dave Halter
a07146f8a5
Deal with indents in diff parser more explicitly
2020-04-03 12:41:28 +02:00
Dave Halter
0c0aa31a91
Don't use max as a variable
2020-04-03 03:35:21 +02:00
Dave Halter
77327a4cea
Make node insertion a bit easier
2020-04-03 03:28:14 +02:00
Dave Halter
8bbd304eb9
Define token types a bit different in diff parser
2020-04-03 01:05:11 +02:00
Dave Halter
62fd03edda
Pass tokens in diff tokenizer
2020-04-03 01:01:37 +02:00
Dave Halter
12063d42fc
When debugging print
2020-04-03 00:56:59 +02:00
Dave Halter
c86af743df
Initialize start pos properly in diff parser
2020-04-03 00:54:13 +02:00
Dave Halter
fb2ea551d5
Move the tokenizer/diff parser closer together
2020-04-03 00:18:35 +02:00
Dave Halter
ce170e8aae
WIP: Try to use the tokenizer in a more native way
2020-04-02 02:00:35 +02:00
Dave Halter
d674bc9895
Fix a backslash issue
2020-03-29 23:59:53 +02:00
Dave Halter
0d9886c22a
Diff parser: Rewrite tokenizer modifications a bit
2020-03-29 22:41:59 +02:00
Dave Halter
9f8a68677d
Tokenizer: It's now clearer when an error dedent appears
2020-03-29 13:50:36 +02:00
Dave Halter
a950b82066
Fix tokenizer for random invalid unicode points
2020-03-28 21:02:04 +01:00
Dave Halter
38b7763e9a
Use _assert_nodes_are_equal in the fuzzer
2020-03-28 14:51:27 +01:00
Dave Halter
cf880f43d4
Tokenizer: Add error dedents only if parens are not open
2020-03-28 14:41:10 +01:00
Dave Halter
8e49d8ab5f
Fix tokenizer fstring end positions
2020-03-28 11:22:32 +01:00
Dave Halter
77b3ad5843
Small flake8 refactoring
2020-03-28 10:41:00 +01:00
Dave Halter
29e3545241
Fix adding error indents/dedents only at the right places
2020-03-27 17:05:05 +01:00
Dave Halter
3d95b65b21
Fix an issue with unfinished f string literals
2020-03-27 11:17:31 +01:00
Dave Halter
b86ea25435
Add a bit to the CHANGELOG
2020-03-24 22:38:18 +01:00
Dave Halter
4c42a82ebc
Allow multiple newlines in a suite, this makes the diff parser easier
2020-03-24 22:35:21 +01:00
Dave Halter
43651ef219
Diff parser: Make sure dedent start pos are matching
2020-03-24 22:27:04 +01:00
Dave Halter
419d9e3174
Diff parser: Fix a few more indentation issues
2020-03-24 22:03:29 +01:00
Dave Halter
2bef3cf6ff
Fix an issue where indents where repeated unnessecarily
2020-03-24 00:24:53 +01:00
Dave Halter
8e95820d78
Don't show logs in pytest, because they already appear by default
2020-03-23 23:53:23 +01:00
Dave Halter
c18c89eb6b
Diff parser: Correctly add indent issues
2020-03-23 00:16:47 +01:00
Dave Halter
afc556d809
Diff parser: Prepare for indent error leaf insertion
2020-03-22 22:57:58 +01:00
Dave Halter
cdb791fbdb
Diff parser: Add error dedents if necessary, see also davidhalter/jedi#1499
2020-03-22 21:37:25 +01:00
Dave Halter
93f1cdebbc
Try to make parsed trees more similar for incomplete dedents, see also davidhalter/jedi#1499
2020-03-22 21:15:22 +01:00
Dave Halter
d3ceafee01
Specify in tests how another dedent issue is recovered from
2020-03-22 19:34:12 +01:00
Dave Halter
237dc9e135
Diff parser: Make sure to pop nodes directly after error nodes, see also davidhalter/jedi#1499
2020-03-22 14:49:22 +01:00
Dave Halter
bd37353042
Move a bit of code
2020-03-22 13:46:13 +01:00
