734a4b0e67
Remove support for specialized treatment of form feeds
...
This is a very intentional change. Previously form feeds were handled very
poorly and sometimes where not counted as indentation. This obviously makes
sense. But at the same time indentation is very tricky to deal with (both for
editors and parso).
Especially in the diff parser this led to a lot of very weird issues. The
decision probably makes sense since:
1. Almost nobody uses form feeds in the first place.
2. People that use form feeds like Barry Warsaw often put a newline ater them.
(e.g Python's email.__init__)
3. If you write an editor you want to be able to identify a unicode character
with a clear line/column. This would not be the case if form feeds were just
ignored when counting.
Form feeds will still work in Jedi, will not cause parse errors and in general
you should be fine using them. It might just cause Jedi to count them as
indentation **if** you use it like '\f foo()'. This is however confusing for
most editors anyway. It leads to a weird display e.g. in VIM, even if it's
perfectly valid code in Python.
Since parso is a code analysis parser and not the languages parser I think it's
fine to ignore this edge case.
2020-04-04 15:38:10 +02:00
1047204654
Small tokenizer refactoring
2020-04-04 13:13:00 +02:00
ae6af7849e
Diff parser: All indent checks should use _get_indent
2020-04-04 13:08:47 +02:00
e1632cdadc
Fix some issues with async funcs
2020-04-04 04:01:15 +02:00
7f0dd35c37
Remove the piece of shit _get_insertion_node function
2020-04-04 03:51:28 +02:00
ad88783ac9
Remove get_first_indentation
2020-04-03 16:47:00 +02:00
8550a52e48
Remove indents from _NodesTreeNode
2020-04-03 16:26:01 +02:00
c88a736e35
Fix indent issues
2020-04-03 16:24:26 +02:00
a07146f8a5
Deal with indents in diff parser more explicitly
2020-04-03 12:41:28 +02:00
0c0aa31a91
Don't use max as a variable
2020-04-03 03:35:21 +02:00
77327a4cea
Make node insertion a bit easier
2020-04-03 03:28:14 +02:00
8bbd304eb9
Define token types a bit different in diff parser
2020-04-03 01:05:11 +02:00
62fd03edda
Pass tokens in diff tokenizer
2020-04-03 01:01:37 +02:00
12063d42fc
When debugging print
2020-04-03 00:56:59 +02:00
c86af743df
Initialize start pos properly in diff parser
2020-04-03 00:54:13 +02:00
fb2ea551d5
Move the tokenizer/diff parser closer together
2020-04-03 00:18:35 +02:00
ce170e8aae
WIP: Try to use the tokenizer in a more native way
2020-04-02 02:00:35 +02:00
d674bc9895
Fix a backslash issue
2020-03-29 23:59:53 +02:00
0d9886c22a
Diff parser: Rewrite tokenizer modifications a bit
2020-03-29 22:41:59 +02:00
9f8a68677d
Tokenizer: It's now clearer when an error dedent appears
2020-03-29 13:50:36 +02:00
a950b82066
Fix tokenizer for random invalid unicode points
2020-03-28 21:02:04 +01:00
38b7763e9a
Use _assert_nodes_are_equal in the fuzzer
2020-03-28 14:51:27 +01:00
cf880f43d4
Tokenizer: Add error dedents only if parens are not open
2020-03-28 14:41:10 +01:00
8e49d8ab5f
Fix tokenizer fstring end positions
2020-03-28 11:22:32 +01:00
77b3ad5843
Small flake8 refactoring
2020-03-28 10:41:00 +01:00
29e3545241
Fix adding error indents/dedents only at the right places
2020-03-27 17:05:05 +01:00
3d95b65b21
Fix an issue with unfinished f string literals
2020-03-27 11:17:31 +01:00
b86ea25435
Add a bit to the CHANGELOG
2020-03-24 22:38:18 +01:00
4c42a82ebc
Allow multiple newlines in a suite, this makes the diff parser easier
2020-03-24 22:35:21 +01:00
43651ef219
Diff parser: Make sure dedent start pos are matching
2020-03-24 22:27:04 +01:00
419d9e3174
Diff parser: Fix a few more indentation issues
2020-03-24 22:03:29 +01:00
2bef3cf6ff
Fix an issue where indents where repeated unnessecarily
2020-03-24 00:24:53 +01:00
8e95820d78
Don't show logs in pytest, because they already appear by default
2020-03-23 23:53:23 +01:00
c18c89eb6b
Diff parser: Correctly add indent issues
2020-03-23 00:16:47 +01:00
afc556d809
Diff parser: Prepare for indent error leaf insertion
2020-03-22 22:57:58 +01:00
cdb791fbdb
Diff parser: Add error dedents if necessary, see also davidhalter/jedi#1499
2020-03-22 21:37:25 +01:00
93f1cdebbc
Try to make parsed trees more similar for incomplete dedents, see also davidhalter/jedi#1499
2020-03-22 21:15:22 +01:00
d3ceafee01
Specify in tests how another dedent issue is recovered from
2020-03-22 19:34:12 +01:00
237dc9e135
Diff parser: Make sure to pop nodes directly after error nodes, see also davidhalter/jedi#1499
2020-03-22 14:49:22 +01:00
bd37353042
Move a bit of code
2020-03-22 13:46:13 +01:00
51a044cc70
Fix diff parser: Invalid dedents meant that sometimes the wrong parents were chosen, fixes davidhalter/jedi#1499
2020-03-22 12:41:19 +01:00
2cd0d6c9fc
Fix: Dedent omission was wrong, see davidhalter/jedi#1499
2020-03-22 12:41:19 +01:00
287a86c242
ci: Travis: use Python 3.8.2
...
Ref: https://github.com/davidhalter/parso/issues/103
2020-02-28 00:51:06 +01:00
0234a70e95
Python 3.8.2 was released and an error message changed, fixes #103
2020-02-28 00:31:58 +01:00
7ba49a9695
Prepare the 0.6.2 release
2020-02-27 02:10:06 +01:00
53da7e8e6b
Fix get_next_sibling on module, fixes #102
2020-02-21 18:31:13 +01:00
6dd29c8efb
Fix ExprStmt.get_rhs for annotations
2020-02-21 18:31:13 +01:00
e4a9cfed86
Give parso refactoring tools
2020-02-21 18:31:13 +01:00
a7f4499644
Add venv to .gitignore ( #101 )
2020-02-14 14:28:07 +01:00
4306e8b34b
Change the release date for 0.6.1
2020-02-03 21:46:25 +01:00
Dave Halter