Dave Halter
4c65368056
Some minor changes to file_io
2019-03-27 01:02:27 +01:00
Dave Halter
3e2956264c
Add FileIO to make it possible to cache e.g. files from zip files
2019-03-25 00:48:59 +01:00
Dave Halter
c7c464e5e9
Avoid nasty side effects in creation of Node
...
This issue led to bugs in Jedi, because Jedi used the nodes in a wrong way.
2019-01-01 23:35:20 +01:00
Dave Halter
e20f2069ba
Move the grammar to a fitting file.
2018-06-26 10:20:05 +02:00
Dave Halter
03de9cebb8
Introduce TokenTypes
2018-06-24 16:24:09 +02:00
Dave Halter
532aef2342
Remove nonterminal2number and number2nonterminal, they are no longer used
2018-06-22 12:52:44 +02:00
Dave Halter
73ce57428b
Try to completely remove the word symbol and use nonterminal
...
The ones that we could not remove are in grammar.py, because that's the public documented API.
2018-06-17 18:30:20 +02:00
Jonas Tranberg
bf5a4b7c2c
Added path param to load_grammar for loading custom grammar files
2018-04-19 10:16:33 +02:00
Dave Halter
8f1a436ba1
Remove the old f-string grammar and fix the tests with the new syntax
2018-04-07 02:11:26 +02:00
lcolaholicl
e689f3dce6
Fix typo
...
keyworda → keyword
2017-12-30 13:52:33 +01:00
Dave Halter
a29ec25598
Use os.path.join instead of a slash
...
Fixes #20
2017-11-04 14:58:57 +01:00
Mike Morearty
04360cdfe7
Load from cache even when code was given
...
Change Grammar._parse() to try load_module() even if code was passed
to it.
In many cases, _parse() is passed both the code and the path to the .py
file. E.g. in jedi-vim if you type "import foo.", then every .py file in
directory "foo" will reach Grammar._parse() with both the `code` and the
`path` variables filled in. This change allows the cache to be used in
those cases.
2017-11-04 11:16:28 +01:00
Dave Halter
2067845cef
Try to recommend using parso without diff_cache.
2017-09-04 22:03:12 +02:00
Dave Halter
7a277c7302
Move to python 3.6 intersphinx.
2017-09-03 23:32:43 +02:00
Dave Halter
435d310c2b
More general overhaul of the documentation.
2017-09-03 22:20:03 +02:00
Dave Halter
8aa280342a
Rework the parse documentation.
2017-09-03 22:08:28 +02:00
Dave Halter
60ed141d80
A few documentation improvements.
2017-09-03 14:02:53 +02:00
Dave Halter
5c3304f7c2
Move start symbols a bit.
2017-08-31 19:39:27 +02:00
Dave Halter
b921e280b0
Some last fstring fixes.
2017-08-28 18:10:30 +02:00
Dave Halter
66606403c7
Most fstrings issues should be fixed now.
2017-08-26 19:15:09 +02:00
Dave Halter
ede8a2139f
Add some first f-string issues.
2017-08-25 22:09:58 +02:00
Dave Halter
3c3e7f5317
Make some parsing work.
2017-08-24 01:39:17 +02:00
Dave Halter
033e880408
Add fstrings to the grammar.
2017-08-24 00:01:42 +02:00
Dave Halter
929593701a
Remove opmap from pgen.
2017-08-22 08:45:10 +02:00
Dave Halter
51f2de28c6
source_to_unicode -> python_bytes_to_unicode.
2017-08-15 20:07:24 +02:00
Dave Halter
ab027885c7
Refactor splitlines -> split_lines.
2017-08-15 19:54:21 +02:00
Dave Halter
86aa185136
Use source_to_unicode by default with errors='strict'.
2017-08-15 19:27:27 +02:00
Dave Halter
2322b3aed3
Don't use pickle in pypy. It's really slow. (Armin Rigo told me a while ago).
2017-08-06 18:08:29 +02:00
Dave Halter
8c3373a0d9
Rename normalizer -> errors.
2017-08-06 17:12:22 +02:00
Dave Halter
524f332e7d
Small refactoring.
2017-08-06 16:11:31 +02:00
Dave Halter
90edb2d0cf
Move the iter_errors and normalizer methods to grammar.py.
2017-08-06 15:32:50 +02:00
Dave Halter
31675e74a2
Add a python3.3 grammar, this is just cleaner IMO.
2017-08-06 03:00:14 +02:00
Dave Halter
d6c624bd34
Finally fix Python 2.6.
...
Also added a syntax for 2.6. There are some thing that just don't work in 2.6.
2017-07-31 21:33:59 +02:00
Dave Halter
78c371f73a
Refactor the version info to use a tuple, always.
2017-07-19 09:09:33 +02:00
Dave Halter
f6b3b1cc24
Make version a keyword only argument in load_grammar.
2017-07-14 18:28:03 +02:00
Dave Halter
9d10ceeff1
Give jedi a better API for tokenizing.
2017-07-13 17:25:35 +02:00
Dave Halter
e731eecdd8
Make the tokenizer version independent.
2017-07-11 23:29:44 +02:00
Dave Halter
8ad37f6036
Move parsing the version around.
2017-07-09 00:49:23 +02:00
Dave Halter
c28ec5f182
Move tokenize.py and token.py to the python folder.
2017-07-09 00:06:14 +02:00
Dave Halter
814b16cc6c
Remove the remove_newline hack and build that stuff into error recovery instead.
...
Tests are passing except for diff parser tests.
2017-05-31 21:24:24 +02:00
Dave Halter
7c1f3b3a65
Use escapes for some regex expressions.
2017-05-26 14:19:06 -04:00
Dave Halter
406064c457
Remove a place of saving a module that was not necessary.
2017-05-26 13:43:34 -04:00
Dave Halter
dafffdc9b4
The parser cache in RAM has now grammar versioning.
2017-05-26 12:48:59 -04:00
Dave Halter
8ae5fc60d6
Moved the grammar object out of the cache module.
2017-05-26 12:17:45 -04:00
Dave Halter
976c5151af
Rename source_tokens to tokenize and generate_tokens to tokenize_lines.
2017-05-26 12:01:35 -04:00
Dave Halter
b4259b2b4f
Change the default of use_exact_op_types in the tokenizer.
2017-05-26 11:32:00 -04:00
Dave Halter
6283577bc3
Actually implement the load_grammar properly.
2017-05-25 16:45:29 -04:00
Dave Halter
b037321d1e
Change the docstring for the cache parameter.
2017-05-25 14:40:09 -04:00
Dave Halter
68038a82ee
Reorder some arguments.
2017-05-25 13:44:19 -04:00
Dave Halter
b7df52edb5
Use the diff parser as well.
2017-05-25 13:41:45 -04:00
