1
0
forked from VimPlug/jedi
Commit Graph

3115 Commits

Author SHA1 Message Date
Dave Halter
2252271bf5 fix unicode issues with python2.7 2014-02-25 12:54:06 +01:00
Dave Halter
3a23c80ae5 prepare for eventual? tokenizer end_pos replacement. 2014-02-25 11:59:10 +01:00
Dave Halter
246118f851 start using @ganwell's new token class (modified in some ways) as the main token class - hope to gain a little bit of memory/cpu/pickling performance 2014-02-25 02:06:26 +01:00
Dave Halter
e2a6d1dd43 remove offset param for Parser 2014-02-25 01:31:24 +01:00
Dave Halter
44d560c53a finally removed self-contained iterator from Parser - after knowing this since almost the beginning. 2014-02-25 01:26:19 +01:00
Dave Halter
a7a8a73a2c removed Parser._start_pos as well 2014-02-25 01:20:55 +01:00
Dave Halter
add8259d7e successfully removed end_pos from parser 2014-02-25 01:05:00 +01:00
Dave Halter
936c7dfde4 protect start_pos/end_pos in parser to hopefully remove them soon 2014-02-25 00:27:39 +01:00
Dave Halter
ef8d3633dd use the module end_pos in fast parser 2014-02-25 00:18:24 +01:00
Dave Halter
a5a6e9ac12 parser: remove current 2014-02-24 20:43:00 +01:00
Dave Halter
50f8b8bf0c start using tokens all the way through in the parser 2014-02-24 20:35:36 +01:00
Dave Halter
6058e8b9c3 remove unnecessary checks 2014-02-24 11:40:05 +01:00
Dave Halter
9943bb6205 remove some old parameters from Parser and FastTokenizer 2014-02-24 11:24:54 +01:00
Dave Halter
7db090a48a moved NoErrorTokenizer to fast.FastTokenizer 2014-02-24 11:05:31 +01:00
Dave Halter
9257062910 remove NoErrorTokenizer dependency for all but the fast parsers 2014-02-24 10:31:15 +01:00
Dave Halter
553ff66c8b remove last_previous from NoErrorTokenizer 2014-02-23 12:51:05 +01:00
Dave Halter
8e847f4982 fix python 2.7 issues. the parser now only takes unicode inputs 2014-02-23 11:44:32 +01:00
Dave Halter
c5fcebde82 changed _compatibility.utf8 -> 'u' and removed a lot of the issues with the now enforced unicode source input of the parser 2014-02-23 11:29:00 +01:00
Dave Halter
5478e50f8b Merge branch 'dev' of github.com:davidhalter/jedi into dev 2014-02-21 17:52:44 +01:00
Dave Halter
9c2fcfffd7 StringIo is now always used as part of the io library 2014-02-21 17:52:09 +01:00
Dave Halter
d9bf9be35e small refactorings of the tokenizer 2014-02-21 17:45:56 +01:00
Dave Halter
efba0cd80c deleted a big part of the generate_tokens code that doesn't seem to be needed anymore 2014-02-21 16:38:15 +01:00
Dave Halter
73057d4176 removed the INDENT usages that were left in the parser code 2014-02-21 15:39:14 +01:00
Dave Halter
fe02088dd7 restructure user_context reverse tokenizer to remove INDENT tokens in the future 2014-02-21 15:30:46 +01:00
Dave Halter
9c3b7b9c46 get rid of more variables 2014-02-21 10:44:33 +01:00
Dave Halter
184e90a3a0 removed some line checks from tokenizer 2014-02-21 00:13:23 +01:00
Dave Halter
a69eee5876 error tokens only exist in the end in a tokenizer 2014-02-21 00:09:49 +01:00
Dave Halter
3232ae5b0c removed parentheses counting from generate_tokens 2014-02-20 18:45:22 +01:00
Dave Halter
7e651684ff fix problems with debug mode if not correctly initialized. 2014-02-20 14:17:05 +01:00
Dave Halter
d7033726fd tokenize removed NL/NEWLINE ambiguity 2014-02-20 01:52:30 +01:00
Dave Halter
c26ae3c00d ignore dedents in tokenizer 2014-02-20 01:21:20 +01:00
Dave Halter
3e9b72b636 created a PushBackTokenizer specifically for the parser 2014-02-20 01:17:19 +01:00
Dave Halter
c8d6fbb0a1 temporary changes for the tokenizer issues 2014-02-20 00:43:42 +01:00
Dave Halter
000e929e3c Merge pull request #375 from ColinDuquesnoy/dev
Fix import of compiled module with python3.
2014-02-19 22:11:27 +04:30
ColinDuquesnoy
9d7ecae27c Add precompiled extension modules 2014-02-19 08:14:51 +01:00
ColinDuquesnoy
2c97d01bd0 Move extensions dir into test 2014-02-19 08:10:43 +01:00
ColinDuquesnoy
f808dbbd28 Change sys.path for the test to succeed.
Tested locally with a python3 extension module (in
/extensions/compiled33).

Also tested that reverting a75773cf9f make
the test fail.
2014-02-18 22:14:43 +01:00
ColinDuquesnoy
7977d57169 Add test_compiled
Should pass on travis since there is no precompiled modules yet
2014-02-18 20:31:06 +01:00
ColinDuquesnoy
7aed62cb99 Add compiled modules sources + readme 2014-02-18 20:12:30 +01:00
ColinDuquesnoy
5a706265bc Move dummy pyc to dummy package 2014-02-18 19:18:17 +01:00
ColinDuquesnoy
18a31dcbf1 Fix read mode for pyc files
Now all test should pass on travis.
2014-02-18 19:00:55 +01:00
ColinDuquesnoy
ea43efc9d8 Fix pyc test for python3
To import pyc modules, we must move them out of the __pycache__
directory and rename them to remove ".cpython-%s%d".

This should still faild with python3 (UnicodeDecodeError)
2014-02-18 19:00:55 +01:00
ColinDuquesnoy
d80caa7108 Add test pyc
Should succeed for python2 and faild for python3
2014-02-18 17:36:01 +01:00
Dave Halter
1b6df4602d replace parser tokenizer offset with line offset 2014-02-17 23:02:04 +01:00
Dave Halter
6952596117 implement an offset in generate_tokens 2014-02-17 10:08:32 +01:00
ColinDuquesnoy
a75773cf9f Fix import of compiled module with python3.
This at least fix #331
2014-02-17 09:08:30 +01:00
Dave Halter
bb111daf91 removed line from tokenizer 2014-02-16 15:28:18 +01:00
Dave Halter
22928dbcd0 simplified some crazy iterator stuff within parser 2014-02-16 15:04:57 +01:00
Dave Halter
82f27569b2 little changes to clean up the code (flake8) 2014-02-14 12:53:22 +01:00
Dave Halter
d704743422 more consistent __slots__ usage in the parser -> 20% memory decrease for wx 2014-02-14 00:56:42 +01:00