 Dave Halter
|
40be00826e
|
clean up tokenize
|
2014-02-25 17:17:33 +01:00 |
|
|
Dave Halter
|
761c28ef00
|
remove __getitem__ from Token
|
2014-02-25 17:03:56 +01:00 |
|
|
Dave Halter
|
66aca8eba1
|
user_context now doesn't work with the token tuple anymore
|
2014-02-25 16:49:44 +01:00 |
|
|
Dave Halter
|
18e985a961
|
TokenInfo -> Token
|
2014-02-25 16:44:48 +01:00 |
|
|
Dave Halter
|
6439d6c848
|
removed the now redundant token.Token version in favor of tokenize.TokenInfo
|
2014-02-25 16:38:19 +01:00 |
|
|
Dave Halter
|
2db26abf72
|
start and end don't exst anymore in parser.token.Token, it's now start_pos/end_pos as everywhere else
|
2014-02-25 16:34:27 +01:00 |
|
|
Dave Halter
|
ee7108cc11
|
successfully removed __str__ and __unicode__ methods from token.Token
|
2014-02-25 16:21:53 +01:00 |
|
|
Dave Halter
|
aea2c4620f
|
more unicode switches in the parser
|
2014-02-25 14:27:50 +01:00 |
|
|
Dave Halter
|
f4f79317fe
|
start uniting tokenize.TokenInfo and token.Token
|
2014-02-25 13:54:18 +01:00 |
|
|
Dave Halter
|
5b84f0b27f
|
remove end_pos stuff from tokenizer, the tokens can do that themselves
|
2014-02-25 13:29:27 +01:00 |
|
|
Dave Halter
|
2252271bf5
|
fix unicode issues with python2.7
|
2014-02-25 12:54:06 +01:00 |
|
|
Dave Halter
|
3a23c80ae5
|
prepare for eventual? tokenizer end_pos replacement.
|
2014-02-25 11:59:10 +01:00 |
|
|
Dave Halter
|
246118f851
|
start using @ganwell's new token class (modified in some ways) as the main token class - hope to gain a little bit of memory/cpu/pickling performance
|
2014-02-25 02:06:26 +01:00 |
|
|
Dave Halter
|
e2a6d1dd43
|
remove offset param for Parser
|
2014-02-25 01:31:24 +01:00 |
|
|
Dave Halter
|
44d560c53a
|
finally removed self-contained iterator from Parser - after knowing this since almost the beginning.
|
2014-02-25 01:26:19 +01:00 |
|
|
Dave Halter
|
a7a8a73a2c
|
removed Parser._start_pos as well
|
2014-02-25 01:20:55 +01:00 |
|
|
Dave Halter
|
add8259d7e
|
successfully removed end_pos from parser
|
2014-02-25 01:05:00 +01:00 |
|
|
Dave Halter
|
936c7dfde4
|
protect start_pos/end_pos in parser to hopefully remove them soon
|
2014-02-25 00:27:39 +01:00 |
|
|
Dave Halter
|
ef8d3633dd
|
use the module end_pos in fast parser
|
2014-02-25 00:18:24 +01:00 |
|
|
Dave Halter
|
a5a6e9ac12
|
parser: remove current
|
2014-02-24 20:43:00 +01:00 |
|
|
Dave Halter
|
50f8b8bf0c
|
start using tokens all the way through in the parser
|
2014-02-24 20:35:36 +01:00 |
|
|
Dave Halter
|
6058e8b9c3
|
remove unnecessary checks
|
2014-02-24 11:40:05 +01:00 |
|
|
Dave Halter
|
9943bb6205
|
remove some old parameters from Parser and FastTokenizer
|
2014-02-24 11:24:54 +01:00 |
|
|
Dave Halter
|
7db090a48a
|
moved NoErrorTokenizer to fast.FastTokenizer
|
2014-02-24 11:05:31 +01:00 |
|
|
Dave Halter
|
9257062910
|
remove NoErrorTokenizer dependency for all but the fast parsers
|
2014-02-24 10:31:15 +01:00 |
|
|
Dave Halter
|
553ff66c8b
|
remove last_previous from NoErrorTokenizer
|
2014-02-23 12:51:05 +01:00 |
|
|
Dave Halter
|
8e847f4982
|
fix python 2.7 issues. the parser now only takes unicode inputs
|
2014-02-23 11:44:32 +01:00 |
|
|
Dave Halter
|
c5fcebde82
|
changed _compatibility.utf8 -> 'u' and removed a lot of the issues with the now enforced unicode source input of the parser
|
2014-02-23 11:29:00 +01:00 |
|
|
Dave Halter
|
5478e50f8b
|
Merge branch 'dev' of github.com:davidhalter/jedi into dev
|
2014-02-21 17:52:44 +01:00 |
|
|
Dave Halter
|
9c2fcfffd7
|
StringIo is now always used as part of the io library
|
2014-02-21 17:52:09 +01:00 |
|
|
Dave Halter
|
d9bf9be35e
|
small refactorings of the tokenizer
|
2014-02-21 17:45:56 +01:00 |
|
|
Dave Halter
|
efba0cd80c
|
deleted a big part of the generate_tokens code that doesn't seem to be needed anymore
|
2014-02-21 16:38:15 +01:00 |
|
|
Dave Halter
|
73057d4176
|
removed the INDENT usages that were left in the parser code
|
2014-02-21 15:39:14 +01:00 |
|
|
Dave Halter
|
fe02088dd7
|
restructure user_context reverse tokenizer to remove INDENT tokens in the future
|
2014-02-21 15:30:46 +01:00 |
|
|
Dave Halter
|
9c3b7b9c46
|
get rid of more variables
|
2014-02-21 10:44:33 +01:00 |
|
|
Dave Halter
|
184e90a3a0
|
removed some line checks from tokenizer
|
2014-02-21 00:13:23 +01:00 |
|
|
Dave Halter
|
a69eee5876
|
error tokens only exist in the end in a tokenizer
|
2014-02-21 00:09:49 +01:00 |
|
|
Dave Halter
|
3232ae5b0c
|
removed parentheses counting from generate_tokens
|
2014-02-20 18:45:22 +01:00 |
|
|
Dave Halter
|
7e651684ff
|
fix problems with debug mode if not correctly initialized.
|
2014-02-20 14:17:05 +01:00 |
|
|
Dave Halter
|
d7033726fd
|
tokenize removed NL/NEWLINE ambiguity
|
2014-02-20 01:52:30 +01:00 |
|
|
Dave Halter
|
c26ae3c00d
|
ignore dedents in tokenizer
|
2014-02-20 01:21:20 +01:00 |
|
|
Dave Halter
|
3e9b72b636
|
created a PushBackTokenizer specifically for the parser
|
2014-02-20 01:17:19 +01:00 |
|
|
Dave Halter
|
c8d6fbb0a1
|
temporary changes for the tokenizer issues
|
2014-02-20 00:43:42 +01:00 |
|
|
Dave Halter
|
000e929e3c
|
Merge pull request #375 from ColinDuquesnoy/dev
Fix import of compiled module with python3.
|
2014-02-19 22:11:27 +04:30 |
|
|
ColinDuquesnoy
|
9d7ecae27c
|
Add precompiled extension modules
|
2014-02-19 08:14:51 +01:00 |
|
|
ColinDuquesnoy
|
2c97d01bd0
|
Move extensions dir into test
|
2014-02-19 08:10:43 +01:00 |
|
|
ColinDuquesnoy
|
f808dbbd28
|
Change sys.path for the test to succeed.
Tested locally with a python3 extension module (in
/extensions/compiled33).
Also tested that reverting a75773cf9f make
the test fail.
|
2014-02-18 22:14:43 +01:00 |
|
|
ColinDuquesnoy
|
7977d57169
|
Add test_compiled
Should pass on travis since there is no precompiled modules yet
|
2014-02-18 20:31:06 +01:00 |
|
|
ColinDuquesnoy
|
7aed62cb99
|
Add compiled modules sources + readme
|
2014-02-18 20:12:30 +01:00 |
|
|
ColinDuquesnoy
|
5a706265bc
|
Move dummy pyc to dummy package
|
2014-02-18 19:18:17 +01:00 |
|