Dave Halter
|
18e985a961
|
TokenInfo -> Token
|
2014-02-25 16:44:48 +01:00 |
|
Dave Halter
|
6439d6c848
|
removed the now redundant token.Token version in favor of tokenize.TokenInfo
|
2014-02-25 16:38:19 +01:00 |
|
Dave Halter
|
2db26abf72
|
start and end don't exst anymore in parser.token.Token, it's now start_pos/end_pos as everywhere else
|
2014-02-25 16:34:27 +01:00 |
|
Dave Halter
|
ee7108cc11
|
successfully removed __str__ and __unicode__ methods from token.Token
|
2014-02-25 16:21:53 +01:00 |
|
Dave Halter
|
aea2c4620f
|
more unicode switches in the parser
|
2014-02-25 14:27:50 +01:00 |
|
Dave Halter
|
f4f79317fe
|
start uniting tokenize.TokenInfo and token.Token
|
2014-02-25 13:54:18 +01:00 |
|
Dave Halter
|
5b84f0b27f
|
remove end_pos stuff from tokenizer, the tokens can do that themselves
|
2014-02-25 13:29:27 +01:00 |
|
Dave Halter
|
2252271bf5
|
fix unicode issues with python2.7
|
2014-02-25 12:54:06 +01:00 |
|
Dave Halter
|
3a23c80ae5
|
prepare for eventual? tokenizer end_pos replacement.
|
2014-02-25 11:59:10 +01:00 |
|
Dave Halter
|
246118f851
|
start using @ganwell's new token class (modified in some ways) as the main token class - hope to gain a little bit of memory/cpu/pickling performance
|
2014-02-25 02:06:26 +01:00 |
|
Dave Halter
|
e2a6d1dd43
|
remove offset param for Parser
|
2014-02-25 01:31:24 +01:00 |
|
Dave Halter
|
44d560c53a
|
finally removed self-contained iterator from Parser - after knowing this since almost the beginning.
|
2014-02-25 01:26:19 +01:00 |
|
Dave Halter
|
a7a8a73a2c
|
removed Parser._start_pos as well
|
2014-02-25 01:20:55 +01:00 |
|
Dave Halter
|
add8259d7e
|
successfully removed end_pos from parser
|
2014-02-25 01:05:00 +01:00 |
|
Dave Halter
|
936c7dfde4
|
protect start_pos/end_pos in parser to hopefully remove them soon
|
2014-02-25 00:27:39 +01:00 |
|
Dave Halter
|
ef8d3633dd
|
use the module end_pos in fast parser
|
2014-02-25 00:18:24 +01:00 |
|
Dave Halter
|
a5a6e9ac12
|
parser: remove current
|
2014-02-24 20:43:00 +01:00 |
|
Dave Halter
|
50f8b8bf0c
|
start using tokens all the way through in the parser
|
2014-02-24 20:35:36 +01:00 |
|
Dave Halter
|
6058e8b9c3
|
remove unnecessary checks
|
2014-02-24 11:40:05 +01:00 |
|
Dave Halter
|
9943bb6205
|
remove some old parameters from Parser and FastTokenizer
|
2014-02-24 11:24:54 +01:00 |
|
Dave Halter
|
7db090a48a
|
moved NoErrorTokenizer to fast.FastTokenizer
|
2014-02-24 11:05:31 +01:00 |
|
Dave Halter
|
9257062910
|
remove NoErrorTokenizer dependency for all but the fast parsers
|
2014-02-24 10:31:15 +01:00 |
|
Dave Halter
|
553ff66c8b
|
remove last_previous from NoErrorTokenizer
|
2014-02-23 12:51:05 +01:00 |
|
Dave Halter
|
8e847f4982
|
fix python 2.7 issues. the parser now only takes unicode inputs
|
2014-02-23 11:44:32 +01:00 |
|
Dave Halter
|
c5fcebde82
|
changed _compatibility.utf8 -> 'u' and removed a lot of the issues with the now enforced unicode source input of the parser
|
2014-02-23 11:29:00 +01:00 |
|
Dave Halter
|
9c2fcfffd7
|
StringIo is now always used as part of the io library
|
2014-02-21 17:52:09 +01:00 |
|
Dave Halter
|
d9bf9be35e
|
small refactorings of the tokenizer
|
2014-02-21 17:45:56 +01:00 |
|
Dave Halter
|
efba0cd80c
|
deleted a big part of the generate_tokens code that doesn't seem to be needed anymore
|
2014-02-21 16:38:15 +01:00 |
|
Dave Halter
|
73057d4176
|
removed the INDENT usages that were left in the parser code
|
2014-02-21 15:39:14 +01:00 |
|
Dave Halter
|
fe02088dd7
|
restructure user_context reverse tokenizer to remove INDENT tokens in the future
|
2014-02-21 15:30:46 +01:00 |
|
Dave Halter
|
9c3b7b9c46
|
get rid of more variables
|
2014-02-21 10:44:33 +01:00 |
|
Dave Halter
|
184e90a3a0
|
removed some line checks from tokenizer
|
2014-02-21 00:13:23 +01:00 |
|
Dave Halter
|
a69eee5876
|
error tokens only exist in the end in a tokenizer
|
2014-02-21 00:09:49 +01:00 |
|
Dave Halter
|
3232ae5b0c
|
removed parentheses counting from generate_tokens
|
2014-02-20 18:45:22 +01:00 |
|
Dave Halter
|
d7033726fd
|
tokenize removed NL/NEWLINE ambiguity
|
2014-02-20 01:52:30 +01:00 |
|
Dave Halter
|
c26ae3c00d
|
ignore dedents in tokenizer
|
2014-02-20 01:21:20 +01:00 |
|
Dave Halter
|
3e9b72b636
|
created a PushBackTokenizer specifically for the parser
|
2014-02-20 01:17:19 +01:00 |
|
Dave Halter
|
c8d6fbb0a1
|
temporary changes for the tokenizer issues
|
2014-02-20 00:43:42 +01:00 |
|
Dave Halter
|
1b6df4602d
|
replace parser tokenizer offset with line offset
|
2014-02-17 23:02:04 +01:00 |
|
Dave Halter
|
6952596117
|
implement an offset in generate_tokens
|
2014-02-17 10:08:32 +01:00 |
|
Dave Halter
|
bb111daf91
|
removed line from tokenizer
|
2014-02-16 15:28:18 +01:00 |
|
Dave Halter
|
22928dbcd0
|
simplified some crazy iterator stuff within parser
|
2014-02-16 15:04:57 +01:00 |
|
Dave Halter
|
82f27569b2
|
little changes to clean up the code (flake8)
|
2014-02-14 12:53:22 +01:00 |
|
Dave Halter
|
d704743422
|
more consistent __slots__ usage in the parser -> 20% memory decrease for wx
|
2014-02-14 00:56:42 +01:00 |
|
Dave Halter
|
bb4d77f2a8
|
minor changes to the NamePart representation
|
2014-02-13 19:53:41 +01:00 |
|
Dave Halter
|
600371632f
|
NamePart is now no str subclass anymore. They are separated, which makes us save a lot of dicts
|
2014-02-13 19:22:36 +01:00 |
|
Dave Halter
|
040ea2b735
|
fix a few annoyances to be quicker to develop now (disabled a few tests for now)
|
2014-02-13 15:40:51 +01:00 |
|
Dave Halter
|
2f3e4152b4
|
Merge branch 'get_code_fidelity' of git://github.com/ganwell/jedi into ganwell
|
2014-02-12 11:09:08 +01:00 |
|
Jean-Louis Fuchs
|
34e89fa1c5
|
Merge branch 'dev' into get_code_fidelity
Conflicts:
jedi/evaluate/docstrings.py
jedi/parser/representation.py
Resolving merge problems:
* Introducing docstring for compiled.fake
* Partly fixing poor decision in TokenDocstring __init__
|
2014-01-28 00:37:06 +01:00 |
|
Dave Halter
|
8193f0c2b6
|
fix an issue with invalid syntax
|
2014-01-26 20:22:51 +01:00 |
|