Dave Halter
|
cd7774f25f
|
lambda can be used as a default param in function, which means there have been slight changes to the parser to allow that (comma in a function definitions doesn't always mean new param), fixes #379
|
2014-03-11 12:24:36 +01:00 |
|
Dave Halter
|
0dcc924cf8
|
precedence working for some simple cases (and invalid statements)
|
2014-03-08 02:24:25 +01:00 |
|
Dave Halter
|
d12e030677
|
use tuples instead of lists if the don't change (immutable is faster and more memory efficient)
|
2014-03-06 00:26:23 +01:00 |
|
Dave Halter
|
a97c91002f
|
replace name and statement end_pos (this way we can get rid of all that end_pos crap soon)
|
2014-03-05 23:46:39 +01:00 |
|
Dave Halter
|
66a488b911
|
clean up parse_dot_name and other small things
|
2014-03-05 22:46:11 +01:00 |
|
Dave Halter
|
1f7e4ca637
|
dict literals are now working (at least the parser)
|
2014-03-05 18:46:43 +01:00 |
|
Dave Halter
|
387319fde5
|
simplify parse_array again (closure in statement parser)
|
2014-03-05 16:54:21 +01:00 |
|
Dave Halter
|
7222d70ecb
|
restructure token breaks in statement parser
|
2014-03-05 14:39:21 +01:00 |
|
Dave Halter
|
7d7a86239c
|
tests for list comprehensions with included dict literals
|
2014-03-05 14:23:53 +01:00 |
|
Dave Halter
|
2322edff8f
|
keyword statement not includes globals
|
2014-03-05 00:44:19 +01:00 |
|
Dave Halter
|
8bd8ba1df8
|
don't need to generate 'pass' anymore if a class is empty in get_code
|
2014-03-04 17:27:26 +01:00 |
|
Dave Halter
|
f54344fd9e
|
KeywordStatements are working except for some of the old ones (global, assert, return, yield)
|
2014-03-04 17:20:29 +01:00 |
|
Dave Halter
|
53fd1f925a
|
create KeywordStatement to fit assert, del, global, etc into a more generalized schema, which can also improve the get_code method
|
2014-03-04 15:35:24 +01:00 |
|
Dave Halter
|
65ce609a3c
|
protect token_list -> _token_list
|
2014-03-04 12:43:37 +01:00 |
|
Dave Halter
|
7de4b14461
|
remove crazy PushBackIterator from statement parser
|
2014-02-27 17:40:01 +01:00 |
|
Dave Halter
|
85b5fdf85f
|
again... statement parser
|
2014-02-27 17:12:16 +01:00 |
|
Dave Halter
|
1eba63760e
|
more change to simplify the statement parser
|
2014-02-27 16:58:08 +01:00 |
|
Dave Halter
|
8688def619
|
more statement parser deletions
|
2014-02-27 11:42:16 +01:00 |
|
Dave Halter
|
38d940cc2b
|
remove start_pos definition from statement parser
|
2014-02-27 11:36:58 +01:00 |
|
Dave Halter
|
2061fbcacc
|
small parse_stmt refactoring
|
2014-02-27 00:43:27 +01:00 |
|
Dave Halter
|
3332eba3f7
|
lambda problem with new operators
|
2014-02-27 00:17:52 +01:00 |
|
Dave Halter
|
fdabca20e9
|
fix get_code method for Operator (still ugly, though)
|
2014-02-26 22:23:21 +01:00 |
|
Dave Halter
|
3330e29748
|
more problems fixed that relate to Operator
|
2014-02-26 22:08:51 +01:00 |
|
Dave Halter
|
2e12eb7861
|
start with the integration of an Operator class to make way for precedences
|
2014-02-26 14:44:51 +01:00 |
|
Dave Halter
|
e152939791
|
remove encoding stuff from tokenizer - encoding is always unicode
|
2014-02-26 12:55:32 +01:00 |
|
Dave Halter
|
1a3cca6edb
|
fixed a problem with the fast parser and its strange end positions
|
2014-02-26 11:23:45 +01:00 |
|
Dave Halter
|
8d1a9f2d46
|
fix long standing issue with import end_pos
|
2014-02-26 11:03:19 +01:00 |
|
Dave Halter
|
acec5fe76f
|
fake docstrs fixed
|
2014-02-26 02:38:28 +01:00 |
|
Dave Halter
|
44e16c11e5
|
fixed docstr problems with unified interfaces
|
2014-02-26 02:33:18 +01:00 |
|
Dave Halter
|
5e5bb618ea
|
remove token.py, docstrings are now just normal tokens until used
|
2014-02-26 01:13:38 +01:00 |
|
Dave Halter
|
40be00826e
|
clean up tokenize
|
2014-02-25 17:17:33 +01:00 |
|
Dave Halter
|
761c28ef00
|
remove __getitem__ from Token
|
2014-02-25 17:03:56 +01:00 |
|
Dave Halter
|
66aca8eba1
|
user_context now doesn't work with the token tuple anymore
|
2014-02-25 16:49:44 +01:00 |
|
Dave Halter
|
18e985a961
|
TokenInfo -> Token
|
2014-02-25 16:44:48 +01:00 |
|
Dave Halter
|
6439d6c848
|
removed the now redundant token.Token version in favor of tokenize.TokenInfo
|
2014-02-25 16:38:19 +01:00 |
|
Dave Halter
|
2db26abf72
|
start and end don't exst anymore in parser.token.Token, it's now start_pos/end_pos as everywhere else
|
2014-02-25 16:34:27 +01:00 |
|
Dave Halter
|
ee7108cc11
|
successfully removed __str__ and __unicode__ methods from token.Token
|
2014-02-25 16:21:53 +01:00 |
|
Dave Halter
|
aea2c4620f
|
more unicode switches in the parser
|
2014-02-25 14:27:50 +01:00 |
|
Dave Halter
|
f4f79317fe
|
start uniting tokenize.TokenInfo and token.Token
|
2014-02-25 13:54:18 +01:00 |
|
Dave Halter
|
5b84f0b27f
|
remove end_pos stuff from tokenizer, the tokens can do that themselves
|
2014-02-25 13:29:27 +01:00 |
|
Dave Halter
|
2252271bf5
|
fix unicode issues with python2.7
|
2014-02-25 12:54:06 +01:00 |
|
Dave Halter
|
3a23c80ae5
|
prepare for eventual? tokenizer end_pos replacement.
|
2014-02-25 11:59:10 +01:00 |
|
Dave Halter
|
246118f851
|
start using @ganwell's new token class (modified in some ways) as the main token class - hope to gain a little bit of memory/cpu/pickling performance
|
2014-02-25 02:06:26 +01:00 |
|
Dave Halter
|
e2a6d1dd43
|
remove offset param for Parser
|
2014-02-25 01:31:24 +01:00 |
|
Dave Halter
|
44d560c53a
|
finally removed self-contained iterator from Parser - after knowing this since almost the beginning.
|
2014-02-25 01:26:19 +01:00 |
|
Dave Halter
|
a7a8a73a2c
|
removed Parser._start_pos as well
|
2014-02-25 01:20:55 +01:00 |
|
Dave Halter
|
add8259d7e
|
successfully removed end_pos from parser
|
2014-02-25 01:05:00 +01:00 |
|
Dave Halter
|
936c7dfde4
|
protect start_pos/end_pos in parser to hopefully remove them soon
|
2014-02-25 00:27:39 +01:00 |
|
Dave Halter
|
ef8d3633dd
|
use the module end_pos in fast parser
|
2014-02-25 00:18:24 +01:00 |
|
Dave Halter
|
a5a6e9ac12
|
parser: remove current
|
2014-02-24 20:43:00 +01:00 |
|