Dave Halter
2e65fbb00f
rename parser.tokenizer -> parser.tokenize
2014-01-05 13:38:14 +01:00
Dave Halter
261f49d3e2
move NoErrorTokenizer to the tokenizer module, where it more or less belongs.
2014-01-05 13:34:29 +01:00
Dave Halter
e115689b7f
move NoErrorTokenizer to the parser, where it more or less belongs.
2014-01-05 13:25:11 +01:00
Dave Halter
7bf6ff768c
first steps to world domination -> removal of the hackish sys.path modifications
2013-12-24 16:23:13 +01:00
Jean-Louis Fuchs
3204a39f6c
* made Token readonly like a tuple by using @property
...
* end_pos calculation didn't respect multiline tokens
* replaced all index access to Token
* wrapped all code that injects token tuples with
Token.from_tuple()
* repr of Token is still its tuple form!?
* PEP8 where I read or wrote code
2013-12-08 01:32:58 +01:00
Jean-Louis Fuchs
989e12e8a7
* Replaced token tuple with token class
...
* Fixed PEP8 where I read code
2013-12-05 23:34:20 +01:00
David Halter
390442dc3b
move all the parser stuff into a seperate package
2013-09-06 00:58:57 +04:30
