Dave Halter c7862925f5 Small tokenizer changes & tokens now have a prefix attribute instead of preceeding_whitespace.

2014-11-27 01:10:45 +01:00

..

pgen2

Change the pgen2 parser and its driver so that it can be accessed easily from the outside. This is a minor change and will allow Jedis tokenizer to work with pgen2.

2014-11-26 15:38:53 +01:00

__init__.py

Fix a few tokenize tests and merge them back together.

2014-11-26 16:09:28 +01:00

fast.py

Added a grammar param to the parser.

2014-11-24 01:10:39 +01:00

grammar3.4.txt

Fixed issues with the Python3.4 grammar file.

2014-10-17 01:34:47 +02:00

grammar.txt

Fixed issues with the Python3.4 grammar file.

2014-10-17 01:34:47 +02:00

tokenize.py

Small tokenizer changes & tokens now have a prefix attribute instead of preceeding_whitespace.

2014-11-27 01:10:45 +01:00

tree.py

First implementation to be ready to complete corrupt imports. Working ok.

2014-11-25 19:35:27 +01:00

user_context.py

Small tokenizer changes & tokens now have a prefix attribute instead of preceeding_whitespace.

2014-11-27 01:10:45 +01:00