Dave Halter f43c371467 Merge @joel-wright's whitespace tokenizer branch. Thanks!

2014-11-26 15:56:11 +01:00

..

pgen2

Change the pgen2 parser and its driver so that it can be accessed easily from the outside. This is a minor change and will allow Jedis tokenizer to work with pgen2.

2014-11-26 15:38:53 +01:00

__init__.py

First implementation to be ready to complete corrupt imports. Working ok.

2014-11-25 19:35:27 +01:00

fast.py

Added a grammar param to the parser.

2014-11-24 01:10:39 +01:00

grammar3.4.txt

Fixed issues with the Python3.4 grammar file.

2014-10-17 01:34:47 +02:00

grammar.txt

Fixed issues with the Python3.4 grammar file.

2014-10-17 01:34:47 +02:00

tokenize.py

Merge @joel-wright's whitespace tokenizer branch. Thanks!

2014-11-26 15:56:11 +01:00

tree.py

First implementation to be ready to complete corrupt imports. Working ok.

2014-11-25 19:35:27 +01:00

user_context.py

Change the backwards tokenizer that keywords always stop.

2014-11-26 02:32:13 +01:00