Dave Halter b2e54ca1eb The tokenizer now includes all newlines and comments in its prefix.

2014-12-17 20:11:42 +01:00

..

pgen2

Now able to remove both tokenize and token from pgen2.

2014-12-16 02:00:33 +01:00

__init__.py

Trying to replace the old pgen2 token module with a token module more tightly coupled to the standard library.

2014-12-16 01:52:15 +01:00

fast.py

Replace the tokenizer's output with a tuple (switching back from a Token class).

2014-12-16 00:10:07 +01:00

grammar3.4.txt

Fixed issues with the Python3.4 grammar file.

2014-10-17 01:34:47 +02:00

grammar.txt

Fixed issues with the Python3.4 grammar file.

2014-10-17 01:34:47 +02:00

token.py

Python 2 compatibility with the new tokens.

2014-12-16 02:03:05 +01:00

tokenize.py

The tokenizer now includes all newlines and comments in its prefix.

2014-12-17 20:11:42 +01:00

tree.py

parser tree docstring updates.

2014-12-16 18:10:28 +01:00

user_context.py

The tokenizer now includes all newlines and comments in its prefix.

2014-12-17 20:11:42 +01:00