9 Commits

Author	SHA1	Message	Date
Dave Halter	d1d019b3c0	Some small grammar changes for 2.7.	2017-07-13 08:44:25 +02:00
Dave Halter	e731eecdd8	Make the tokenizer version independent.	2017-07-11 23:29:44 +02:00
Dave Halter	c28ec5f182	Move tokenize.py and token.py to the python folder.	2017-07-09 00:06:14 +02:00
Dave Halter	976c5151af	Rename source_tokens to tokenize and generate_tokens to tokenize_lines.	2017-05-26 12:01:35 -04:00
Dave Halter	0d93a2a2df	Simplify pgen a bit.	2017-05-26 11:46:22 -04:00
Dave Halter	8948c9053b	use_exact_op_types -> exact_op_types.	2017-05-26 11:39:29 -04:00
Dave Halter	b4259b2b4f	Change the default of use_exact_op_types in the tokenizer.	2017-05-26 11:32:00 -04:00
Dave Halter	3357613ac2	Make some methods private when creating grammars.	2017-05-19 16:39:14 -04:00
Dave Halter	150fb4c86e	Add an initial version of the code from Jedi. ... Still includes imports that have to be removed.	2017-05-11 07:13:23 -04:00