606 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Dave Halter	71003bc20e	Just use caching instead of strange transitions	2018-06-24 17:29:03 +02:00
Dave Halter	c5d141bf60	Make some more things faster	2018-06-24 16:43:28 +02:00
Dave Halter	e958b241c7	Use some tokenize names directly	2018-06-24 16:39:48 +02:00
Dave Halter	34ab35558f	Remove a lot of the old token code	2018-06-24 16:31:58 +02:00
Dave Halter	03de9cebb8	Introduce TokenTypes	2018-06-24 16:24:09 +02:00
Dave Halter	6098d89150	Add PythonTokens to get rid of a lot of the token module eventually	2018-06-24 13:03:07 +02:00
Dave Halter	ff4358cd97	Merge remote-tracking branch 'origin/master' into pgen	2018-06-24 11:49:34 +02:00
Dave Halter	b5378e4602	Use token.OP and use reserved words ... This change breaks the tokenizer backwards compatibility a bit. Details of operators is now part of the parser and not the tokenizer anymore. The parser does this anyway, so we don't need the complexity in the tokenizer.	2018-06-24 11:28:23 +02:00
Dave Halter	33e321a539	Don't set the root node before it's not actually defined	2018-06-22 13:04:00 +02:00
Dave Halter	a890ddd6cc	Remove make_first	2018-06-22 12:54:27 +02:00
Dave Halter	1362d4f05d	Remove more unused grammar stuff	2018-06-22 12:53:38 +02:00
Dave Halter	532aef2342	Remove nonterminal2number and number2nonterminal, they are no longer used	2018-06-22 12:52:44 +02:00
Dave Halter	878b4b2d3b	Use nonterminals instead of numbers if possible	2018-06-22 12:47:02 +02:00
Dave Halter	87299335c4	Remove more unused code	2018-06-22 12:36:48 +02:00
Dave Halter	4f0e9c0fd7	Remove old dfas and states from the parser generator	2018-06-22 11:47:18 +02:00
Dave Halter	67ca091631	delete a lot of the old parser code	2018-06-22 11:44:42 +02:00
Dave Halter	4e5ba02dbb	Fix the final issues of the new parser	2018-06-22 11:38:34 +02:00
Dave Halter	a85f544901	Fix all tests except diff tests. Mostly error recovery fixes	2018-06-22 11:12:10 +02:00
Dave Halter	9e8066c6fd	Fix a lot of the old error recovery	2018-06-22 09:56:58 +02:00
Dave Halter	68eab72229	Some slight changes to error recovery	2018-06-22 01:59:39 +02:00
Dave Halter	d9264609f2	Get quite a bit of the error recovery working	2018-06-22 01:56:29 +02:00
Dave Halter	79c7e0b59d	Use the new parser. Error recovery is not yet working	2018-06-22 00:38:18 +02:00
Dave Halter	f03a87b876	Actually parse some first things with the new approach	2018-06-21 23:56:34 +02:00
Dave Halter	2a082d69df	Add better reprs	2018-06-21 22:13:27 +02:00
Dave Halter	e6fc739670	Get most things ready for plans	2018-06-21 21:32:05 +02:00
Dave Halter	12e11b3d16	Remove a while loop that is not necessary	2018-06-21 21:14:14 +02:00
Dave Halter	cc8038966b	isfinal -> is_final	2018-06-21 21:11:46 +02:00
Dave Halter	31aecf2d35	Calculate the first plans in a very messy way	2018-06-21 21:10:28 +02:00
Dave Halter	d8554d86d1	A lot of new code to hopefully transition to a better parsing mechanism in the future	2018-06-21 18:17:32 +02:00
Dave Halter	d691bf0fd1	Some minor changes	2018-06-20 09:42:21 +02:00
Dave Halter	5712ffb5ca	Introduce a label cache that is currently not used	2018-06-18 20:14:29 +02:00
Dave Halter	55d6a69aad	Some more renames	2018-06-18 01:14:09 +02:00
Dave Halter	453471eeb6	Move some ParserGenerator stuff into the Grammar class	2018-06-18 00:15:31 +02:00
Dave Halter	a06c3a3129	name -> nonterminal	2018-06-17 23:10:27 +02:00
Dave Halter	73ce57428b	Try to completely remove the word symbol and use nonterminal ... The ones that we could not remove are in grammar.py, because that's the public documented API.	2018-06-17 18:30:20 +02:00
Dave Halter	640f544af9	One instance of symbol -> terminal	2018-06-17 18:12:13 +02:00
Dave Halter	b6cbf306d7	Use the name first_terminals instead of first	2018-06-17 18:09:12 +02:00
Dave Halter	95e4ecf592	next -> next_	2018-06-17 17:49:43 +02:00
Dave Halter	fbed1ecfe0	More dict to set	2018-06-17 17:45:40 +02:00
Dave Halter	1f27fa9320	Use more nonterminal/terminal terminology	2018-06-17 17:37:15 +02:00
Dave Halter	23362ec2d3	Start using the term nonterminal	2018-06-17 16:40:11 +02:00
Dave Halter	6b391af071	Use sets instead of dicts if possible	2018-06-17 16:36:27 +02:00
Dave Halter	c43cb21a0e	Add docstring	2018-06-15 17:57:59 +02:00
Dave Halter	24346a0d32	Add some comments	2018-06-15 10:21:00 +02:00
Dave Halter	9d452ec66a	Use a set instead of dict if it's not necessary	2018-06-15 00:35:39 +02:00
Dave Halter	567e0d7aed	Simplify some code	2018-06-15 00:06:56 +02:00
Dave Halter	1f02327cff	Some refactorings for simplicity	2018-06-14 18:18:57 +02:00
Dave Halter	8c348aee6f	Use NFAArc as a class	2018-06-14 10:29:19 +02:00
Dave Halter	a277ccf288	Some more renames	2018-06-14 01:10:23 +02:00
Dave Halter	a5ce2caab6	Another rename	2018-06-13 23:19:52 +02:00