Commit Graph

396 Commits

Author SHA1 Message Date
Dave Halter 3c3e7f5317 Make some parsing work. 2017-08-24 01:39:17 +02:00
Dave Halter 033e880408 Add fstrings to the grammar. 2017-08-24 00:01:42 +02:00
Dave Halter 2ba107872d A first iteration of an fstring tokenizer/grammar. 2017-08-23 23:49:11 +02:00
Dave Halter 929593701a Remove opmap from pgen. 2017-08-22 08:45:10 +02:00
Dave Halter d7f2051f8a Remove some use cases from token.tok_name in pgen. 2017-08-21 20:33:29 +02:00
Dave Halter 6630542847 Make a method in the tokenizer private. 2017-08-21 18:31:10 +02:00
Dave Halter c0d3734e81 Remove exact_type from PythonToken, because the support for this differentiation of exact_type and type was never really there. 2017-08-21 18:30:14 +02:00
Dave Halter 88c3c0567a Rename TokenInfo to PythonToken. 2017-08-21 09:33:41 +02:00
Dave Halter 1912551296 Move some star_expr checking around. 2017-08-19 12:59:49 +02:00
Dave Halter d5d7518ca4 Allow multiple values to be used when registering rules. 2017-08-19 12:51:10 +02:00
Dave Halter 820e94e03a Move the error node checking to a rule. 2017-08-19 12:43:47 +02:00
Dave Halter 5ff33c3736 Refactor the error signatures. 2017-08-19 10:54:29 +02:00
Dave Halter 5916b676de Move the TryStmtRule. 2017-08-18 18:53:11 +02:00
Dave Halter 30acb69caa Move all of the check_assignment stuff into classes. 2017-08-18 18:47:49 +02:00
Dave Halter 84e5adb364 Move the parameter rules. 2017-08-18 18:37:19 +02:00
Dave Halter ceb7121dc6 Move the big arglist rule. 2017-08-18 18:32:24 +02:00
Dave Halter 8ba7001b44 Protect some classes that don't need to be public. 2017-08-18 18:21:18 +02:00
Dave Halter 76578218d9 Move argument and nonlocal rule. 2017-08-18 18:20:37 +02:00
Dave Halter 904f293830 Move annassign and starexprs. 2017-08-18 18:15:24 +02:00
Dave Halter a53477321b Move some import rule stuff around. 2017-08-17 20:49:27 +02:00
Dave Halter e8196c0c06 Move the first node (atom) to rules. 2017-08-17 20:42:32 +02:00
Dave Halter 33ad290512 More rules, yay. 2017-08-17 20:39:34 +02:00
Dave Halter eb2c0ab93b Move string checks to a rule. 2017-08-17 20:25:23 +02:00
Dave Halter 0fd2e169bd More rule moves. 2017-08-17 20:22:32 +02:00
Dave Halter 657baabb0b Move continue checks to a rule. 2017-08-17 20:12:23 +02:00
Dave Halter f055ba198f Move more stuff and fix tests. 2017-08-17 10:38:45 +02:00
Dave Halter 06d7da3f3e Move the first issue to the new rule engine. 2017-08-17 10:25:11 +02:00
Dave Halter 8d20f3d469 Finish refactoring rules. 2017-08-17 09:33:46 +02:00
Dave Halter e37e2e1ff6 Swich the add_sisue signature around. 2017-08-16 22:44:33 +02:00
Dave Halter fc80cebfb8 More normalizer Rule refactoring. 2017-08-16 22:13:11 +02:00
Dave Halter 412da07893 Refactor the Rule class a bit. 2017-08-16 18:17:51 +02:00
Dave Halter 51f2de28c6 source_to_unicode -> python_bytes_to_unicode. 2017-08-15 20:07:24 +02:00
Dave Halter ab027885c7 Refactor splitlines -> split_lines. 2017-08-15 19:54:21 +02:00
Dave Halter 476d5fb0d1 Some Python2.7 fixes. 2017-08-15 19:34:47 +02:00
Dave Halter 86aa185136 Use source_to_unicode by default with errors='strict'. 2017-08-15 19:27:27 +02:00
Dave Halter a305a911f1 Create some examples for fstring fails. 2017-08-14 18:54:28 +02:00
Dave Halter 13cc39c3f0 Add pypy to the tests on travis. 2017-08-06 18:25:40 +02:00
Dave Halter 2322b3aed3 Don't use pickle in pypy. It's really slow. (Armin Rigo told me a while ago). 2017-08-06 18:08:29 +02:00
Dave Halter e2c97e9675 Skip tests in PyPy that don't make sense to test. 2017-08-06 18:04:31 +02:00
Dave Halter c23ae8a69b Add the syntax/indentation error list that can be found in CPython (mostly 3.6). 2017-08-06 17:38:17 +02:00
Dave Halter 3c961e2b70 Rename .params to .get_params(). 2017-08-06 17:29:29 +02:00
Dave Halter 8c3373a0d9 Rename normalizer -> errors. 2017-08-06 17:12:22 +02:00
Dave Halter 860fe2cd1e Refactor the initialization of contexts a bit so a varity of nodes can be taken as input. 2017-08-06 16:44:53 +02:00
Dave Halter 524f332e7d Small refactoring. 2017-08-06 16:11:31 +02:00
Dave Halter 1a30afdd9e Move a few imports out of functions. 2017-08-06 15:34:47 +02:00
Dave Halter 90edb2d0cf Move the iter_errors and normalizer methods to grammar.py. 2017-08-06 15:32:50 +02:00
Dave Halter 3e39a04bb1 Fix opening contexts at the right position. 2017-08-06 14:39:26 +02:00
Dave Halter 31675e74a2 Add a python3.3 grammar, this is just cleaner IMO. 2017-08-06 03:00:14 +02:00
Dave Halter 8382f338ca Fix some small issues around expr_stmt. 2017-08-06 02:52:52 +02:00
Dave Halter f4b51d91ee Add python3.4/3.3 issue 'can use starred expression only as assignment target'. 2017-08-06 01:38:05 +02:00