542 Commits

Author	SHA1	Message	Date
Dave Halter	30efdc5e4e	Because we replaced and simplified strings in the last commits (including string ERRORTOKENs), we are now able to remove an error recovery in the backwards tokenizer.	2015-02-25 13:34:12 +01:00
Dave Halter	8c08a4e574	Call signatures again: function definitions and other things that cannot be a part of call signatures stop the process of scanning for them. Also strings get replaced and simplified.	2015-02-25 13:33:09 +01:00
Dave Halter	48392a7dac	Fix some issues in call signatures.	2015-02-24 16:55:33 +01:00
Dave Halter	b8386d29d5	Whitespace before brackets should still show call signatures.	2015-02-24 01:48:25 +01:00
Dave Halter	4f2d4992da	Fix an mro resolution issue.	2015-02-23 19:04:35 +01:00
Dave Halter	a91e240c8b	ALWAYS_BREAK_TOKEN -> ALWAYS_BREAK_TOKENS	2015-02-23 14:10:29 +01:00
Dave Halter	aebeafccc4	Rewrite last newlines in the fast parser to get correct get_code outputs even with the fast parser.	2015-02-23 13:36:43 +01:00
Dave Halter	489ea8fc83	Replace set_parser with direct ParserNode instance calls.	2015-02-23 13:10:40 +01:00
Dave Halter	2fcb1b9b65	Fast parser fix.	2015-02-23 01:00:17 +01:00
Dave Halter	3a5b2d396e	Failed statements should not lead to parser fails.	2015-02-22 20:29:22 +01:00
Dave Halter	3ec96b25cc	Issue with backslashes again in the fast parser.	2015-02-21 18:07:21 +01:00
Dave Halter	0b5a509e83	Small correction: mixed up a re.match and re.search.	2015-02-20 00:48:05 +01:00
Dave Halter	ce96af5e04	Fix an issue with open parentheses and function definitions right after. The fast parser should behave like the normal one and just ignore the open brackets.	2015-02-19 11:02:11 +01:00
Dave Halter	0e73bf7d80	Account for code parts that were not parsed in the fast parser.	2015-02-19 01:42:13 +01:00
Dave Halter	39bf9f426b	Handle backslash escaping.	2015-02-18 17:32:34 +01:00
Dave Halter	595da50ab8	The fast parser splitting now also checks for parentheses levels, because without that, sometimes we split in very strange positions, while ignoring others.	2015-02-18 13:49:03 +01:00
Dave Halter	506d602795	Fix multi line param issues in the fast parser.	2015-02-17 15:24:49 +01:00
Dave Halter	7663703989	Fix issues with multi line for loops in the fast parser.	2015-02-17 14:57:00 +01:00
Dave Halter	4d9608ea6f	Check more precisely for flow keywords.	2015-02-16 16:04:48 +01:00
Dave Halter	e1c28d2c3f	variables starting with 'class' and 'def' should not slow down the parser, changed the check to 'class ' and 'def '.	2015-02-16 10:08:22 +01:00
Dave Halter	db31e0e37d	The fast parser works now faster in case of for flows with a simple_stmt after.	2015-02-14 18:57:04 +01:00
Dave Halter	4613a810a5	Some small refactorings to the names_dict/deep_ast_copy logic.	2015-02-12 13:24:08 +01:00
Dave Halter	774b3d5ce8	Python 2 compatibility.	2015-02-12 11:36:36 +01:00
Dave Halter	a8d3a9ab42	Remove old deep_ast_copy code.	2015-02-12 11:24:17 +01:00
Dave Halter	6e5ba3de87	Fix remaining issue siwh the Param refactoring.	2015-02-11 01:40:18 +01:00
Dave Halter	cdbe26786a	Trying to get ird of the weird param generation in the parser tree.	2015-02-10 15:49:26 +01:00
Dave Halter	a095f8d9e0	Replace some isinstance checks in the parser tree with .type checks.	2015-02-09 12:27:29 +01:00
Dave Halter	a9a3387cb0	Refactor user scope search.	2015-02-05 21:52:57 +01:00
Dave Halter	8125d5f562	Remove asserts and calculate them dynamically.	2015-02-05 20:16:55 +01:00
Dave Halter	2dd08594fc	Simplify the indent calculation in the fast parser.	2015-02-05 14:37:24 +01:00
Dave Halter	d0f1fd5267	Rename Simple -> BaseNode.	2015-02-05 14:18:30 +01:00
Dave Halter	0c1bbf78e2	Rename SubModule to Module, because that's a more fitting description. There were reasons for the name before the new fast parser, but those don't exist anymore.	2015-02-05 14:16:43 +01:00
Dave Halter	c689573b0b	Removed the line_offset from tokenize, we have better ways to modify positions, now.	2015-02-05 14:00:58 +01:00
Dave Halter	59cf1bce5d	Delete legacy code from the fast parser.	2015-02-05 13:47:35 +01:00
Dave Halter	4ace58e29e	Make get_statement_for_position faster.	2015-02-05 13:35:43 +01:00
Dave Halter	a77ecdbed6	Remove param from get_statement_for_position.	2015-02-05 12:28:55 +01:00
Dave Halter	109fdc53e0	Fix the remaining fast parser issues.	2015-02-05 01:13:00 +01:00
Dave Halter	fdfe17ada5	Import the token IDs directly, this way we minimize lookups.	2015-02-05 00:44:01 +01:00
Dave Halter	3a4235eb33	The interpreter is not using the fast parser anymore.	2015-02-05 00:28:54 +01:00
Dave Halter	dce952aec6	Fix an issue with omited dedents in the parser.	2015-02-05 00:11:12 +01:00
Dave Halter	e1c623d3f3	Python 2 compatibility.	2015-02-04 17:09:18 +01:00
Dave Halter	e23e354fe8	Simplified the line splitting and with that a few other things in the fast parser.	2015-02-03 22:22:57 +01:00
Dave Halter	66dfa59286	Fix some endmarker prefix issues in the fast parser.	2015-02-03 22:09:55 +01:00
Dave Halter	6cdfecb541	Fix a number of issues in the fast parser around functions with only one statement (no suite) and wrong indentations).	2015-02-02 15:03:57 +01:00
Dave Halter	f9fe6b47eb	Fix error statement stacks positions.	2015-02-02 10:43:47 +01:00
Dave Halter	a4bd412801	Fix an issue with the positions of InstanceNames that used the original position_modifier.	2015-02-02 02:29:39 +01:00
Dave Halter	c58cdbbf9b	Fix an issue that comes from a combination of property/__slots__/pickle	2015-02-02 00:45:17 +01:00
Dave Halter	e913872192	Merged the tokenize is_identifier changes.	2015-02-01 20:32:01 +01:00
Dave Halter	9a0f1363e3	Start removing the print statements that were used for debugging.	2015-02-01 02:32:52 +01:00
Dave Halter	bc118e8047	Simplify the fast parser tokenizer more. Now it is more readable and less buggy (+bugfixes).	2015-01-31 20:09:44 +01:00