start uniting tokenize.TokenInfo and token.Token

This commit is contained in:
Dave Halter
2014-02-25 13:54:18 +01:00
parent 5b84f0b27f
commit f4f79317fe
4 changed files with 15 additions and 14 deletions

View File

@@ -36,8 +36,9 @@ tok_name[ENCODING] = 'ENCODING'
class TokenInfo(object):
"""The token object is an efficient representation of the structure
(token_type, token, (start_pos_line, start_pos_col)). It has indexer
"""
The token object is an efficient representation of the structure
(type, token, (start_pos_line, start_pos_col)). It has indexer
methods that maintain compatibility to existing code that expects the above
structure.