Closed
Description
The name really get me confused because i was doing something like that
(just a little context here, in this piece of code , I use a custom lexer which pop token::EOF also for a custom token ( %> for a <% xxxx %> templating system ), EOF was chosen because
- i needed to have a value in the token::Token enum to be able to leverage the other parser functions
- it needed to be something that would not be parsed as part of an expression
- something for which i will still have a way to make the difference between the actual token and my custom one (for EOF i still have parser.reader.is_eof() for me )
match (
parser.parse_ident(),
parser.parse_fn_decl(true),
parser.bump_and_get()
) {
(
functioname,
ref function_decl,
token::EOF,
) if parser.token == token::EOF && !parser.reader.is_eof() => {
and it was not working because parser.token == token::EOF was actually testing the NEXT token , because bump_and_get was actually first returning the last token (the one used in the match) and AFTER moving the parser by one more token, (the one stored in parser.token)
anyway I've already been to fix my problem, so the problem is more about naming
my proposition
- replace
bump_and_getbyget_and_bump - maybe permit one more value in the Token enum a token::USER_DEFINED(uint), so in order to help people writing syntax extension that superset rust, to still be able to use the normal parser, and just have to implement their own custom lexer::Reader trait.