the end of its input stream by setting the lookahead character to nil. At
this point it is still possible to push back characters."))
-(export '(token-type token-value))
-(defgeneric token-type (lexer)
- (:documentation
- "Return the type of the LEXER's current lookahead token
-
- When the lexer is first created, there is no lookahead token: you must
- `prime the pump' by calling NEXT-TOKEN."))
-(defgeneric token-value (lexer)
- (:documentation
- "Return the value of the LEXER's current lookahead token
-
- When the lexer is first created, there is no lookahead token: you must
- `prime the pump' by calling NEXT-TOKEN."))
-
;;;--------------------------------------------------------------------------
;;; Formatting tokens.