| TOKENS | LEXEMES | NOTES | TOKENS | LEXEMES | NOTES |
|---|---|---|---|---|---|
| IFTOK | if | COMMA | , | ||
| THENTOK | then | SEMICOL | ; | ||
| ELSETOK | else | COLON | : | ||
| PROC | procedure | WHILETOK | while | ||
| FUNC | function | DOTOK | do | ||
| RELOP | = <> < <= >= > | BEGINTOK | begin | ||
| DOTDOT | .. | ENDTOK | end | ||
| ADDOP | + - |
See Note 1 | REALTOK | real | |
| UNARYOP | + - |
See Note 1 | INTTOK | integer | |
| MULOP | * / div mod | BOOLTOK | boolean | ||
| OROP | or | ARRAYTOK | array | ||
| ANDOP | and | OFTOK | of | ||
| NOTOP | not | VARTOK | var | ||
| ASSIGNOP | := | PROGRAM | program | ||
| LPAR | ( | BCONST | true false | ||
| RPAR | ) | ID | See Note 2 | ||
| LBRK | [ | NUM | See Note 3 | ||
| RBRK | ] | DOT | . |
Note 1: A plus-sign or a minus-sign is a lexeme for an ADDOP token if it immediately follows an ID, NUM, RPAR, or RBRK token, otherwise it is a lexeme for a UNARYOP token.
Note 2: The regular expression for an ID token is: letter (letter|digit)*
Note 3: The regular expression for a NUM token is: digits (. digits)? (E (+|-)? digits)?