Commodore‎ > ‎BASIC‎ > ‎

Tokens

Most of the keywords used by Commodore BASIC are "compressed" into (typically) a 1-byte code known as a token (rarely a 2-byte code).  This has several advantages: longer programs can be stored in memory, scanning through a program line is faster, program files are smaller, loading/saving the program is faster, and executing the program is faster.  Although Commodore BASIC is said to be an interpreter, the conversion into tokens is essentially a form of compilation.  So when the program runs, BASIC does not need to parse the text of "PRINT" for example, it instead just does a quick look-up based on the token.  (Off-topic, the main reason it is called an interpreter is because the variables and line numbers must be searched when the program runs; a compiler would hard-code such references.)
 
Below is a table listing all keywords ordered by their token value (if it has one).  (Here you can find a table ordered by name.) This table also includes the version number(s) to which it applies, and a classification.
Token (hex) Keyword Version(s) Classification
44 53** DS 3.5+ Reserved variable
44 53 24** DS$ 3.5+ Reserved variable
45 4C** EL 3.5+ Reserved variable
45 52** ER 3.5+ Reserved variable
53 54** ST 1.0+ Reserved variable
54 49** TI 1.0+ Reserved variable
54 49 24** TI$ 1.0+ Reserved variable
80 END 1.0+ Command* and Statement
81 FOR 1.0+ Command and Statement
82 NEXT 1.0+ Command, Preposition and Statement
83 DATA 1.0+ Command* and Statement
84 INPUT# 1.0+ Statement
85 INPUT 1.0+ Statement
86 DIM 1.0+ Command and Statement
87 READ 1.0+ Command and Statement
88 LET 1.0+ Command and Statement
89 GOTO 1.0+ Command and Statement
8A RUN 1.0+ Command and Statement
8B IF 1.0+ Command and Statement
8C RESTORE 1.0+ Command and Statement
8D GOSUB 1.0+ Command and Statement
8E RETURN 1.0+ Statement
8F REM 1.0+ Command and Statement
90 STOP 1.0+ Command* and Statement
91 ON 1.0+ Preposition
92 WAIT 1.0+ Command and Statement
93 LOAD 1.0+ Command* and Statement
94 SAVE 1.0+ Command and Statement
95 VERIFY 1.0+ Command* and Statement
96 DEF 1.0+ Statement
97 POKE 1.0+ Command and Statement
98 PRINT# 1.0+ Command and Statement
99 PRINT 1.0+ Command and Statement
9A CONT 1.0+ Command* and Statement
9B LIST 1.0+ Command and Statement*
9C CLR 1.0+ Command and Statement
9D CMD 1.0+ Command and Statement
9E SYS 1.0+ Command and Statement
9F OPEN 1.0+ Command and Statement
A0 CLOSE 1.0+ Command and Statement
A1 GET 1.0+ Statement
A1 23 GET# 1.0+ Statement
A1 F9 GETKEY 3.5, 7.0  Statement 
A2 NEW 1.0+ Command and Statement*
A3 TAB( 1.0+ Preposition
A4 TO 1.0+ Preposition
A5 FN 1.0+ Function/Preposition
A6 SPC( 1.0+ Preposition
A7 THEN 1.0+ Preposition
A8 NOT 1.0+ Operator (5)
A9 STEP 1.0+ Preposition
AA + 1.0+ Operator (3)
AB - 1.0+ Operator (3)
AC * 1.0+ Operator (2)
AD / 1.0+ Operator (2)
AE ^ 1.0+ Operator (1)
AF AND 1.0+ Operator (6)
B0 OR 1.0+ Operator (7)
B1 < 1.0+ Operator (4)
B2 = 1.0+ Operator (4)
B3 > 1.0+ Operator (4)
B4 SGN 1.0+ Function
B5 INT 1.0+ Function
B6 ABS 1.0+ Function
B7 USR 1.0+ Function
B8 FRE 1.0+ Function
B9 POS 1.0+ Function
BA SQR 1.0+ Function
BB RND 1.0+ Function
BC LOG 1.0+ Function
BD EXP 1.0+ Function
BE COS 1.0+ Function
BF SIN 1.0+ Function
C0 TAN 1.0+ Function
C1 ATN 1.0+ Function
C2 PEEK 1.0+ Function
C3 LEN 1.0+ Function
C4 STR$ 1.0+ Function
C5 VAL 1.0+ Function
C6 ASC 1.0+ Function
C7 CHR$ 1.0+ Function
C8 LEFT$ 1.0+ Function
C9 RIGHT$ 1.0+ Function
CA MID$ 1.0+ Function
CB GO 2.0+ Command and Statement
CC RGR 3.5,7.0 Function
CC CONCAT 4.x Command and Statement
CD RCLR 3.5,7.0 Function
CD DOPEN 4.x Command and Statement
CE RLUM 3.5 Function
CE DCLOSE 4.x Command and Statement
CE 02 POT 7.0 Function
CE 03 BUMP 7.0 Function
CE 04 PEN 7.0 Function
CE 05 RSPPOS 7.0 Function
CE 06 RSPRITE 7.0 Function
CE 07 RSPCOLOR 7.0 Function
CE 08 XOR 7.0 Function
CE 09 RWINDOW 7.0 Function
CE 0A POINTER 7.0 Function
CF JOY 3.5,7.0 Function
CF RECORD 4.x Command and Statement
D0 RDOT 3.5,7.0 Function
D0 HEADER 4.x Command and Statement
D1 DEC 3.5,7.0 Function
D1 COLLECT 4.x Command and Statement
D2 HEX$ 3.5,7.0 Function
D2 BACKUP 4.x Command and Statement
D3 ERR$ 3.5,7.0 Function
D3 COPY 4.x Command and Statement
D4 INSTR 3.5,7.0 Function
D4 APPEND 4.x Command and Statement
D5 ELSE 3.5,7.0 Command*, Preposition, and Statement*
D5 DSAVE 4.x Command and Statement
D6 RESUME 3.5,7.0 Statement
D6 DLOAD 4.x Command* and Statement
D7 TRAP 3.5,7.0 Statement
D7 CATALOG 4.x Command and Statement
D8 TRON 3.5,7.0 Command and Statement
D8 RENAME 4.x Command and Statement
D9 TROFF 3.5,7.0 Command and Statement
D9 SCRATCH 4.x Command and Statement
DA SOUND 3.5,7.0 Command and Statement
DA DIRECTORY 4.x Command and Statement
DB VOL 3.5,7.0 Command and Statement
DB DCLEAR 4.7 Command and Statement
DC AUTO 3.5,7.0 Command and Statement
DC BANK 4.7 Command and Statement
DD PUDEF 3.5,7.0 Command and Statement
DD BLOAD 4.7 Command and Statement
DE GRAPHIC 3.5,7.0 Command and Statement
DE BSAVE 4.7 Command and Statement
DF PAINT 3.5,7.0 Command and Statement
DF KEY 4.7 Command and Statement
E0 CHAR 3.5,7.0 Command and Statement
E0 DELETE 4.7 Command
E1 BOX 3.5,7.0 Command and Statement
E1 ELSE 4.7 Preposition
E2 CIRCLE 3.5,7.0 Command and Statement
E2 TRAP 4.7 Statement
E3 GSHAPE 3.5,7.0 Command and Statement
E3 RESUME 4.7 Statement
E4 SSHAPE 3.5,7.0 Command and Statement
E4 DISPOSE 4.7 Command and Statement
E5 DRAW 3.5,7.0 Command and Statement
E5 PUDEF 4.7 Command and Statement
E6 LOCATE 3.5,7.0 Command and Statement
E6 USING 4.7 Preposition
E7 COLOR 3.5,7.0 Command and Statement
E7 ERR$ 4.7 Function
E8 SCNCLR 3.5,7.0 Command and Statement
E8 INSTR 4.7 Function
E9 SCALE 3.5,7.0 Command and Statement
EA HELP 3.5,7.0 Command and Statement*
EB DO 3.5,7.0 Command and Statement
EC LOOP 3.5,7.0 Command and Statement
ED EXIT 3.5,7.0 Command and Statement
EE DIRECTORY 3.5,7.0 Command and Statement
EF DSAVE 3.5,7.0 Command and Statement
F0 DLOAD 3.5,7.0 Command and Statement*
F1 HEADER 3.5,7.0 Command and Statement
F2 SCRATCH 3.5,7.0 Command and Statement
F3 COLLECT 3.5,7.0 Command and Statement
F4 COPY 3.5,7.0 Command and Statement
F5 RENAME 3.5,7.0 Command and Statement
F6 BACKUP 3.5,7.0 Command and Statement
F7 DELETE 3.5,7.0 Command and Statement
F8 RENUMBER 3.5,7.0 Command
F9 KEY 3.5,7.0 Command, Preposition, and Statement
FA MONITOR 3.5,7.0 Command and Statement*
FB USING 3.5,7.0 Preposition
FC UNTIL 3.5,7.0 Preposition
FD WHILE 3.5,7.0 Preposition
FE 02 BANK 7.0 Command and Statement
FE 03 FILTER 7.0 Command and Statement
FE 04 PLAY 7.0 Command and Statement
FE 05 TEMPO 7.0 Command and Statement
FE 06 MOVSPR 7.0 Command and Statement
FE 07 SPRITE 7.0 Command and Statement
FE 08 SPRCOLOR 7.0 Command and Statement
FE 09 RREG 7.0 Command and Statement
FE 0A ENVELOPE 7.0 Command and Statement
FE 0B SLEEP 7.0 Command and Statement
FE 0C CATALOG 7.0 Command and Statement
FE 0D DOPEN 7.0 Command and Statement
FE 0E APPEND 7.0 Command and Statement
FE 0F DCLOSE 7.0 Command and Statement
FE 10 BSAVE 7.0 Command and Statement
FE 11 BLOAD 7.0 Command and Statement
FE 12 RECORD 7.0 Command and Statement
FE 13 CONCAT 7.0 Command and Statement
FE 14 DVERIFY 7.0 Command and Statement
FE 15 DCLEAR 7.0 Command and Statement
FE 16 SPRSAV 7.0 Command and Statement
FE 17 COLLISION 7.0 Command and Statement
FE 18 BEGIN 7.0 Preposition
FE 19 BEND 7.0 Command* and Statement*
FE 1A WINDOW 7.0 Command and Statement
FE 1B BOOT 7.0 Command and Statement
FE 1C WIDTH 7.0 Command and Statement
FE 1D SPRDEF 7.0 Command and Statement
FE 1E QUIT 7.0 Command and Statement
FE 1F STASH 7.0 Command and Statement
FE 21 FETCH 7.0 Command and Statement
FE 23 SWAP 7.0 Command and Statement
FE 24 OFF 7.0 Preposition
FE 25 FAST 7.0 Command and Statement
FE 26 SLOW 7.0 Command and Statement
FF π (pi) 1.0+ Reserved variable
Wow, you read all that?! A few notes about the the table. The "tokens" marked ** are really just the character code(s) for the letters that make up the name (you should see this only applies to reserved variable names).  Most keywords can be used as commands (in direct mode) or as statements (in a running program), although a few can only be used one way but not the other.  However, there are several that in theory could be used as either commands or statements, but certain uses are uncommon in practice due to no effect or undesirable side-effects; these are marked with an asterisk (*).
 
For example, LIST is classified as "Command or Statement*" (note the * following Statement).  This is because you wouldn't normally include LIST as a statement within a program.  The reason is, when it is used inside a program, it has the (usually undesired) side-effect of ending the program.  Another example would be END; classified as "Command* or Statement" (note * by Command) because it is normally only used as a statement within a program.  Commanding the computer to END in direct mode does not cause any error, but it doesn't do anything useful because (by definition) no program is running in direct mode.
 
Expressions in BASIC are evaluated using a fixed precedence (order of operations) for the different math operators.  For example, LET Y= 1+2*X would evaluate 2 times X before adding 1.  This is because multiplication (* operator) has higher precedence than addition (+ operator).  Each operator listed has its precedence level shown in parentheses.  For example, addition (+) is listed as "Operator (3)" which means it has 3rd (third) highest priority.  Precedence levels range from 1 (highest) to 7 (lowest).  Note that any sub-expressions enclosed in parentheses will be evaluated with an even higher precendence.  One obscure/interesting thing to note is that although most operators are 1-character symbols (thus only need 1 byte to encode), their token values are radically different from their ASCII/PETSCII codes.

© H2Obsession, 2014
Comments