Course Content‎ > ‎Session 2‎ > ‎

Tokens

What is a token?

A token is the smallest element of a program that is meaningful to the compiler. In the programs we have seen so far, class, public, static, void were some of the tokens. 

When a Java program is submitted to the Java compiler, the compiler parses the text and extracts individual tokens. These tokens define the structure of the Java language. Java tokens can be broken into following categories:

Identifiers

Identifiers are the tokens that represent the names assigned to variables, methods and classes to uniquely identify them. In the program, FirstProgram is an identifier that assigns the name FirstProgram to the class residing in FirstProgram.java source file.  There are some rules for naming identifiers in java, which are given below:

  • All Java identifiers are case-sensitive.
  • All Java identifiers must begin with a letter. But subsequent characters can also include digits (numbers from 0-9), underscore ( _ ) and dollar sign($).
  • Identifier names in Java cannot include Java keywords. For example, class, boolean, and char are all Java keywords. To see a complete list of keywords in Java, refer to Appendix 1.
Examples of VALID identifier names:
  • FirstProgram
  • counter
  • List12
  • Name_of_employee

Examples of INVALID identifier names: 
  • First Program
  • #age
  • 28Age
  • Count!

Literals

These are the program elements that are used in an invariant manner. In the program FirstProgram.java

    This is my first program in Java

is a literal or more specifically a String literal. Literals can be numbers, characters or strings. Literals are of the following types:

Integer literals: These are the primary literals used in Java programming and come in decimal, hexadecimal and octal formats(described in appendix1). Decimal literals appear as ordinary numbers with no special notation. Octal literals appear as numbers with a leading 0 in front of the digits. Hexadecimal numbers appear with a leading 0x or 0X. For example an integer literal for the decimal number 12 is represented in Java as 12 in decimal notation, 0xC in hexadecimal notation and 014 in octal notation.

Floating point literals: These represent decimal numbers with fractional parts. For example, 23.343,65.432,-1211.34 etc.

Boolean literals: Java provides boolean type with two possible states: true and false. Boolean is used to represent a state or a condition that can be either true or false.

Character literals: Character literals represent a single Unicode character and appear within a pair of single quotation marks. For example: 'a', '1', 'Y' etc. Special characters i.e. control characters and characters that cannot be printed are represented by \(backslash) followed by character code. Given below is a list of special characters supported by java:

 Description Representation
 Backslash \\ 
 Backspace \b
 Carriage Return \r
 Form Feed \f
 Horizontal Tab \t
 Newline \n
 Single Quote \' 
 Double Quote \" 


String literals: String literals represent multiple characters and appear within a pair of double quotation marks. In Java, string literals are implemented by the String class. When Java encounters a string literal, it creates an instance of the String class.

Operators

Operators specify an operation or an evaluation to be performed on data objects or objects. These operands can be variables, literals etc. Some of the operators supported by Java are: + , - , / , * , >=.

Separators

Separators tell the Java compiler how things are grouped in the code. The separators supported by Java are:

 SeparatorDescription
 {}Curly Braces 
 ;Semi colon 
 ,Comma 
 :Colon 

Comments