What are tokens?

Asked 07-Dec-2019
Viewed 734 times

1 Answer


Tokens are the smallest individual elements of a program. Everything that one sees inside a program is called token. There are basically five types of tokens:
Keywords: Reserved word which is already defined by the programming language for a specific purpose. Examples: int, for
Identifiers: The names given to various programming elements are known as identifiers.
Operators: The symbols used to perform an arithmetic or logical operation. Examples: +, =
• Separators: These are used to separate different programming elements. Examples: \t, \n
Literals: The various constants used throughout the program. Example: “Hello”, 12

What are tokens?
Tokens at a glance