What are tokens?

0 votes
104 views
Shikhar Arora asked 07-Dec-2019 in Technology by Shikhar Arora

What are tokens

1 Answer

0 votes
Sangsaptak Pal answered 17-Jan-2020 by Sangsaptak Pal
Tokens are the smallest individual elements of a program. Everything that one sees inside a program is called token. There are basically five types of tokens:
Keywords: Reserved word which is already defined by the programming language for a specific purpose. Examples: int, for
Identifiers: The names given to various programming elements are known as identifiers.
Operators: The symbols used to perform an arithmetic or logical operation. Examples: +, =
• Separators: These are used to separate different programming elements. Examples: \t, \n
Literals: The various constants used throughout the program. Example: “Hello”, 12

What are tokens?
Tokens at a glance