A token is the smallest individual unit of a C++ program. Every C++ program is made up of different types of tokens that help the compiler understand your code.
For example, keywords, variable names, numbers, operators, and symbols are all considered tokens in C++.
Example
int age = 20;
In the above example:
int→ Keywordage→ Identifier=→ Operator20→ Literal;→ Separator
All of these are tokens because each one has a specific meaning in the program.
Types of Tokens in C++
C++ has the following main types of tokens:
1. Keywords
Keywords are reserved words that have a predefined meaning in C++. They cannot be used as variable or function names.
Examples: int, float, if, while, return
2. Identifiers
Identifiers are the names you give to variables, functions, classes, and other user-defined elements.
Examples: age, marks, calculateSum
3. Literals
Literals are fixed values written directly in a program.
Examples: 10, 3.14, 'A', "Hello"
4. Operators
Operators are special symbols used to perform operations on values.
Examples: +, -, *, /, =, ==
5. Separators
Separators help organize the program and separate different parts of the code.
Examples: ;, {}, (), ,
Why Are Tokens Important?
Understanding tokens makes it easier to read and write C++ programs. They are the basic building blocks of the language, and every line of code you write is made up of one or more tokens.
Key Points
- A token is the smallest unit of a C++ program.
- Every C++ statement is made up of one or more tokens.
- The five main types of tokens are keywords, identifiers, literals, operators, and separators.
- Learning tokens helps you understand C++ syntax more easily.
Written by Shubhranshu Shekhar, who has trained 20000+ students in coding.
