Tokens in C

Before Understanding Tokens in C programming, it’s important to understand basic concepts on which concept of Tokens in C is built.

Any language whether it is a natural language, for instance, English, French, German, etc or computer programming language, you have to learn basic constituents first. For instance, alphabets are basic constituents of the English language, which when combined together produces words. When words are combined together it becomes sentences and when few sentences are combined together it becomes a paragraph.

Similarly, you have to first learn basic constituents of C language. The basic constituents are alphabets, digits, special symbol.

Basic Letters ⇒ Identifiers  ⇒ Instructions  ⇒ Program.

Constituents Of C

  • Alphabets                   A to Z and a to z
  • Digits                           0-9
  • Special Symbol          ~! @ # $ % ^ & ( ) – _ = + { } [ ] : ; ” ‘< > , . ? /

Tokens in C

A C program consists of various tokens and a token is either a keyword, an identifier, a constant, a string literal, or a symbol. For example, the following C statement consists of five tokens.

printf(“Hello, World! \n”);

The individual tokens are
"Hello, World! \n"

There are six types of Tokens in C.

1. Keywords

Keywords are special words that are used to give a special meaning to the program and can’t be used as variable and constant. They are basically a sequence of characters that have fixed to mean for example break, for, while, do-while, do, if, int, long, char.

The following list shows the reserved words in C. These reserved words may not be used as constants or variables or any other identifier names.

auto else long switch
break enum register typedef
case extern return union
char float short unsigned
const for signed void
continue goto sizeof volatile
default if static while
do int struct _Packed

2. Identifiers

Identifiers are the sequence of alphabets and digit, but keyword should not be used as an identifier.

A C identifier is a name used to identify a variable, function, or any other user-defined item.

An identifier starts with a letter A to Z, a to z, or an underscore ‘_’ followed by zero or more letters, underscores, and digits (0 to 9).

C does not allow punctuation characters such as @, $, and % within identifiers. C is a case-sensitive programming language.

Thus, Student and student are two different identifiers in C.

Have You enjoyed the article on Tokens in C!

3. Constants 

The quantity which does not change during the execution of a program is known as constant. I have written a separate article on constants.

4. Variables

Variables are used to give the name and allocate memory space. An entity that may vary data during execution. For example, a, b, age, sex. I have written a separate article on variables.

5. String

A string is a collection of more than one character. For example, “JOHN”, “CALIFORNIA” String is represented by a pair of double quotes. I have written a separate article on Strings as this is a very important & big topic.

6. Operators

Operators act as connectors and they indicate what type of operation is being carried out. The values that can be operated by these operators are called operands. They are used to perform basic operations, comparison, manipulation of bits and so on. I have written a separate article on Operators as this is a very important & big topic.

Other Tokens in C, You Might Consider


In a C program, the semicolon is a statement terminator. That is, each individual statement must be ended with a semicolon. It indicates the end of one logical entity.

Given below are two different statements:

printf("Welcome To MyCTEACHER \n");
printf("Are You enjoying the article on Tokens in C");

Semicolon (;) is used to mark the end of a statement and beginning of another statement.

An absence of semicolon at the end of any statement will mislead the compiler to think that this statement is not yet finished. And it will add the next consecutive statement after it, which may lead to compilation(syntax) error.

#include <stdio.h>
int main()
return 0;

In the above program, I have omitted the semicolon from the printf(“…”)  statement, hence the compiler will start execution from printf until the semicolon after return 0; statement, by assuming it as a single statement and this will lead to compilation error.


Comments are plain simple text in a C program that is not compiled by the compiler. Programmers write comments for a better understanding of the program.

Though writing comments is not compulsory, but it is recommended to make your program more descriptive. It make the code more readable.

There are two ways in which we can write comments.

  • Using // This is used to write a single line comment.
  • Using /* */: The statements enclosed within /* and */ , are used to write multi-line comments.

Example of comments :

// This is a single line comment

/* This is a multi-line comment */

/* This is a multi-line
valid comment */

// this is not 
a valid comment because it's written in multiple lines.

Have You enjoyed the article on Tokens in C!

Whitespace in C

A line containing only whitespace, possibly with a comment, is known as a blank line, and a C compiler totally ignores it.

Whitespace is the term used in C to describe blanks, tabs, newline characters and comments. Whitespace separates one part of a statement from another and enables the compiler to identify where one element in a statement, such as an int, ends and the next element begins.

Therefore, in the following statement, there must be at least one whitespace character (usually space) between float and salary for the compiler to be able to distinguish them.

float salary = 5000$;

On the other hand, no whitespace characters are necessary between salary and =, or between = and 5000$, although you are free to include some if you wish to increase readability.

<< Prev                                                                                                              Next >>

Your friends are waiting for this content. Please Share To Aware Them!