Tokens in C | C Tokens

Each and every smallest individual unit in a C program is known as tokens. Tokens are the basic building blocks in C language which are constructed together to write a program. A compiler breaks a C program into tokens and then proceeds ahead to the next stage used in the compilation process.

 

C has following six types of tokens:

• Keywords       e.g. int, for
• Identifiers       e.g. sum, total
• Constants       e.g. 10, 20
• Strings            e.g. “welcome”
• Operators       e.g. +, -, *, /
• Special Characters   e.g. ( ), { }

Keywords:

Keywords are reserved words whose meaning has already been fixed to the C compiler. Each keyword is meant to perform a specific function in a program. Since keywords are referred names for a compiler, they cannot be used as variable names because if we do so, we are trying to assign a new meaning to the keyword, which is not allowed. We canoe redefine keywords. All keywords have a predefined meaning and these meanings cannot be changed. All keywords must be written in lowercase.

For example:

int age;

Here, int is a keyword that indicates age is a variable of type int (integer).

There are 32 keywords available in C.

auto	break	case	char	const	continue
default	do	double	else	enum	extern
float	for	goto	if	int	long
register	return	short	signed	sizeof	static
struct	switch	typedef	union	unsigned	void
volatile	while

 

Identifiers:

Identifiers are user defined words. These are used to name the variables, functions, arrays, structures, etc. These names can be in uppercase letters, lowercase letter or a combination. Identifiers must be unique. They are created to give a unique name to an entity to identify it during the execution of the program.

For example:

int age;

int roll;

Here, age and roll are identifiers.

Identifier names must be different from keywords. We cannot use int as an identifier because int is a keyword.

Constants:

Constants in C are fixed values that do not change throughout the program. The constant can be viewed as a read only memory variable. Constants can be declared using the keyword const. 

Syntax: const variable_name=value;

Constants are broadly classified into numeric constants and character constants.

 Numeric Constants:

    i. Integer Constants

    ii. Real Constants

2    

      Character Constants:

        i. Character constants

        ii. String Constants

Integer Constant:

Integer constants are whole number which has no decimal point. In ‘C’ there can be three types of integer constants:

·    Decimal Integer:            0-------9(base 10)

·    Octal Integer:                 0-------7(base 8)

·    Hexadecimal Integer:    0----9, A------F(base 16)

     Example: 15, -265, 0, 99818, +25, 045, 0X6

Real Constant:

Real constant is also called floating point constant. These have fractional parts to represent quantities like average, fees, height, area etc. which cannot be represented by integer number precisely.

A real constant can be represented in 2 forms.

1.  Decimal form      E.g. 15.5

2.  Exponent form    E.g. 1275E-2, -14E-2

 

Character Constant:

 A character constant is a single character surrounded by a pair of single quotes. A character value occupies 1 byte of memory.

Example: 'X', '5', ';'

The character constant '5' is not the same as the integer 5. Each character constant has an ASCII (American Standard Code for Information Interchange) value associated with it. For example, the following statement will print 65 and A respectively:

printf(“%d”, ‘A’);

printf(“%c”, ‘65’);

 

String Constant:

 

A string constant is a group of characters surrounded by double quotes. These characters may be letters, numbers or any special characters.

Example:   "2015", "welcome"

Operators:

 An Operator can be defined as a symbol that specifies an operation to be performed. C language supports a rich set of operators. Operators help the programmer to perform computation on values.

An operator is a symbol that tells the compiler to perform certain mathematical or logical computations. The data items on which the operators act upon are called operands. Some operators require a single operand to perform operation while others might require two operands.

Arithmetic operators are the most common. Other operators are used for comparison of values, combination of logical states, manipulation of individual binary digits etc.

Categories of Operators:

Operators in C can be classified into a number of categories as follows:

1	Assignment operator	=
2	Arithmetic operators	+, -, *, /, %
3	Relational operators	==, !=, <=, >=, <, >
4	Logical operators	&&, ||, !
5	Increment/Decrement operators	++, --
6	Conditional operator	?:
7	Bitwise operators	&,  |,  ^,  << , >>,  ~
8	Other operators	Address operator & Pointer operator(arrow)-> Indirection operator *

Special Characters:

C language supports a rich set of special character for some special purposes, such as logical operations, mathematical operations, checking of conditions etc.

We can also use these characters for defining the identifiers in a much better way. For instance, we use underscores for constructing a longer name for a variable, etc.

The C programming language provides support for the following types of special characters:

Character	Description	Character	Description
,	Comma	&	Ampersand
.	Period	^	Caret
;	Semicolon	*	Asterisk
:	Colon	-	Minus sign
?	Question Mark	+	Plus sign
’	Apostrophe	<	Less than
!	Exclamation	>	Greater than
|	Vertical Slash	=	Equal to
/	Slash	(	Left parenthesis
\	Back slash	)	Right parenthesis
“	Quotation mark	[	Left square bracket
~	Tilde	]	Right square bracket
_	Underscore	{	Left curly bracket
$	Dollar Sign	}	Right curly bracket
%	Percentage sign	#	Number sign(hash)