Tokens and Character Set in Python

Python is a high-level, all-purpose coding language. Its syntax enables developers to articulate their notions in minimal lines of code, referred to as scripts. It was built with a focus on script readability. Character sets and tokens are all included in these scripts. We shall discover more about various character sets and tokens in this tutorial.

Character Set

A character set in Python is a collection of legal characters that a scripting language will recognize when writing a script. We are referring to the Python coding language in this instance. Therefore, the character set in Python is a legitimate collection of characters that the Python language can recognize. These represent the Python scripting language's supported characters. Python is compatible with all ASCII and Unicode characters, including:

• Alphabets: These include all the small (a-z) and capital (A-Z) alphabets.
• Digits: It includes all the single digits 0-9.
• Special Symbols: It includes all the types of special characters," 'l ; : ! ~ @ # $ % ^ ` & * ( ) _ + - = { } [ ] \ .
• White Spaces: White spaces are also a part of the character set. These are tab space, newline, blank space, and carriage return.
• Other: Python supports all the types of ASCII and UNICODE characters that constitute the Python character set.

Tokens

The smallest distinct element in a Python program is called a token. Tokens are used to construct each phrase and command in a program. The different Python tokens include:

Keywords

In a computer language, keywords are English words with particular importance or meaning. We cannot utilize them for any arbitrary reason, including as names for functions or variables. They are utilized for their unique qualities. The 33 keywords in Python include, among others: try, break, continue, except, or, False, True, class, and, as, assert, print, import, raise, while, for, in, not, if, elif, etc.

Code

Output:

12
18
21
25
30
32

Identifiers

The names assigned to any variable, class, function, list, method, etc., for their recognition are known as identifiers. Python has certain guidelines for naming identifiers and is a case-sensitive programming language. To create an identifier, follow these guidelines: -

• Python is case-sensitive, as was already mentioned. So, when defining identifiers, the case is important. Thus, the terms "python" and "Python" are not interchangeable.
• A capital alphabet (A-Z), a small alphabet (a-z), or an underscore (_) precedes the word "identifier." No other character can be used as the starting point.
• Digits are also allowed to be a member of the identifier, along with the letters and underscores, but they cannot be the initial character.
• Any additional special symbols or whitespace must be avoided when creating an identity.
• A keyword cannot be an identifier.

Code

Output:

Hello
Javatpoint
World
!!

Literals or Values

The constant values or data objects utilized in a source program are known as literals. Various types of literal are supported by Python, including:

String Literals: The Python string literals are represented by text enclosed in solo, double, or triple quotations. Examples include "Computer Programming," "sigma," etc. Multi-line phrases can also be written using triple quotations.

Code

Output:

Hello
World
Python is a
    high-level and
    general purpose language

Character Literals: Another string literal is called the character literal, which encloses the letter in single or double quotes.

Numeric Literals: These are literals that have been written as numbers. Python supports the following three numerical literals:

• Integer Literal: Contains 0 as well as positive and negative values. There are no fractional pieces included. In addition, literals in binary, decimal, octal, and hexadecimal are permitted.
• Float Literal: It contains real numbers in the positive and negative ranges. Fractional portions are also a part of it.
• Complex Literal: This contains the numerals a+bi, where a stands for the fundamental component and b for the complex component.

Code

Output:

literal 50 is of type <class 'int'>
literal 13.3 is of type <class 'float'>
literal -14 is of type <class 'int'>

Boolean Literals

In Python, Boolean tokens only have two possible values. Every one of these is true or false.

Code

Output:

The Boolean value of v is True
The Boolean value of w is False
The Boolean value of y is True
The value of z is 11

Special Literals

The special literal "None" in Python exists. It is employed to signify emptiness, the lack of values, or nothingness.

Code

Output:

None

Literals Collections

List, dictionary, tuple, and sets are examples of Python literal collections.

• List: It is a comma-separated list of components enclosed in square brackets. These elements can be of any data type and can be changed.
• Tuple: In round brackets, this is similar to a list having comma-separated items or values. The values are fixed and can have any Python data type.
• Dictionary: This data structure is an unordered set of key-value combinations.
• Set: It is the group of components enclosed in curly braces, "{}"

Code

Output:

Literal [23, 'Python', 1.2, 'Character'] is of type <class 'list'>
Literal (1, 3, 6, 'world') is of type <class 'tuple'>
Literal {'a': 1, 'b': 2, 'c': 3} is of type <class 'dict'>
Literal {9, 3, 5, 7} is of type <class 'set'>

Operators

These represent the tokens in an expression in charge of carrying out an operation. Operands are the elements to which an operation is executed. Operators come in binary and unary forms. Unary operators operate on a single argument, such as complementing and others. At the same time, the operands for binary operators require two.

Code

Output:

x == y = False
x * y = 10
x & y = 0
x | y = 7
x += y = 7
x /= y = 3.5