1. What is Unicode?

Unicode is a character encoding standard that assigns a unique number (code point) to every character, symbol, or glyph used in written languages. It supports over 1.1 million characters, covering almost all scripts and symbols used worldwide. Python 3.x uses Unicode by default for all strings.

            
# Example of Unicode characters
var = "3/4"
print(var)  # Output: 3/4

var = "\u00BE"
print(var)  # Output: ¾
            
        

Explanation: The string "3/4" is displayed as is, while "\u00BE" represents the Unicode character for ¾.

2. Character Encoding

Character encoding is the process of converting Unicode code points into a sequence of bytes for storage or transmission. The most common encodings are UTF-8, UTF-16, and UTF-32. Python uses UTF-8 as the default encoding for source code and strings.

            
# Example of encoding and decoding
string = "Hello"
tobytes = string.encode('utf-8')  # Encode to bytes
print(tobytes)  # Output: b'Hello'

string = tobytes.decode('utf-8')  # Decode back to string
print(string)  # Output: Hello
            
        

Explanation: The encode() method converts a string to bytes, while the decode() method converts bytes back to a string.

3. Working with Unicode Characters

You can use Unicode values directly in Python strings by prefixing them with \u (for 16-bit values) or \U (for 32-bit values). Here’s an example:

            
# Example of using Unicode values
string = "\u20B9"  # Rupee symbol
print(string)  # Output: ₹

tobytes = string.encode('utf-8')  # Encode to bytes
print(tobytes)  # Output: b'\xe2\x82\xb9'

string = tobytes.decode('utf-8')  # Decode back to string
print(string)  # Output: ₹
            
        

Explanation: The Unicode value \u20B9 represents the Indian Rupee symbol (₹). The string is encoded to bytes and then decoded back to a string.

💡 Pro Tip: Always Use UTF-8 Encoding

UTF-8 is the most widely used encoding format because it is backward-compatible with ASCII and supports all Unicode characters. Always use UTF-8 when working with text in Python to avoid encoding issues.