Logging in Python - Rishan Solutions

Logging is a crucial aspect of software development, helping developers track, debug, and analyze program behavior. Python provides a built-in logging module for logging messages in a structured way.

Why Use Logging?

Provides better debugging than print().
Supports different log levels (INFO, ERROR, WARNING, etc.).
Allows saving logs to files for later analysis.
Supports log formatting for better readability.

Importing the logging Module:

import logging

1. Basic Logging in Python

Example: Simple Logging

import logging

logging.basicConfig(level=logging.INFO)  # Set log level
logging.info("This is an INFO message")
logging.warning("This is a WARNING message")
logging.error("This is an ERROR message")

Output:

WARNING:root:This is a WARNING message
ERROR:root:This is an ERROR message

Key Observations:

Default log level is WARNING, so INFO logs are not displayed.
To show all logs, we must set level=logging.INFO.

2. Logging Levels

Python provides five standard log levels:

Level	Value	Purpose
`DEBUG`	10	Detailed debugging information.
`INFO`	20	General information about program execution.
`WARNING`	30	Indication of possible issues.
`ERROR`	40	Errors that may prevent execution.
`CRITICAL`	50	Serious errors requiring immediate attention.

Example: Using All Log Levels

import logging

logging.basicConfig(level=logging.DEBUG)  # Show all log levels

logging.debug("This is a DEBUG message")
logging.info("This is an INFO message")
logging.warning("This is a WARNING message")
logging.error("This is an ERROR message")
logging.critical("This is a CRITICAL message")

Output:

DEBUG:root:This is a DEBUG message
INFO:root:This is an INFO message
WARNING:root:This is a WARNING message
ERROR:root:This is an ERROR message
CRITICAL:root:This is a CRITICAL message

3. Logging to a File

Instead of printing logs to the console, we can save them to a log file.

import logging

logging.basicConfig(filename="app.log", level=logging.INFO)

logging.info("Application started")
logging.warning("This is a warning")
logging.error("An error occurred")

This creates app.log with the following content:

INFO:root:Application started
WARNING:root:This is a warning
ERROR:root:An error occurred

Key Parameters in basicConfig()

filename="app.log" → Saves logs to a file.
level=logging.INFO → Logs INFO and higher messages.

4. Formatting Log Messages

We can customize log messages with timestamps, log levels, and module names.

import logging

logging.basicConfig(
    filename="formatted.log",
    level=logging.DEBUG,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

logging.debug("Debugging mode enabled")
logging.info("Application is running")
logging.warning("Low disk space warning")
logging.error("File not found")
logging.critical("Critical system failure")

This creates formatted.log with the following content:

2025-03-10 12:30:00 - DEBUG - Debugging mode enabled
2025-03-10 12:30:01 - INFO - Application is running
2025-03-10 12:30:02 - WARNING - Low disk space warning
2025-03-10 12:30:03 - ERROR - File not found
2025-03-10 12:30:04 - CRITICAL - Critical system failure

Common Formatting Codes

%(asctime)s → Timestamp
%(levelname)s → Log level (INFO, ERROR, etc.)
%(message)s → Log message
%(name)s → Logger name
%(filename)s → Filename where log was generated

5. Using Custom Loggers

For large applications, custom loggers help manage logs from different modules.

import logging

# Create a custom logger
logger = logging.getLogger("MyLogger")

# Set log level
logger.setLevel(logging.DEBUG)

# Create file handler
file_handler = logging.FileHandler("custom.log")

# Create formatter
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")

# Add formatter to file handler
file_handler.setFormatter(formatter)

# Add file handler to logger
logger.addHandler(file_handler)

# Logging messages
logger.debug("Debugging information")
logger.info("Process started")
logger.warning("This is a warning")
logger.error("An error occurred")
logger.critical("System failure")

This creates custom.log with structured logs:

2025-03-10 12:30:05 - MyLogger - DEBUG - Debugging information
2025-03-10 12:30:06 - MyLogger - INFO - Process started
2025-03-10 12:30:07 - MyLogger - WARNING - This is a warning
2025-03-10 12:30:08 - MyLogger - ERROR - An error occurred
2025-03-10 12:30:09 - MyLogger - CRITICAL - System failure

Advantages of Custom Loggers:

Allows separate logging for different modules.
Provides better log organization and filtering.

6. Rotating Log Files (`RotatingFileHandler`)

To prevent large log files, we can use rotating logs.

import logging
from logging.handlers import RotatingFileHandler

# Create a rotating file handler (max size: 1KB, keep 3 backups)
handler = RotatingFileHandler("rotating.log", maxBytes=1024, backupCount=3)

# Create logger
logger = logging.getLogger("RotatingLogger")
logger.setLevel(logging.INFO)
logger.addHandler(handler)

for i in range(50):
    logger.info(f"Log message {i}")

This keeps the latest 3 log files (rotating.log, rotating.log.1, rotating.log.2).

7. Logging Exceptions (`exc_info`)

To log exceptions with error details, use exc_info=True.

import logging

logging.basicConfig(filename="errors.log", level=logging.ERROR)

try:
    result = 10 / 0
except ZeroDivisionError:
    logging.error("An error occurred", exc_info=True)

This logs the full traceback to errors.log:

ERROR:root:An error occurred
Traceback (most recent call last):
  File "script.py", line 6, in <module>
    result = 10 / 0
ZeroDivisionError: division by zero

8. Disabling Logging

Disable logs to prevent unnecessary output.

import logging

logging.disable(logging.CRITICAL)  # Disable all logs of CRITICAL level and below
logging.info("This will not be logged")
logging.critical("This will also not be logged")

9. Summary Table

Feature	Method
Basic Logging	`logging.basicConfig(level=logging.INFO)`
Log to File	`filename="app.log"`
Custom Format	`format="%(asctime)s - %(levelname)s - %(message)s"`
Custom Logger	`logging.getLogger("MyLogger")`
Rotating Log Files	`RotatingFileHandler("logfile.log", maxBytes=1024, backupCount=3)`
Exception Logging	`logging.error("Error!", exc_info=True)`
Disable Logging	`logging.disable(logging.CRITICAL)`