Application monitoring is essential for maintaining performance, uptime, and security. Python offers powerful libraries and tools for real-time monitoring, logging, and alerting. This guide covers:
Why Monitoring is Important
Types of Monitoring
Monitoring System Performance (CPU, Memory, Disk, Network)
Application Logging and Error Tracking
Using Prometheus and Grafana for Monitoring
Real-time Alerts with Python
1. Why Monitoring is Important?
Monitoring helps in:
🔹 Detecting performance bottlenecks
🔹 Ensuring high availability
🔹 Identifying security vulnerabilities
🔹 Tracking resource utilization
🔹 Sending alerts for failures or anomalies
2. Types of Monitoring
2.1 System Monitoring
✔ Tracks CPU, memory, disk, and network usage
✔ Detects high resource consumption
✔ Prevents server crashes and downtime
2.2 Application Monitoring
✔ Logs errors, warnings, and exceptions
✔ Tracks API response times
✔ Monitors database performance
2.3 Log Monitoring
✔ Captures system logs and application logs
✔ Helps in debugging and troubleshooting
✔ Detects security incidents and threats
2.4 Real-Time Alerting
✔ Sends notifications via email, Slack, or Telegram
✔ Detects anomalies and failures automatically
3. Monitoring System Performance with Python
Python’s psutil module helps monitor CPU, memory, disk, and network usage.
3.1 Install psutil
pip install psutil
3.2 Monitor CPU and Memory Usage
import psutil
cpu_usage = psutil.cpu_percent(interval=1)
memory_info = psutil.virtual_memory()
print(f"CPU Usage: {cpu_usage}%")
print(f"Memory Usage: {memory_info.percent}%")
3.3 Monitor Disk Usage
disk_usage = psutil.disk_usage('/')
print(f"Disk Usage: {disk_usage.percent}%")
3.4 Monitor Network Activity
net_io = psutil.net_io_counters()
print(f"Bytes Sent: {net_io.bytes_sent}")
print(f"Bytes Received: {net_io.bytes_recv}")
4. Application Logging and Error Tracking
Python’s logging module is useful for tracking errors and performance.
4.1 Setup Logging in Python
import logging
logging.basicConfig(filename='app.log', level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s')
logging.info("Application Started")
logging.warning("This is a warning")
logging.error("An error occurred")
4.2 Capture Exceptions in Logs
try:
x = 10 / 0
except Exception as e:
logging.error(f"Exception occurred: {e}")
5. Monitoring Web Applications with Flask
Integrate monitoring in a Flask web application.
5.1 Install Flask
pip install flask
5.2 Create a Flask Monitoring Route
from flask import Flask, jsonify
import psutil
app = Flask(__name__)
@app.route('/health')
def health_check():
return jsonify({
"cpu_usage": psutil.cpu_percent(interval=1),
"memory_usage": psutil.virtual_memory().percent,
"disk_usage": psutil.disk_usage('/').percent
})
if __name__ == '__main__':
app.run(port=5000)
Run the Flask app and check http://localhost:5000/health to monitor system stats.
6. Monitoring with Prometheus and Grafana
6.1 Install Prometheus Client for Python
pip install prometheus_client
6.2 Expose Metrics with Prometheus in Flask
from flask import Flask
from prometheus_client import start_http_server, Gauge
import psutil
app = Flask(__name__)
cpu_metric = Gauge('cpu_usage', 'CPU Usage Percentage')
memory_metric = Gauge('memory_usage', 'Memory Usage Percentage')
@app.route('/')
def index():
cpu_metric.set(psutil.cpu_percent())
memory_metric.set(psutil.virtual_memory().percent)
return "Metrics Updated"
if __name__ == '__main__':
start_http_server(8000)
app.run(port=5000)
Now, Prometheus can scrape http://localhost:8000 for metrics.
7. Sending Alerts with Python
7.1 Email Alerts for High CPU Usage
import smtplib
import psutil
def send_email_alert(subject, message):
server = smtplib.SMTP("smtp.gmail.com", 587)
server.starttls()
server.login("your-email@gmail.com", "your-password")
server.sendmail("your-email@gmail.com", "recipient-email@gmail.com",
f"Subject: {subject}\n\n{message}")
server.quit()
cpu_usage = psutil.cpu_percent(interval=1)
if cpu_usage > 80:
send_email_alert("High CPU Usage Alert", f"CPU usage is at {cpu_usage}%!")
(Use App Passwords for Gmail instead of the actual password.)
7.2 Telegram Alerts for Errors
import requests
def send_telegram_alert(message):
token = "YOUR_TELEGRAM_BOT_TOKEN"
chat_id = "YOUR_CHAT_ID"
url = f"https://api.telegram.org/bot{token}/sendMessage"
data = {"chat_id": chat_id, "text": message}
requests.post(url, data=data)
try:
x = 10 / 0
except Exception as e:
send_telegram_alert(f"Application Error: {e}")