CI/CD with Python

Continuous Integration (CI) and Continuous Deployment (CD) are essential practices in modern software development. They help automate testing, integration, and deployment, ensuring faster and more reliable software releases. Python, with its rich ecosystem, supports CI/CD through tools like GitHub Actions, Jenkins, GitLab CI/CD, Travis CI, and Docker.

In this guide, we will cover:

• What CI/CD is and why it matters
• Setting up a CI/CD pipeline for a Python project
• Using different tools for CI/CD in Python

---

1. Understanding CI/CD

1.1 What is CI/CD?

• Continuous Integration (CI): Automatically integrates and tests code changes whenever developers push to a repository.
• Continuous Deployment (CD): Automatically deploys tested and validated code to production.

1.2 Benefits of CI/CD

Reduces manual errors
Ensures faster feedback on code changes
Automates testing and deployment
Improves software quality

---

2. Setting Up a CI/CD Pipeline for Python

To demonstrate, we will set up a basic CI/CD pipeline for a Python project using GitHub Actions.

2.1 Prerequisites

• A GitHub repository with a Python project
• A requirements.txt file listing dependencies
• Unit tests using pytest

---

2.2 Continuous Integration (CI) with GitHub Actions

Step 1: Create a .github/workflows/ci.yml file

Inside your Python project, create the folder structure:

.github/workflows/ci.yml

Add the following configuration:

yamlCopyEditname: Python CI

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout Code
      uses: actions/checkout@v3

    - name: Set up Python
      uses: actions/setup-python@v3
      with:
        python-version: '3.9'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt

    - name: Run Tests
      run: pytest

What This Does

• Triggers on code push and pull requests to the main branch
• Checks out the code and sets up Python
• Installs dependencies from requirements.txt
• Runs tests using pytest

---

2.3 Continuous Deployment (CD) with GitHub Actions

After testing, we can automate deployment.

Step 1: Create a .github/workflows/cd.yml file

name: Python CD

on:
  push:
    branches:
      - main

jobs:
  deploy:
    runs-on: ubuntu-latest

    steps:
    - name: Checkout Code
      uses: actions/checkout@v3

    - name: Set up Python
      uses: actions/setup-python@v3
      with:
        python-version: '3.9'

    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt

    - name: Deploy Application
      env:
        SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
        SERVER_IP: ${{ secrets.SERVER_IP }}
      run: |
        echo "$SSH_PRIVATE_KEY" > private_key.pem
        chmod 600 private_key.pem
        ssh -i private_key.pem user@$SERVER_IP "cd /var/www/myapp && git pull && systemctl restart myapp"

What This Does

• Deploys the application when code is pushed to main
• Uses SSH to connect to a remote server and update the code
• Restarts the application

---

3. CI/CD with Jenkins

Jenkins is a powerful open-source CI/CD tool.

3.1 Install Jenkins

1. Install Java: sudo apt update sudo apt install openjdk-11-jdk
2. Install Jenkins: wget -q -O - https://pkg.jenkins.io/debian/jenkins.io.key | sudo apt-key add - sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list' sudo apt update sudo apt install jenkins
3. Start Jenkins: sudo systemctl start jenkins

---

3.2 Setting Up a Jenkins CI/CD Pipeline

1. Open Jenkins UI (http://localhost:8080) and create a new Pipeline Job.
2. Add the following Pipeline Script:

pipeline {
    agent any

    stages {
        stage('Checkout Code') {
            steps {
                git 'https://github.com/your-repo.git'
            }
        }

        stage('Install Dependencies') {
            steps {
                sh 'pip install -r requirements.txt'
            }
        }

        stage('Run Tests') {
            steps {
                sh 'pytest'
            }
        }

        stage('Deploy') {
            steps {
                sh 'scp -r . user@server:/var/www/myapp'
                sh 'ssh user@server "systemctl restart myapp"'
            }
        }
    }
}

What This Does

• Fetches the latest code
• Installs dependencies
• Runs tests using pytest
• Deploys the application

---

4. CI/CD with Docker

Using Docker ensures a consistent deployment environment.

4.1 Create a Dockerfile

FROM python:3.9

WORKDIR /app
COPY . .
RUN pip install -r requirements.txt

CMD ["python", "app.py"]

4.2 Automate Deployment with GitHub Actions

Add the following step in .github/workflows/cd.yml:

    - name: Build and Push Docker Image
      run: |
        docker build -t myapp:latest .
        docker tag myapp:latest myrepo/myapp:latest
        docker login -u ${{ secrets.DOCKER_USERNAME }} -p ${{ secrets.DOCKER_PASSWORD }}
        docker push myrepo/myapp:latest

This Builds and Pushes the Docker Image to DockerHub