PyTorch Basics: A Comprehensive Guide

Introduction to PyTorch

PyTorch is an open-source deep learning framework developed by Facebook’s AI Research (FAIR) lab. It provides an easy-to-use platform for tensor computations, automatic differentiation, and neural network training. PyTorch is widely used for research and production, offering flexibility, GPU acceleration, and dynamic computation graphs.

Why PyTorch?

✅ Easy to Learn – Pythonic and intuitive syntax.
✅ Dynamic Computation Graphs – Unlike TensorFlow’s static graphs, PyTorch builds computation graphs on the fly.
✅ GPU Acceleration – Seamlessly moves tensors between CPU and GPU.
✅ Strong Community Support – Widely adopted in academia and industry.
✅ Built-in Automatic Differentiation – Useful for backpropagation.

---

1. Installing PyTorch

To install PyTorch, follow these steps:

Step 1: Set Up a Virtual Environment (Optional)

python -m venv pytorch_env
source pytorch_env/bin/activate   # Mac/Linux
pytorch_env\Scripts\activate      # Windows

Step 2: Install PyTorch

To install the latest version of PyTorch, visit the official website: PyTorch Install and run the recommended command.

For example:

pip install torch torchvision torchaudio

Step 3: Verify Installation

import torch
print(torch.__version__)  # Should print the installed PyTorch version

---

2. Tensors in PyTorch

Tensors are multi-dimensional arrays similar to NumPy arrays but optimized for GPU acceleration.

2.1 Creating Tensors

import torch

# Scalar (0D Tensor)
scalar = torch.tensor(5)
print(scalar)

# Vector (1D Tensor)
vector = torch.tensor([1, 2, 3])
print(vector)

# Matrix (2D Tensor)
matrix = torch.tensor([[1, 2], [3, 4]])
print(matrix)

# 3D Tensor
tensor_3d = torch.tensor([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]])
print(tensor_3d)

2.2 Tensor Operations

# Element-wise addition
a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])
c = a + b
print(c)

# Multiplication
d = a * b
print(d)

# Dot Product
dot_product = torch.dot(a, b)
print(dot_product)

2.3 Reshaping Tensors

tensor = torch.arange(9)
reshaped_tensor = tensor.view(3, 3)
print(reshaped_tensor)

---

3. Moving Tensors to GPU

PyTorch allows easy computation on GPUs for faster performance.

# Check if GPU is available
device = "cuda" if torch.cuda.is_available() else "cpu"
print("Using device:", device)

# Move tensor to GPU
tensor = torch.tensor([1, 2, 3])
tensor = tensor.to(device)
print(tensor)

---

4. Automatic Differentiation with Autograd

PyTorch provides automatic differentiation for gradient computation using autograd.

4.1 Enabling Autograd

x = torch.tensor(2.0, requires_grad=True)
y = x ** 3 + 5 * x

# Compute gradients
y.backward()
print(x.grad)  # dy/dx

4.2 Tracking Gradients in Neural Networks

a = torch.tensor([2.0, 3.0], requires_grad=True)
b = a ** 2
c = b.sum()
c.backward()
print(a.grad)  # dc/da

---

5. Creating Neural Networks with PyTorch

5.1 Import Required Libraries

import torch
import torch.nn as nn
import torch.optim as optim

5.2 Define a Simple Neural Network

class SimpleNN(nn.Module):
    def __init__(self):
        super(SimpleNN, self).__init__()
        self.fc1 = nn.Linear(2, 4)  # Input layer
        self.fc2 = nn.Linear(4, 1)  # Output layer

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = torch.sigmoid(self.fc2(x))
        return x

model = SimpleNN()
print(model)

5.3 Define Loss and Optimizer

criterion = nn.MSELoss()  # Mean Squared Error Loss
optimizer = optim.SGD(model.parameters(), lr=0.01)  # Stochastic Gradient Descent

---

6. Training a Neural Network

6.1 Prepare a Sample Dataset

# Example dataset
x_train = torch.tensor([[1.0, 2.0], [2.0, 3.0], [3.0, 4.0]], requires_grad=True)
y_train = torch.tensor([[0.0], [1.0], [0.0]])

6.2 Training Loop

epochs = 100
for epoch in range(epochs):
    # Forward pass
    y_pred = model(x_train)

    # Compute loss
    loss = criterion(y_pred, y_train)

    # Backpropagation
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if epoch % 10 == 0:
        print(f'Epoch {epoch}, Loss: {loss.item()}')

---

7. Loading Pretrained Models

PyTorch provides pre-trained models through torchvision.

from torchvision import models

# Load a pretrained ResNet model
resnet = models.resnet18(pretrained=True)
print(resnet)

---

8. Saving and Loading Models

8.1 Save the Model

torch.save(model.state_dict(), "model.pth")

8.2 Load the Model

model.load_state_dict(torch.load("model.pth"))

---

9. PyTorch for Computer Vision

import torchvision.transforms as transforms
from torchvision.datasets import MNIST
from torch.utils.data import DataLoader

# Define transforms
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))])

# Load Dataset
train_dataset = MNIST(root='./data', train=True, transform=transform, download=True)
train_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)

# Get a batch of images
images, labels = next(iter(train_loader))
print(images.shape)  # (batch_size, channels, height, width)

---

10. Deploying a PyTorch Model with Flask

10.1 Create a Flask API

from flask import Flask, request, jsonify
import torch

app = Flask(__name__)
model = SimpleNN()
model.load_state_dict(torch.load("model.pth"))

@app.route('/predict', methods=['POST'])
def predict():
    data = request.get_json()
    tensor_input = torch.tensor(data['input'])
    prediction = model(tensor_input).detach().numpy().tolist()
    return jsonify({'prediction': prediction})

if __name__ == '__main__':
    app.run(port=5000)

---