Sending HTTP Requests with Requests Module

The requests module in Python is a popular library for sending HTTP requests. It simplifies interactions with web services by handling headers, cookies, authentication, and more.

Installing the Requests Module

Before using requests, install it using pip:

pip install requests

Sending a Simple GET Request

A GET request is used to retrieve data from a server.

import requests

response = requests.get("https://jsonplaceholder.typicode.com/posts/1")
print(response.status_code)  # Output: 200 (OK)
print(response.json())  # Output: JSON response

✔ response.status_code → HTTP status code (e.g., 200 OK, 404 Not Found).
✔ response.text → Returns response as a string.
✔ response.json() → Parses response as JSON.

Sending Parameters in a GET Request

You can pass query parameters using a dictionary.

params = {"userId": 1}
response = requests.get("https://jsonplaceholder.typicode.com/posts", params=params)

print(response.url)  # URL with query parameters
print(response.json())  # JSON response

The query parameters are automatically appended to the URL:
https://jsonplaceholder.typicode.com/posts?userId=1

Sending a POST Request

A POST request is used to send data to a server.

data = {"title": "New Post", "body": "This is a new post.", "userId": 1}
response = requests.post("https://jsonplaceholder.typicode.com/posts", json=data)

print(response.status_code)  # 201 Created
print(response.json())  # JSON response with the new post

✔ Use json=data to send data as JSON.
✔ Alternatively, use data=data to send form-encoded data.

Sending Headers with a Request

You can send custom headers like User-Agent, Authorization, etc.

headers = {"User-Agent": "Mozilla/5.0"}
response = requests.get("https://jsonplaceholder.typicode.com/posts/1", headers=headers)

print(response.status_code)
print(response.text)

Handling Authentication

For APIs requiring authentication, use Basic Auth:

from requests.auth import HTTPBasicAuth

response = requests.get("https://httpbin.org/basic-auth/user/pass", auth=HTTPBasicAuth("user", "pass"))
print(response.status_code)  # 200 if authenticated

For Bearer Token Authentication (commonly used in APIs):

headers = {"Authorization": "Bearer YOUR_ACCESS_TOKEN"}
response = requests.get("https://api.example.com/protected", headers=headers)

Handling Timeouts

Avoid hanging requests by setting a timeout.

response = requests.get("https://jsonplaceholder.typicode.com/posts/1", timeout=5)  # 5-second timeout

✔ Raises requests.exceptions.Timeout if it takes too long.

Handling Errors with Try-Except

Gracefully handle exceptions like connection errors.

try:
    response = requests.get("https://invalid-url.com", timeout=5)
    response.raise_for_status()  # Raise an HTTPError for bad responses
except requests.exceptions.RequestException as e:
    print(f"Error: {e}")

✔ raise_for_status() raises an exception for HTTP errors (e.g., 404, 500).

Uploading Files with POST

Use files parameter to send files.

files = {"file": open("example.txt", "rb")}
response = requests.post("https://httpbin.org/post", files=files)
print(response.json())

✔ "rb" opens the file in binary mode for upload.

Downloading Files

To save a file from a URL:

response = requests.get("https://example.com/file.zip", stream=True)
with open("file.zip", "wb") as f:
    for chunk in response.iter_content(chunk_size=1024):
        f.write(chunk)

✔ stream=True ensures efficient downloading in chunks.

Handling Cookies

Retrieve and send cookies automatically.

response = requests.get("https://httpbin.org/cookies")
print(response.cookies)

cookies = {"session_id": "123456"}
response = requests.get("https://httpbin.org/cookies", cookies=cookies)
print(response.text)

Session Handling (Maintaining Login)

Use requests.Session() to persist cookies across multiple requests.

session = requests.Session()
session.auth = ("user", "pass")

response = session.get("https://httpbin.org/cookies/set/sessionid/12345")
print(session.cookies)

Redirects Handling

By default, requests follows redirects.

response = requests.get("http://github.com", allow_redirects=False)
print(response.status_code)  # 301 or 302 (Redirected)
print(response.url)  # Final redirected URL

✔ Use allow_redirects=False to disable automatic redirection.

Summary of HTTP Methods

HTTP Method	Description
`GET`	Retrieve data from a server
`POST`	Send data to a server
`PUT`	Update existing data
`DELETE`	Remove data
`PATCH`	Partially update data