Implementing pagination for API responses

Implementing pagination in API responses is essential for efficiently handling large datasets and improving performance. Below, I’ll guide you through different types of pagination and how to implement them in a REST API using Node.js (Express.js) and MongoDB.

---

Types of Pagination

1. Offset-Based Pagination

• Uses limit and offset parameters.
• Best for simple datasets but inefficient for large data sets.

Request Example:

GET /api/items?limit=10&offset=20

• limit: Number of records per request.
• offset: Number of records to skip.

Implementation in Node.js (Express.js)

const express = require('express');
const mongoose = require('mongoose');

const app = express();
mongoose.connect('mongodb://localhost:27017/testdb');

const ItemSchema = new mongoose.Schema({ name: String });
const Item = mongoose.model('Item', ItemSchema);

app.get('/api/items', async (req, res) => {
    let { limit = 10, offset = 0 } = req.query;
    limit = parseInt(limit);
    offset = parseInt(offset);

    const items = await Item.find().skip(offset).limit(limit);
    res.json({ data: items, limit, offset });
});

app.listen(3000, () => console.log('Server running on port 3000'));

Pros: Simple to implement.
Cons: Slows down for large datasets because of .skip() usage.

---

2. Cursor-Based Pagination

• Uses cursors (IDs or timestamps) instead of offsets.
• More efficient for large datasets than offset-based pagination.

Request Example:

GET /api/items?limit=10&cursor=65ffb1e39ac2e9f2d8d3e123

• cursor: ID of the last retrieved item.

Implementation in Node.js (Express.js)

app.get('/api/items', async (req, res) => {
    let { limit = 10, cursor } = req.query;
    limit = parseInt(limit);

    let query = cursor ? { _id: { $gt: cursor } } : {};
    const items = await Item.find(query).limit(limit);

    const nextCursor = items.length ? items[items.length - 1]._id : null;

    res.json({ data: items, nextCursor });
});

Pros: Faster for large datasets.
Cons: Can’t jump to arbitrary pages.

---

3. Page-Based Pagination

• Uses page numbers instead of offsets.
• Works similar to offset pagination.

Request Example:

GET /api/items?page=3&limit=10

• page: The current page number.
• limit: Number of records per request.

Implementation in Node.js (Express.js)

app.get('/api/items', async (req, res) => {
    let { limit = 10, page = 1 } = req.query;
    limit = parseInt(limit);
    page = parseInt(page);

    const skip = (page - 1) * limit;
    const items = await Item.find().skip(skip).limit(limit);

    res.json({ data: items, page, limit });
});

Pros: Easy to implement and user-friendly.
Cons: Slower for large datasets (similar to offset pagination).

---

4. Keyset Pagination

• Uses sorted unique keys instead of offsets.
• Best for time-sensitive data (e.g., social media feeds, logs).

Request Example:

GET /api/items?limit=10&lastCreatedAt=2024-03-27T12:00:00.000Z

• lastCreatedAt: Timestamp of the last item retrieved.

Implementation in Node.js (Express.js)

app.get('/api/items', async (req, res) => {
    let { limit = 10, lastCreatedAt } = req.query;
    limit = parseInt(limit);

    let query = lastCreatedAt ? { createdAt: { $lt: new Date(lastCreatedAt) } } : {};
    const items = await Item.find(query).sort({ createdAt: -1 }).limit(limit);

    const nextLastCreatedAt = items.length ? items[items.length - 1].createdAt : null;

    res.json({ data: items, nextLastCreatedAt });
});

Pros: Fast for time-ordered data.
Cons: Doesn’t support jumping to arbitrary pages.

---

Best Pagination Approach for Different Use Cases

Pagination Type	Best For	Pros	Cons
Offset-Based	Small datasets, Simple UI	Easy to implement	Slow for large data
Cursor-Based	Large datasets, Real-time apps	Faster, scalable	No jumping to pages
Page-Based	Paginated UI (e.g., eCommerce)	User-friendly	Slower for large data
Keyset-Based	Time-sensitive data (e.g., feeds, logs)	Fast	No jumping to pages