Working with C Extensions in Python

Python is an interpreted language, which makes it easy to use but relatively slower for computational-heavy tasks. To improve performance, we can write C extensions that allow Python to call C functions directly, enabling faster execution.

---

1. Why Use C Extensions in Python?

Performance Boost – C code runs much faster than pure Python.
Low-Level Control – Direct access to memory and system resources.
Leverage Existing C Libraries – Use well-optimized C libraries in Python.
Parallel Processing – Avoid Python’s Global Interpreter Lock (GIL).

---

2. Basics of Python C Extensions

Python provides the Python C API for writing and integrating C extensions. The process involves:

1. Creating a C file containing the function definitions.
2. Compiling the C code into a shared library (.so on Linux, .pyd on Windows).
3. Importing the C extension in Python and calling the functions.

---

3. Writing a Simple C Extension

Step 1: Create a C File (example.c)

#define PY_SSIZE_T_CLEAN
#include <Python.h>

// A simple function that adds two numbers
static PyObject* add(PyObject* self, PyObject* args) {
    int a, b, result;
    if (!PyArg_ParseTuple(args, "ii", &a, &b)) {
        return NULL;
    }
    result = a + b;
    return PyLong_FromLong(result);
}

// Define methods in the module
static PyMethodDef ExampleMethods[] = {
    {"add", add, METH_VARARGS, "Adds two numbers"},
    {NULL, NULL, 0, NULL}  // Sentinel (marks end of methods)
};

// Define the module
static struct PyModuleDef examplemodule = {
    PyModuleDef_HEAD_INIT,
    "example",  // Module name
    NULL,       // Documentation
    -1,         // Size (-1 means global state)
    ExampleMethods
};

// Initialization function (called when module is imported)
PyMODINIT_FUNC PyInit_example(void) {
    return PyModule_Create(&examplemodule);
}

---

4. Compiling the C Extension

We use setup.py to compile the C extension.

Step 2: Create a setup.py File

from setuptools import setup, Extension

# Define the extension module
example_module = Extension("example", sources=["example.c"])

# Setup the module
setup(
    name="example",
    version="1.0",
    description="A simple C extension for Python",
    ext_modules=[example_module],
)

Step 3: Build and Install

Run the following command in the terminal:

python setup.py build
python setup.py install

This compiles the C code and installs it as a Python module.

---

5. Using the C Extension in Python

After installation, we can use the module like any other Python module.

Step 4: Import and Use the Module

import example

result = example.add(10, 5)
print(f"10 + 5 = {result}")

Output:

CopyEdit10 + 5 = 15

---

6. Passing Strings to C Extensions

We can also create functions that work with strings.

Example: Reverse a String in C (string_ops.c)

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <string.h>

// Reverse a string
static PyObject* reverse_string(PyObject* self, PyObject* args) {
    const char* input;
    if (!PyArg_ParseTuple(args, "s", &input)) {
        return NULL;
    }

    int len = strlen(input);
    char reversed[len + 1];

    for (int i = 0; i < len; i++) {
        reversed[i] = input[len - 1 - i];
    }
    reversed[len] = '\0';

    return Py_BuildValue("s", reversed);
}

// Define module methods
static PyMethodDef StringMethods[] = {
    {"reverse", reverse_string, METH_VARARGS, "Reverses a string"},
    {NULL, NULL, 0, NULL}
};

// Define module
static struct PyModuleDef stringmodule = {
    PyModuleDef_HEAD_INIT,
    "string_ops",
    NULL,
    -1,
    StringMethods
};

// Initialize module
PyMODINIT_FUNC PyInit_string_ops(void) {
    return PyModule_Create(&stringmodule);
}

Compile and Use

1. Create setup.py similar to the previous example.
2. Install and use the module: import string_ops print(string_ops.reverse("hello")) # Output: "olleh"

---

7. Working with NumPy Arrays in C Extensions

For numerical computing, we can create C extensions that process NumPy arrays efficiently.

Example: Multiply NumPy Array Elements by 2 (numpy_ops.c)

#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <numpy/arrayobject.h>

// Multiply array elements by 2
static PyObject* multiply_by_two(PyObject* self, PyObject* args) {
    PyObject* input_array;
    
    if (!PyArg_ParseTuple(args, "O!", &PyArray_Type, &input_array)) {
        return NULL;
    }

    PyArrayObject* np_array = (PyArrayObject*)input_array;
    int len = PyArray_SIZE(np_array);
    double* data = (double*)PyArray_DATA(np_array);

    for (int i = 0; i < len; i++) {
        data[i] *= 2;
    }

    Py_INCREF(input_array);
    return input_array;
}

// Define module methods
static PyMethodDef NumpyMethods[] = {
    {"multiply_by_two", multiply_by_two, METH_VARARGS, "Multiply elements of a NumPy array by 2"},
    {NULL, NULL, 0, NULL}
};

// Define module
static struct PyModuleDef numpymodule = {
    PyModuleDef_HEAD_INIT,
    "numpy_ops",
    NULL,
    -1,
    NumpyMethods
};

// Initialize module
PyMODINIT_FUNC PyInit_numpy_ops(void) {
    import_array(); // Required for NumPy API
    return PyModule_Create(&numpymodule);
}

Usage in Python

import numpy as np
import numpy_ops

arr = np.array([1.0, 2.0, 3.0])
result = numpy_ops.multiply_by_two(arr)

print(result)  # Output: [2.0, 4.0, 6.0]