Wrapping with your own code

A Python extension is a module written in C or C++ that can be loaded into Python like a regular module. It provides a bridge between Python’s friendly scripting environment and lower-level languages. We can create an extension for anything that would require the use of a lower-level language, including when we want to interface with an existing C++ library and offer access to GPU functionality while still benefiting from Python’s ease of use. A good example is PyTorch, which has a C++ backend and also CUDA support while offering a higher level API through Python.

Creating an extension requires the Python C API, which lets us define new types, expose C++ functions to Python, manage reference counts, and handle Python objects within C or C++ code. The general workflow consists of defining wrapper functions that translate between Python objects and native C++ structures, compiling them into a shared library and then loading them as Python...