About this book

Cython is a very powerful combination of Python and C. Using Cython, you can write Python code that calls back and forth from and to C or C++ code natively at any point. It is a language with extra syntax allowing for optional static type declarations. It is also a very popular language as it can be used for multicore programming.

Learning Cython Programming will provide you with a detailed guide to extending your native applications in pure Python; imagine embedding a twisted web server into your native application with pure Python code. You will also learn how to get your new applications up and running by reusing Python’s extensive libraries such as Logging and Config Parser to name a few.

With Learning Cython Programming, you will learn that writing your own Python module in C from scratch is not only hard, but is also unsafe. Cython will automatically handle all type-conversion issues as well as garbage collection on your code. You can also still write all your code in Python but have it compiled and called directly in C as if it was just another function or data.

This book also demonstrates how you can take the open source project Tmux and extend it to add new commands directly in pure Python. With this book, you will learn everything you need to know to get up and running with Cython and how you can reuse examples in a practical way.

Publication date:
September 2013
Publisher
Packt
Pages
110
ISBN
9781783280797

 

Chapter 1. Cython Won't Bite

Cython makes writing C extensions for Python as easy as Python itself. Its main use within the community is the Mathematics software package, SAGE, which is used to perform fast and scalable calculations. Most notably, it provides a safe and maintainable way of building native modules for Python via autogeneration of the required code.

Personally, I have used Cython to take control of legacy applications where the system has been implemented in C/C++ and in which adding functionality can become painful; we can then use it to generate bindings so that the native application and the Python one can work together! With this, you are able to perform high-level logic within Python but leverage the power of your native system.

 

What this book is


Python has become a great exception in software engineering in the last few years; it can be used in any way you can think of to create or extend software systems with low cost in regards to development time. We can also use it to extend software ranging from system-level distributed systems to high-level web applications.

This book will demonstrate how to gain more from Python. In case you're not aware, Python can be extended via native C/C++ code using extension modules over PyObject or by using C types. Doing this manually is generally not a good idea, as you really need to know how Python works internally. For example, you need to know about garbage collection so your Python objects don't get collected. But this is where Cython comes in; it will generate all of the C Python API wrapper code necessary and correctly.

 

What this book isn't


It's good to be clear that in this book, I will assume you have experience and knowledge of C and Python, but more importantly, you should be comfortable with the C compilation and linking process to create shared libraries and executables. This is important to get the most out of Cython because the examples seen on the Internet generally deal with very small single Cython file projects, and those aren't that helpful for most of us. I hope that after reading this book you will be comfortable with Cython. The online documentation will provide all the references you will need.

 

Installing Cython


Now let's get Cython installed. Think of Cython as a tool like Bison, flex, or GCC; it takes an input source and generates another that you compile and link:

  • Fedora – Fedora comes with the yum package manager. So, you can simply run yum install Cython.

  • Ubuntu/Debian – As with Fedora, Ubuntu has a package available via aptitude: apt-get install Cython.

  • Mac – Install Xcode and the command-line tools. Then, run the following:

      $ curl -O http://www.cython.org/release/Cython-0.18.tar.gz
    
      $ tar zxvf Cython-0.18.tar.gz
    
      $ cd Cython-0.18
      $ sudo python setup.py install
    
  • Windows – Although there are a plethora of options available, following this wiki is the safest option to stay up to date: http://wiki.cython.org/InstallingOnWindows.

The emacs mode

There is an emacs mode for Cython available, as the Python emacs mode doesn't work correctly. So, you can add the Tools/cython-mode.el mode to your ~/.emacs.d directory and then add require to your ~/.emacs file.

(add-to-list 'load-path "~/.emacs.d/")
(require 'cython-mode)

Getting the code examples

Throughout this book, I intend to show real examples that are easy to digest to help you get a feel of the different things you can achieve with Cython. To access and download the code used in these examples, visit GitHub at the following link:

$ git clone git://github.com/redbrain/cython-book.git
 

Hello World


Hopefully by now you've got Cython down and compiled and installed it. Let's check this by running the following command:

$ cython --version

Let's do a sanity test and run the typical "Hello World" program:

[email protected]:~/workspace/cython-book/chapter1/helloworld$ make

We have now created the Cython helloworld.so module! You can see it within Python (make sure you are in the same directory as the helloworld.so module):

[email protected]:~/workspace/cython-book/chapter1/helloworld$ python
Python 2.7.3 (default, Aug  1 2012, 05:16:07)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import helloworld
Hello World from cython!

We import helloworld since this module is now a valid Python module that can be loaded. And, on importing, we declared the Cython code to simply print our message. Not very exciting, but that's how the "Hello World" module is.

Let's look at what we did; Cython files have the extensions .pyx and .pxd. For now, all we will care about are the .pyx files. Later in this book, I will introduce the use of .pxd and what you can use this for. For purposes of clarity, it's good to understand the basic pipeline of what's going on to generate this helloworld.so module for Python. Cython works in the same way as any other code generator.

The flow depicted in the following figure illustrates how Cython works:

I wrote a basic Makefile so you can simply run make to compile these examples. It uses the setup.py style to let Python handle compiling and setting up these modules. Here's the code to do this manually:

$ cython helloworld.pyx
$ gcc -g -O2 -fpic `python-config --cflags` -c helloworld.c -o helloworld.o
$ gcc -shared -o helloworld.so helloworld.o `python-config --libs`

I feel this is a very important skill to learn with C development because you will start thinking of your code in terms of how you can share it more easily.

 

Module on your own


Now that you've seen the "Hello World" module, let's see how you can write your own module to do something! Then, you can link it against some of your own code. Later, we'll introduce the idea of wrapping your code.

Calling into your C code

Cython is a superset of Python. Although the syntax and keywords will work in the same way, we should be careful when talking about Python and Cython for clarity. To see Cython in action, let's build a hello-world-style module but perform something basic, just to be sure we are on the same page.

Open a file called mycode.c and insert the following code in to it:

#include <stdio.h>

int myfunc (int a, int b)
{
  printf ("look we are within your c code!!\n");
  return a + b;
}

This is the C code we will call—just a simple function to add two integers you've probably seen before. Now let's get Python to call it. Open a file called mycode.h, wherein we will declare our prototypes for Cython as follows:

#ifndef __MYCODE_H__
#define __MYCODE_H__
extern int myfunc (int, int);
#endif //__MYCODE_H__

We need this so that Cython can see the prototype for the function we want to call. In practice, you will already have your headers in your own project with your prototypes and declarations.

Open a file called mycodecpy.pyx and insert the following code in to it:

cdef extern from "mycode.h":
    cdef int myfunc (int, int)

def callCfunc ():
    print myfunc (1,2)

Within this Cython code, we initially have to declare what C code we care about. cdef is a keyword signifying that this is from the C code that will be linked in. Now that we have declared the header with the prototype to squash any undeclared function warnings from our compiler, we can make a wrapper function. At this point, we will specify how Python will call this native code, since calling directly into C code is dangerous. Therefore, Cython handles all type-conversion problems for us. A basic wrapper function, callCfunc, is all we need—it calls the myfunc function and passes the integers 1 and 2; then it simply prints the result.

To compile this, use the following:

$ cython mycodecpy.pyx
$ gcc -g -O2 -fpic -c mycode.c -o mycode.o
$ gcc -g -O2 -fpic -c mycodecpy.c -o mycodecpy `python-config --cflags`
$ gcc -shared -o mycodecpy.so mycode.o mycodecpy.o `python-config --libs`

We have to remember to link in the code that has the C function; in this case, mycode.c. If you're not familiar with what I mean here, you may need to revisit some C tutorials on compilation, as every C file is compiled to an object file and then we link all object files into a binary. Therefore, you need to be sure you link in all necessary object files.

[email protected]:~/workspace/cython-book/chapter1/ownmodule$ python
Python 2.7.3 (default, Aug  1 2012, 05:16:07)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from mycodepy import callCfunc
>>> callCfunc ()
look we are within your c code!!
3

So, we have now compiled and called our native code from Python code. I hope this gives you an idea of what you can do with it. With this, you can have a native module linking against some native libraries, making bindings to such libraries very simple.

Type conversion

You might have noticed that we called a Cython function directly with no arguments, which in turn called our C (the cdef prototype) function with two integer arguments. What if we wanted the Cython code to handle arguments? We could execute the following:

 def callCfunc2 (int x, int y):
    print myfunc (x, y)

We have now added the int arguments to the Python wrapper function we defined. This will require Python code to be type-safe and to convert PyObjects to C types for us automatically. When you create an integer Python object, the type is not integer, it's PyObject. If you want to use this in C, you need to get the data via the Python C API, but Cython will do this for us automatically. For example, if you pass illegal arguments, you will get the following:

>>> import mycodepy
>>> mycodepy.callCfunc2 (1, 'string')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "mycodepy.pyx", line 7, in mycodepy.callCfunc2 (mycodepy.c:733)
    def callCfunc2 (int x, int y):
TypeError: an integer is required
>>>

Even if you simply add more type safety to your Python code via the use of C types from Cython, you will find that you gain a bit of speed and some nice code. This is because the Cython compiler can optimize much more aggressively to avoid using Python calls.

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

 

Summary


Right, so I know this isn't the most exciting thing you've ever done, but if you could just take a step back and consider this: pure Python code calling into C code directly! Think what you could do with it—it's pretty exciting! In the next chapter, I will show you more on cdef and the keywords around it and how to share your Cython functions so that they are callable from C code. With this, anything can call anything! Not only that, we can look at how we compile our normal Python code and tell it to use some C types to try and get some more efficiency by looking into more syntax and more ways to use Cython.

About the Author

  • Philip Herron

    Philip Herron is a developer who focuses his passion toward compilers and virtual machine implementations. When he was first accepted to Google Summer of Code 2010, he used inspiration from Paul Biggar's PhD on the optimization of dynamic languages to develop a proof of the concept GCC frontend to compile Python. This project sparked his deep interest in how Python works.

    After completing a consecutive year on the same project in 2011, Philip applied to Cython under the Python foundation to gain a deeper appreciation of the standard Python implementation. Through this he started leveraging the advantages of Python to control the logic in systems or even add more high-level interfaces, such as embedding Flask web servers in a REST API to a system-level piece of software, without writing any C code.

    Philip currently works as a software consultant for Instil Software based in Northern Ireland. He develops mobile applications with embedded native code for video streaming. Instil has given him a lot of support in becoming a better engineer.

    He has written several tutorials for the UK-based Linux Format magazine on Python and loves to share his passion for the Python programming language.

    Browse publications by this author
Book Title
Access this book, plus 7,500 other titles for FREE
Access now