Chapter 4. NumPy Core and Libs Submodules

After covering so many NumPy ufuncs in the previous chapter, I hope you still remember the very core of NumPy, which is the ndarray object. We are going to finish the last important attribute of ndarray: strides, which will give you the full picture of memory layout. Also, it's time to show you that NumPy arrays can deal not only with numbers but also with various types of data; we will talk about record arrays and date time arrays. Lastly, we will show how to read/write NumPy arrays from/to files, and start to do some real-world analysis using NumPy.

The topics that will be covered in this chapter are:

• The core of NumPy arrays: memory layout
• Structure arrays (record arrays)
• Date-time in NumPy arrays
• File I/O in NumPy arrays

Introducing strides

Strides are the indexing scheme in NumPy arrays, and indicate the number of bytes to jump to find the next element. We all know the performance improvements of NumPy come from a homogeneous multidimensional array object with fixed-size items, the numpy.ndarray object. We've talked about the shape (dimension) of the ndarray object, the data type, and the order (the C-style row-major indexing arrays and the Fortran style column-major arrays.) Now it's time to take a closer look at strides.

Let's start by creating a NumPy array and changing its shape to see the differences in the strides.

1. Create a NumPy array and take a look at the strides:

         In [1]: import numpy as np
         In [2]: x = np.arange(8, dtype = np.int8)
         In [3]: x
         Out[3]: array([0, 1, 2, 3, 4, 5, 6, 7])
         In [4]: x.strides
         Out[4]: (1,)
         In [5]: str(x.data)
         Out[5]: '\x00\x01\x02\x03\x04\x05\x06\x07'

   A one-dimensional array x is created and its...

Structured arrays

Structured arrays or record arrays are useful when you perform computations, and at the same time you could keep closely related data together. For example, when you process incident data and each incident contains geographic coordinates and the occurrence time, while you calculate the final result, you can easily find the associated geographic locations and timepoint for further visualization. NumPy also provides powerful capabilities to create arrays of records, as multiple data types live in one NumPy array. However, one principle in NumPy that still needs to be honored is that the data type in each field (you can think of this as a column in the records) needs to be homogeneous. Here are some simple examples that show you how it works:

In [20]: x = np.empty((2,), dtype = ('i4,f4,a10')) 
In [21]: x[:] = [(1,0.5, 'NumPy'), (10,-0.5, 'Essential')] 
In [22]: x 
Out[22]: 
array([(1, 0.5, 'NumPy'), (10, -0.5, 'Essential')], 
      dtype=[('f0', '<...

Summary

In this chapter, we covered the last important component of the ndarray object: strides. We saw a huge difference in memory layouts and also in performance when you use different ways to initialize your NumPy array. We also got to know the record array (structured array) and how to manipulate the date/time in NumPy. Most importantly, we saw how to read and write our data with NumPy.

NumPy is powerful not only because of its performance or ufuncs, but also because of how easy it can make your analysis. Use NumPy with your data as much as you can!

Next, we will look at linear algebra and matrix computation using NumPy.