Advanced Numpy

Many libraries have nice and easy to use APIs. All you need to do is to invoke the provided API function and library will handle the rest of it for you. What's happening under the hood is not your concern and you are only interested in output. This is fine for most cases, however it's important to understand at least the basic internal of a library you are using. Understanding internals will help you to grasp what's going on with your code and what are the red flags that you should avoid while developing your applications.

In this chapter, NumPy internals will be reviewed such as NumPy's type hierarchy and memory usage. At the end of this chapter, you will learn about the code profiling to inspect your programs line by line.

NumPy internals

As you have seen in previous chapters, NumPy arrays make numerical computations efficient and its API is intuitive and easy to use. NumPy array are also core to other scientific libraries as many of them are built on top of NumPy arrays.

In order to write better and more efficient code, you need to understand the internals of data handling. A NumPy array and its metadata live in a data buffer, which is a dedicated block of memory with certain data items.

How does NumPy manage memory?

Once you initialize a NumPy array, its metadata and data are stored at allocated memory locations in Random Access Memory (RAM).

import numpy as np
array_x = np.array([100.12, 120.23, 130.91])

First, Python is a dynamically typed...

Summary

Knowing NumPy's internals is crucially important when you are working with scientific operations. Efficiency is key since many scientific computations are compute and memory intensive. Hence, if your code is not written efficiently, computations will take much longer than they need and this will hurt your research and development timeline.

In this chapter, you have seen some of the internals and performance aspects of the NumPy library and also learned about the vprof library, which helps you inspect the performance of your python programs.

Code profiling will help you a lot to inspect your programs line by line and there are different ways of looking at the same data, as you have seen previously. Once you have identified the most demanding parts of your programs, then you can start searching for more efficient ways or implementations to improve performance and save...