# Up and Running with MXNet

**MXNet** is one of the most used deep learning frameworks and is an Apache open source project. Before 2016, **Amazon Web Services** (**AWS**)’s research efforts did not use a preferred deep learning framework, allowing each team to research and develop according to their choices. Although some deep learning frameworks have thriving communities, sometimes AWS was not able to fix code bugs at the required speed (among other issues). To solve these issues, at the end of 2016, AWS announced MXNet as its deep learning framework of choice, investing in internal teams to develop it further. Research institutions that support MXNet are Intel, Baidu, Microsoft, Carnegie Mellon University, and MIT, among others. It was co-developed by Carlos Guestrin at Carnegie Mellon University and the University of Washington (along with GraphLab).

Some of its advantages are as follows:

- Imperative/symbolic programming and hybridization (which will be covered in
*Chapters 1*and*9*) - Support for multiple GPUs and distributed training (which will be covered in
*Chapters 7*and*8*) - Highly optimized for inference production systems (which will be covered in
*Chapters 7*and*9*) - A large number of pre-trained models on its Model Zoos in the fields of computer vision and natural language processing, among others (covered in
*Chapters 6*,*7*, and*8*)

To start working with MXNet, we need to install the library. There are several different versions of MXNet available to be installed, and in this chapter, we will cover how to choose the right version. The most important parameter will be the available hardware we have. In order to optimize performance, it is always best to maximize the use of our available hardware. We will compare the usage of a well-known linear algebra library, NumPy, with similar operations in MXNet. We will then compare the performance of the different MXNet versions versus NumPy.

MXNet includes its own API for deep learning, Gluon, and moreover, Gluon provides different libraries for computer vision and natural language processing that include pre-trained models and utilities. These libraries are known as GluonCV and GluonNLP.

In this chapter, we will cover the following topics:

- Installing MXNet, Gluon, GluonCV, and GluonNLP
- NumPy and MXNet ND arrays – comparing their performance

# Technical requirements

Apart from the technical requirements specified in the *Preface*, no other requirements apply to this chapter.

The code for this chapter can be found at the following GitHub URL: https://github.com/PacktPublishing/Deep-Learning-with-MXNet-Cookbook/tree/main/ch01.

Furthermore, you can access directly each recipe from Google Colab – for example, use the following for the first recipe of this chapter: https://colab.research.google.com/github/PacktPublishing/Deep-Learning-with-MXNet-Cookbook/blob/main/ch01/1_1_Installing_MXNet.ipynb.

# Installing MXNet, Gluon, GluonCV, and GluonNLP

In order to get the maximum performance out of the available software (programming languages) and hardware (CPU and GPU), there are different MXNet library versions available to install. We shall learn how to install them in this recipe.

## Getting ready

Before getting started with the MXNet installation, let us review the different versions available of the software packages that we will use, including MXNet. The reason we do that is that our hardware configuration must map to the chosen versions of our software packages in order to maximize performance:

**Python**: MXNet is available for different programming languages – Python, Java, R, and C++, among others. We will use MXNet for Python, and Python 3.7+ is recommended.**Jupyter**: Jupyter is an open source web application that provides an easy-to-use interface to show Markdown text, working code, and data visualizations. It is very useful for understanding deep learning, as we can describe concepts, write the code to run through those concepts, and visualize the results (typically comparing them with the input data). Jupyter Core 4.5+ is recommended.**CPUs and GPUs**: MXNet can work with any hardware configuration – that is, any single CPU can run MXNet. However, there are several hardware components that MXNet can leverage to improve performance:**Intel CPUs**: Intel developed a library known as**Math Kernel Library**(**MKL**) for optimized math operations. MXNet has support for this library, and using the optimized version can improve certain operations. Any modern version of Intel MKL is sufficient.**NVIDIA GPUs**: NVIDIA developed a library known as**Compute Unified Device Architecture (CUDA)**for optimized parallel operations (such as matrix operations, which are very common in deep learning). MXNet has support for this library, and using the optimized version can dramatically improve large deep learning workloads, such as model training. CUDA 11.0+ is recommended.

**MXNet version**: At the time of writing, MXNet 1.9.1 is the most up-to-date stable version that has been released. All the code throughout the book has been verified with this version. MXNet, and deep learning in general, can be considered a live ongoing project, and therefore, new versions will be released periodically. These new versions will have improved functionality and new features, but they might also contain breaking changes from previous APIs. If you are revisiting this book in a few months and a new version has been released with breaking changes, how to install MXNet version 1.8.0 specifically is also described here.

Tip

I have used Google Colab as the platform to run the code described in this book. At the time of writing, it provides Python 3.10.12, up-to-date Jupyter libraries, Intel CPUs (Xeon @ 2.3 GHz), and NVIDIA GPUs (which can vary: K80s, T4s, P4s, and P100s) with CUDA 11.8 pre-installed. Therefore, minimal steps are required to install MXNet and get it running.

## How to do it...

Throughout the book, we will not only use code extensively but also clarify comments and headings in that code to provide structure, as well as several types of visual information such as images or generated graphs. For these reasons, we will use Jupyter as the supporting development environment. Moreover, in order to facilitate setup, installation, and experimentation, we will use Google Colab.

Google Colab is a hosted Jupyter Notebook service that requires no setup to use, while providing free access to computing resources, including GPUs. In order to set up Google Colab properly, this section is divided into two main points:

- Setting up the notebook
- Verifying and installing libraries

Important note

If you prefer, you can use any local environment that supports Python 3.7+, such as Anaconda, or any other Python distribution. This is highly encouraged if your hardware specifications are better than Google Colab’s offering, as better hardware will reduce computation time.

### Setting up the notebook

In this section, we will learn how to work with Google Colab and set up a new notebook, which we will use to verify our MXNet installation:

- Open your favorite web browser. In my case, I have used Google Chrome as the web browser throughout the book. Visit https://colab.research.google.com/ and click on
**NEW NOTEBOOK**.

Figure 1.1 – The Google Colab start screen

- Change the title of the notebook – for example, as you can see in the following screenshot, I have changed the title to
`DL with MXNet Cookbook 1.1`

`Installing MXNet`

.

Figure 1.2 – A Google Colab notebook

- Change your Google Colab runtime type to use a GPU:
- Select
**Change runtime type**from the**Runtime**menu.

- Select

Figure 1.3 – Change runtime type

- In
**Notebook settings**, select**GPU**as the**Hardware****accelerator**option.

Figure 1.4 – Hardware accelerator | GPU

### Verifying and installing libraries

In this section, go to the first cell (make sure it is a code cell) and type the following commands:

- Verify the Python version by typing the following:
import platform platform.python_version()

This will yield an output as follows:

3.7.10

Check the version, and make sure that it is 3.7+.

Important note

In Google Colab, you can directly run commands as if you were in the Linux Terminal by adding the `!`

character to the command. Feel free to try other commands such as `!ls`

.

- We now need to verify the Jupyter version (Jupyter Core 4.5.0 or above will suffice):
!jupyter --version

This is one potential output from the previous command:

jupyter core : 4.5.0 jupyter-notebook : 5.2.2 qtconsole : 4.5.2 ipython : 5.5.0 ipykernel : 4.10.1 jupyter client : 5.3.1 jupyter lab : not installed nbconvert : 5.5.0 ipywidgets : 7.5.0 nbformat : 4.4.0 traitlets : 4.3.2

Tip

Jupyter, an open source notebook application, is assumed to be installed, as is the case for Google Colab. For further instructions on how to install it, visit https://jupyter.org/install.

- Verify whether an Intel CPU is present in the hardware:
!lscpu | grep 'Model name'

This will yield a similar output to the following:

Model name: Intel(R) Xeon(R) CPU @ 2.20GHz

The more up to date the processor the better, but for the purposes of this book, the dependency is larger with the GPU than with the CPU.

- Verify the NVIDIA GPU is present in the hardware (there are devices listed below) and that NVIDIA CUDA is installed:
!nvidia-smi

This will yield a similar output to the following:

+-----------------------------------------------------------------+ | NVIDIA-SMI 460.67 Driver Version: 460.32.03 CUDA Version: 11.2 | |---------------------------+--------------+----------------------+ |GPU Name Persistence-M|Bus-Id Disp.A| Volatile Uncorr. ECC | |Fan Temp Perf Pwr:Usage/Cap| Memory-Usage| GPU-Util Compute M. | | | | MIG M. | |===========================+==============+======================| | 0 Tesla T4 Off |0:00:04.0 Off | 0 | | N/A 37C P8 9W / 70W |0MiB/15109MiB | 0% Default | | | N/A | +---------------------------+--------------+----------------------+ +-----------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | ID ID Usage | |=================================================================| | No running processes found | +-----------------------------------------------------------------+

Important note

CUDA 11.0 has known issues with the NVIDIA K80. If you have an NVIDIA K80 and are having issues with the examples described, uninstall CUDA 11.0 and install CUDA 10.2. Afterward, install MXNet for CUDA 10.2 following the steps described here.

- Verify that the CUDA version is 11.0 or above:
!nvcc --version

This will yield a similar output to the following:

nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2020 NVIDIA Corporation Built on Wed_Jul_22_19:09:09_PDT_2020 Cuda compilation tools, release 11.0, V11.0.221 Build cuda_11.0_bu.TC445_37.28845127_0

- Install MXNet, depending on your hardware configuration. The following are the different MXNet versions that you can install:
**Recommended/Google Colab**: The latest MXNet version (1.9.1) with GPU support:!python3 -m pip install mxnet-cu117

**No Intel CPU nor NVIDIA GPU**: Install MXNet with the following command:!python3 -m pip install mxnet

**Intel CPU without NVIDIA GPU**: Install MXNet with Intel MKL, with the following command:!python3 -m pip install mxnet-mkl

**No Intel CPU with NVIDIA GPU**: Install MXNet with NVIDIA CUDA 10.2, with the following command:!python3 -m pip install mxnet-cu102

**Intel CPU and NVIDIA GPU**: Install MXNet with Intel MKL and NVIDIA CUDA 11.0, with the following command:!python3 -m pip install mxnet-cu110

Tip

`pip3`

, a Python 3 package manager, is assumed to be installed, as is the case for Google Colab. If a different installation method for MXNet is preferred, visit https://mxnet.apache.org/versions/master/get_started for instructions.

After version 1.6.0, MXNet is released by default with the Intel MKL library extension; therefore, there is no need to add the `mkl`

suffix anymore when installing the most recent versions, as seen previously in the recommended installation.

- Verify that the MXNet installation has been successful with the following two steps:
- The following commands must not return any error and must successfully display MXNet version 1.9.1:

import mxnet mxnet.__version__

features = mxnet.runtime.Features() print(features) print(features.is_enabled('CUDA')) print(features.is_enabled('CUDNN')) print(features.is_enabled('MKLDNN'))

The output will list all the features and

`True`

for each one. - Install GluonCV and GluonNLP:
!python3 -m pip install gluoncv gluonnlp

This command will install the latest versions of GluonCV and GluonNLP, which at the time of writing were, respectively, 0.10 and 0.10.

## How it works...

The training, inference, and evaluation of deep learning networks are highly complex operations, involving hardware and several layers of software, including drivers, low-level performance libraries such as MKL and CUDA, and high-level programming languages and libraries such as Python and MXNet.

Important note

MXNet is an actively developed project, part of the Apache Incubator program. Therefore, new versions are expected to be released, and they might contain breaking changes. The preceding command will install the latest stable version available. Throughout this book, the version of MXNet used is 1.9.1. If your code fails and it uses a different MXNet version, try installing MXNet version 1.9.1 by running the following:

`!python3 -m pip `

`install mxnet-cu117==1.9.1`

By checking all the hardware and software components, we can install the most optimized version of MXNet. We can use Google Colab, which easily transfers to other local configurations such as the Anaconda distribution.

Moreover, we can identify the right combination of CUDA drivers and MXNet versions that will maximize performance and verify a successful installation.

## There’s more…

It is highly recommended to always use the latest versions of all the software components discussed. Deep learning is an evolving field and there are always improvements such as new functionalities added, changes in the APIs, and updates in the internal functions to increase performance, among other changes.

However, it is very important that all components (CPU, GPU, CUDA, and the MXNet version) are compatible. To match these components, it is highly recommended to visit https://mxnet.apache.org/versions/master/get_started and check for the latest CUDA and MXNet versions you can install to maximize your hardware performance.

As an example, for a Python 3-based Linux distribution, installed using `pip3`

, these are the MXNet versions available (note with/without CPU acceleration and/or with GPU acceleration).

If you are interested in knowing more about Intel’s **MKL**, the following link is a very good starting point: https://software.intel.com/content/www/us/en/develop/articles/getting-started-with-intel-optimization-for-mxnet.html.

# NumPy and MXNet ND arrays

If you have worked with data previously in Python, chances are you have found yourself working with NumPy and its **N-dimensional arrays** (**ND arrays**). These are also known as tensors, and the 0D variants are called **scalars**, the 1D variants are called **vectors**, and the 2D variants are called **matrixes**.

MXNet provides its own ND array type, and there are two different ways to work with them. On one hand, there is the `nd`

module, MXNet’s native and optimized way to work with MXNet ND arrays. On the other hand, there is the `np`

module, which has the same interfaces and syntax as the NumPy ND array type and has also been optimized, but it’s limited due to the interface constraints. With MXNet ND arrays, we can leverage its underlying engine, with compute optimizations such as Intel MKL and/or NVIDIA CUDA, if our hardware configuration is compatible. This means we will be able to use almost the same syntax as when working with NumPy, but accelerated with the MXNet engine and our GPUs, not supported by NumPy.

Moreover, as we will see in the next chapters, a very common operation that we will execute on MXNet is automatic differentiation on these ND arrays. By using MXNet ND array libraries, this operation will also leverage our hardware for optimum performance. NumPy does not provide automatic differentiation out of the box.

## Getting ready

If you have already installed MXNet, as described in the previous recipe, in terms of executing accelerated code, the only remaining steps before using MXNet ND arrays is importing their libraries:

import numpy as np import mxnet as mx

However, it is worth noting here an important underlying difference between NumPy ND array operations and MXNet ND array operations. NumPy follows an eager evaluation strategy – that is, all operations are evaluated at the moment of execution. Conversely, MXNet uses a lazy evaluation strategy, more optimal for large compute loads, where the actual calculation is deferred until the values are actually needed.

Therefore, when comparing performances, we will need to force MXNet to finalize all calculations before computing the time needed for them. As we will see in the examples, this is achieved by calling the `wait_to_read()`

function, Furthermore, when accessing the data with functions such as `print()`

or `.asnumpy()`

, execution is then completed before calling these functions, yielding the wrong impression that these functions are actually time-consuming:

- Let’s check a specific example and start by running it on the CPU:
import time x_mx_cpu = mx.np.random.rand(1000, 1000, ctx = mx.cpu()) start_time = time.time() mx.np.dot(x_mx_cpu, x_mx_cpu).wait_to_read() print("Time of the operation: ", time.time() - start_time)

This will yield a similar output to the following:

Time of the operation: 0.04673886299133301

- However, let’s see what happens if we measure the time without the call to
`wait_to_read()`

:x_mx_cpu = mx.np.random.rand(1000, 1000, ctx = mx.cpu()) start_time = time.time() x_2 = mx.np.dot(x_mx_cpu, x_mx_cpu) print("(FAKE, MXNet has lazy evaluation)") print("Time of the operation : ", time.time() - start_time) start_time = time.time() print(x_2) print("(FAKE, MXNet has lazy evaluation)") print("Time to display: ", time.time() - start_time)

The following will be the output:

(FAKE, MXNet has lazy evaluation) Time of the operation : 0.00118255615234375 [[256.59583 249.70404 249.48639 ... 251.97151 255.06744 255.60669] [255.22629 251.69475 245.7591 ... 252.78784 253.18878 247.78052] [257.54187 254.29262 251.76346 ... 261.0468 268.49127 258.2312 ] ... [256.9957 253.9823 249.59073 ... 256.7088 261.14255 253.37457] [255.94278 248.73282 248.16641 ... 254.39209 252.4108 249.02774] [253.3464 254.55524 250.00716 ... 253.15712 258.53894 255.18658]] (FAKE, MXNet has lazy evaluation) Time to display: 0.042133331298828125

As we can see, the first experiment indicated that the computation took ~50 ms to complete; however, the second experiment indicated that the computation took ~1 ms (50 times less!), and the visualization was more than 40 ms. This is an incorrect result. This is because we measured our performance incorrectly in the second experiment. Refer to the first experiment and the call to `wait_to_read()`

for a proper performance measurement.

## How to do it...

In this section, we will compare performance in terms of computation time for two compute-intensive operations:

- Matrix creation
- Matrix multiplication

We will compare five different compute profiles for each operation:

- Using the NumPy library (no CPU or GPU acceleration)
- Using the MXNet
`np`

module with CPU acceleration but no GPU - Using the MXNet
`np`

module with CPU acceleration and GPU acceleration - Using the MXNet
`nd`

module with CPU acceleration but no GPU - Using the MXNet
`nd`

module with CPU acceleration and GPU acceleration

To finalize, we will plot the results and draw some conclusions.

### Timing data structures

We will store the computation time in five dictionaries, one for each compute profile (`timings_np`

, `timings_mx_cpu`

, and `timings_mx_gpu`

). The initialization of the data structures is as follows:

timings_np = {} timings_mx_np_cpu = {} timings_mx_np_gpu = {} timings_mx_nd_cpu = {} timings_mx_nd_gpu = {}

We will run each operation (matrix generation and matrix multiplication) with matrixes in a different order, namely the following:

matrix_orders = [1, 5, 10, 50, 100, 500, 1000, 5000, 10000]

### Matrix creation

We define three functions to generate matrixes; the first function will use the NumPy library to generate a matrix, and it will receive as an input parameter the matrix order. The second function will use the MXNet np module, and the third function will use the MXNet and module. For the second and third functions, as input parameters we will provide the context where the matrix needs to be created, apart from the matrix order. This context specifies whether the result (the created matrix in this case) must be computed in the CPU or the GPU (and which GPU if there are multiple devices available):

def create_matrix_np(n): """ Given n, creates a squared n x n matrix, with each matrix value taken from a random uniform distribution between [0, 1]. Returns the created matrix a. Uses NumPy. """ a = np.random.rand(n, n) return a def create_matrix_mx(n, ctx=mx.cpu()): """ Given n, creates a squared n x n matrix, with each matrix value taken from a random uniform distribution between [0, 1]. Returns the created matrix a. Uses MXNet NumPy syntax and context ctx """ a = mx.np.random.rand(n, n, ctx=ctx) a.wait_to_read() return a def create_matrix_mx_nd(n, ctx=mx.cpu()): """ Given n, creates a squared n x n matrix, with each matrix value taken from a random uniform distribution between [0, 1]. Returns the created matrix a. Uses MXNet ND native syntax and context ctx """ a = mx.nd.random.uniform(shape=(n, n), ctx=ctx) a.wait_to_read() return a

To store necessary data for our performance comparison later, we use the structures created previously, with the following code:

timings_np["create"] = [] for n in matrix_orders: result = %timeit -o create_matrix_np(n) timings_np["create"].append(result.best) timings_mx_np_cpu["create"] = [] for n in matrix_orders: result = %timeit -o create_matrix_mx_np(n) timings_mx_np_cpu["create"].append(result.best) timings_mx_np_gpu["create"] = [] ctx = mx.gpu() for n in matrix_orders: result = %timeit -o create_matrix_mx_np(n, ctx) timings_mx_np_gpu["create"].append(result.best) timings_mx_nd_cpu["create"] = [] for n in matrix_orders: result = %timeit -o create_matrix_mx_nd(n) timings_mx_nd_cpu["create"].append(result.best) timings_mx_nd_gpu["create"] = [] ctx = mx.gpu() for n in matrix_orders: result = %timeit -o create_matrix_mx_nd(n, ctx) timings_mx_nd_gpu["create"].append(result.best)

### Matrix multiplication

We define three functions to compute the matrixes multiplication; the first function will use the NumPy library and will receive as input parameters the matrixes to multiply. The second function will use the MXNet np module, and the third function will use the MXNet nd module. For the second and third functions, the same parameters are used. The context where the multiplication will happen is given by the context where the matrixes were created; no parameter needs to be added. Both matrixes need to have been created in the same context, or an error will be triggered:

def multiply_matrix_np(a, b): """ Multiplies 2 squared matrixes a and b and returns the result c. Uses NumPy. """ #c = np.matmul(a, b) c = np.dot(a, b) return c def multiply_matrix_mx_np(a, b): """ Multiplies 2 squared matrixes a and b and returns the result c. Uses MXNet NumPy syntax. """ c = mx.np.dot(a, b) c.wait_to_read() return c def multiply_matrix_mx_nd(a, b): """ Multiplies 2 squared matrixes a and b and returns the result c. Uses MXNet ND native syntax. """ c = mx.nd.dot(a, b) c.wait_to_read() return c

To store the necessary data for our performance comparison later, we will use the structures created previously, with the following code:

timings_np["multiply"] = [] for n in matrix_orders: a = create_matrix_np(n) b = create_matrix_np(n) result = %timeit -o multiply_matrix_np(a, b) timings_np["multiply"].append(result.best) timings_mx_np_cpu["multiply"] = [] for n in matrix_orders: a = create_matrix_mx_np(n) b = create_matrix_mx_np(n) result = %timeit -o multiply_matrix_mx_np(a, b) timings_mx_np_cpu["multiply"].append(result.best) timings_mx_np_gpu["multiply"] = [] ctx = mx.gpu() for n in matrix_orders: a = create_matrix_mx_np(n, ctx) b = create_matrix_mx_np(n, ctx) result = %timeit -o multiply_matrix_mx_np(a, b) timings_mx_gpu["multiply"].append(result.best) timings_mx_nd_cpu["multiply"] = [] for n in matrix_orders: a = create_matrix_mx_nd(n) b = create_matrix_mx_nd(n) result = %timeit -o multiply_matrix_mx_nd(a, b) timings_mx_nd_cpu["multiply"].append(result.best) timings_mx_nd_gpu["multiply"] = [] ctx = mx.gpu() for n in matrix_orders: a = create_matrix_mx_nd(n, ctx) b = create_matrix_mx_nd(n, ctx) result = %timeit -o multiply_matrix_mx_nd(a, b) timings_mx_nd_gpu["multiply"].append(result.best)

### Drawing conclusions

The first step before making any assessments is to plot the data we have captured in the previous steps. For this step, we will use the `pyplot`

module from a library called Matplotlib, which will allow us to create charts easily. The following code plots the runtime (in seconds) for the matrix generation and all the matrix orders computed:

import matplotlib.pyplot as plt fig = plt.figure() plt.plot(matrix_orders, timings_np["create"], color='red', marker='s') plt.plot(matrix_orders, timings_mx_np_cpu["create"], color='blue', marker='o') plt.plot(matrix_orders, timings_mx_np_gpu["create"], color='green', marker='^') plt.plot(matrix_orders, timings_mx_nd_cpu["create"], color='yellow', marker='p') plt.plot(matrix_orders, timings_mx_nd_gpu["create"], color='orange', marker='*') plt.title("Matrix Creation Runtime", fontsize=14) plt.xlabel("Matrix Order", fontsize=14) plt.ylabel("Runtime (s)", fontsize=14) plt.grid(True) ax = fig.gca() ax.set_xscale("log") ax.set_yscale("log") plt.legend(["NumPy", "MXNet NumPy (CPU)", "MXNet NumPy (GPU)", "MXNet ND (CPU)", "MXNet ND (GPU)"]) plt.show()

Quite similarly as shown in the previous code block, the following code plots the runtime (in seconds) for the matrix multiplication and all the matrix orders computed:

import matplotlib.pyplot as plt fig = plt.figure() plt.plot(matrix_orders, timings_np["multiply"], color='red', marker='s') plt.plot(matrix_orders, timings_mx_np_cpu["multiply"], color='blue', marker='o') plt.plot(matrix_orders, timings_mx_np_gpu["multiply"], color='green', marker='^') plt.plot(matrix_orders, timings_mx_nd_cpu["multiply"], color='yellow', marker='p') plt.plot(matrix_orders, timings_mx_nd_gpu["multiply"], color='orange', marker='*') plt.title("Matrix Multiplication Runtime", fontsize=14) plt.xlabel("Matrix Order", fontsize=14) plt.ylabel("Runtime (s)", fontsize=14) plt.grid(True) ax = fig.gca() ax.set_xscale("log") ax.set_yscale("log") plt.legend(["NumPy", "MXNet NumPy (CPU)", "MXNet NumPy (GPU)", "MXNet ND (CPU)", "MXNet ND (GPU)"]) plt.show()

These are the plots displayed (the results will vary according to the hardware configuration):

Figure 1.5 – Runtimes – a) Matrix creation, and b) Matrix multiplication

Important note

Note that the charts use a logarithmic scale for both axes, horizontal and vertical (the differences are larger than they seem). Furthermore, the actual values depend on the hardware architecture that the computations are run on; therefore, your specific results will vary.

There are several conclusions that can be drawn, both from each individual operation and collectively:

- For smaller matrix orders, using NumPy is much faster in both operations. This is because MXNet works in a different memory space, and the amount of time to move the data to this memory space is longer than the actual compute time.
- In matrix creation, for larger matrix orders, the difference between NumPy (remember, it’s CPU only) and MXNet with the np module and CPU acceleration is negligible, but with the nd module and CPU, acceleration is ~2x faster. For matrix multiplication, and depending on your hardware, MXNet with CPU acceleration can be ~2x faster (regardless of the module). This is because MXNet uses Intel MKL to optimize CPU computations.
- In the ranges that are interesting for deep learning – that is, large computational loads involving matrix orders > 1,000 (which can represent data such as images composed of several megapixels or large language dictionaries), GPUs deliver typical gains of several orders of magnitude (~200x for creation, and ~40x for multiplication, exponentially growing with every increase of matrix order). This is by far the most compelling reason to work with GPUs when running deep learning experiments.
- When using the GPU, the MXNet np module is faster than the MXNet nd module in creation (~7x), but the difference is negligible in multiplication. Typically, deep learning algorithms are more similar to multiplications to terms of computational loads, and therefore, a priori, there is no significant advantage in using the np module or the nd module. However,
*MXNet recommends using the native MXNet nd module*(and the author subscribes to this recommendation) because some operations on the np module are not supported by`autograd`

(MXNet’s auto-differentiation module). We will see in the upcoming chapters, when we train neural networks, how the`autograd`

module is used and why it is critical.

## How it works...

MXNet provides two optimized modules to work with ND arrays, including one that is an in-place substitute for NumPy. The advantages of operating with MXNet ND arrays are twofold:

- MXNet ND array operations support automatic differentiation. As we will see in the following chapters, automatic differentiation is a key feature that allows developers to concentrate on the forward pass of the models, letting the backward pass be automatically derived.
- Conversely, operations with MXNet ND arrays are optimized for the underlying hardware, yielding impressive results with GPU acceleration. We computed results for matrix creation and matrix multiplication to validate this conclusion experimentally.

## There’s more…

In this recipe, we have barely scratched the surface of MXNet operations with ND arrays. If you want to read more about MXNet and ND arrays, this is the link to the official MXNet API reference: https://mxnet.apache.org/versions/1.0.0/api/python/ndarray/ndarray.html.

Furthermore, a very interesting tutorial can be found in the official MXNet documentation: https://gluon.mxnet.io/chapter01_crashcourse/ndarray.html.

Moreover, we have taken a glimpse at how to measure performance on MXNet. We will revisit this topic in the following chapters; however, a good deep-dive into the topic is given in the official MXNet documentation: https://mxnet.apache.org/versions/1.8.0/api/python/docs/tutorials/performance/backend/profiler.html.