Exporting and Importing Models

In this chapter, we will discuss how to save and load model parameters during and after training. This is quite an important issue since real model training can take a very long time (from days to weeks), and we want to be able to save intermediate results and then load them for use in evaluation mode in production.

Such regular save operations can be beneficial in the case of a random application crash. Another substantial feature of any machine learning (ML) framework is its ability to export the model architecture, which allows us to share models between frameworks and makes model deployment easier. The main topic of this chapter is to show how to export and import model parameters such as weights and bias values with different C++ libraries. The second part of this chapter is all about the Open Neural Network Exchange (ONNX) format, which is...

Technical requirements

The following are the technical requirements for this chapter:

• Dlib library
• Shark-ML library
  
• Shogun library
• PyTorch library
• A modern C++ compiler with C++17 support
• CMake build system version >= 3.8

The code files for this chapter can be found at the following GitHub repo: https://github.com/PacktPublishing/Hands-On-Machine-Learning-with-CPP/tree/master/Chapter12

ML model serialization APIs in C++ libraries

In this section, we will discuss the ML model sharing APIs in the Dlib, Shogun, Shark-ML, and PyTorch libraries. There are three main types of sharing ML models among the different C++ libraries:

• Share model parameters (weights)
• Share the entire model's architecture
• Share both the model architecture and its trained parameters

In the following sections, we will look at what API is available in each library and emphasize what type of sharing it supports.

Model serialization with Dlib

The Dlib library uses the serialization API for decision_function and neural network type objects. Let's learn how to use it by implementing a real example.

First, we define the types for...

Delving into ONNX format

ONNX format is a special file format used to share neural network architectures and parameters between different frameworks. It is based on the Google Protobuf format and library. The reason why this format exists is to test and run the same neural network model in different environments and on different devices. Usually, researchers use a programming framework that they know how to use in order to develop a model, and then run this model in a different environment for production purposes or if they want to share their model with other researchers or developers. This format is supported by all leading frameworks, such as PyTorch, TensorFlow, MXNet, and others. But now, there is a lack of support for this format from the C++ API of these frameworks and at the time of writing, they only have a Python interface for dealing with ONNX format. Some time ago...

Summary

In this chapter, we learned how to save and load model parameters in different ML frameworks. We saw that all the frameworks we used in the Shogun, Shark-ML, Dlib, and PyTorch libraries have an API for model parameter serialization. Usually, these are quite simple functions that work with model objects and some input and output streams. Also, we discussed another type of serialization API that can be used to save and load the overall model architecture. At the time of writing, the frameworks we used don't fully support such functionality. The Shogun toolkit can load neural network architectures from the JSON descriptions, but can't export them. The Dlib library can export neural networks in XML format but can't load them. The PyTorch C++ API lacks a model architecture that supports exporting, but it can load and evaluate model architectures that have been...

Further reading

• Shark-ML documentation: http://www.shark-ml.org/sphinx_pages/build/html/rest_sources/tutorials/tutorials.html
• Dlib documentation: http://dlib.net/
• Shogun toolkit documentation: https://www.shogun-toolbox.org/
• PyTorch C++ API: https://pytorch.org/cppdocs/
• ONNX official page: https://onnx.ai/
• ONNX Model Zoo: https://github.com/onnx/models
• ONNX ResNet models for image classification: https://github.com/onnx/models/tree/master/vision/classification/resnet
• Caffe2 tutorials: https://github.com/leonardvandriel/caffe2_cpp_tutorial/