Hands-on deployment and inference testing (a business use case)

When you have your service (either API or ML) ready and you are about to serve it to the users but you don't have any clue about how many users it can actually handle and how it will react when many users access it simultaneously, that's where load testing is useful to benchmark how many users your service can serve and to validate whether the service can cater to the business requirements.

We will perform load testing for the service we deployed previously (in Chapter 7, Building Robust CI and CD Pipelines). Locust.io will be used for load testing. locust.io is an open source load-testing tool. For this, we will install locust (using pip) and curate a Python script using the locust.io SDK to test an endpoint. Let's get started by installing locust:

1. Install locust: Go to your terminal and execute the following command:

   pip install locust

   Using pip, locust will be installed – it takes around...