Creating a benchmark to measure testing performance

In step 3 of the previous list, we mentioned running a benchmark, which is a piece of code or an algorithm that measures something. In this case, we can measure the time that it takes to run each test on each platform and compare them. The performance measured might be a deciding factor for the platform to use but might not be the only one, as there are other important aspects in the previous list, in steps 1 and 2.

In our case, we will measure the time it takes to run each test and the time it takes to run the entire class. Some cloud providers add extra time to their system setup and teardown or cleanup, and you should be aware of these values too. Sometimes, there is also some extra time taken if you need to send the test for it to be uploaded to their system (as opposed to running it remotely).

Let us see an example of how to make a simple benchmark with two tests using Java and TestNG (don’t mind the code repetition...