Benchmarking

ALP has a specialised class for benchmarking ALP programs, grb::Benchmarker, which is a variant on the grb::Launcher. More...

Classes
class	Benchmarker< mode, implementation >
	A class that follows the API of the grb::Launcher, but instead of launching the given ALP program once, it launches it multiple times while benchmarking its execution times. More...

Detailed Description

ALP has a specialised class for benchmarking ALP programs, grb::Benchmarker, which is a variant on the grb::Launcher.

It codes a particular benchmarking strategy of any given ALP program as described below.

The program is called inner times outer times. Between every inner repetitions there is a one-second sleep that ensures machine variability is taken into account. Several statistics are measured across the outer repetitions: the minimum, maximum, average, and the (unbiased) sample standard deviation. By contrast, for the inner repetitions, only an average is computed – the function of inner repetitions is solely to avoid timing programs that execute in too short a time frame, meaning a time frame that is of a similar order as the time it takes to actually call the system timer functionalities.

Note

As a result, inner should always equal one when benchmarking any non-trivial ALP program, while for benchmarking ALP kernels on small data inner may be taken (much) larger.

In published experiments, inner is chosen such that a single outer repetition takes 10 to 100 milliseconds.