In this work, an automatic optimisation approach for parallel routines on multi-GPU
systems is presented. Several inter-GPU communication libraries (such as CUDA-
Aware MPI or NCCL) are used with a set of routines to perform the numerical oper-
ations among the GPUs located on the compute nodes. The main objective is the
selection of the most appropriate communication library, the number of GPUs to be
used and the workload to be distributed among them in order to reduce the cost of
data movements, which represent a large percentage of the total execution time. To
this end, a hierarchical modelling of the execution time of each routine to be opti-
mised is proposed, combining experimental and theoretical approaches. The results
show that near-optimal decisions are taken in all the scenarios analysed.
La licencia del ítem se describe como Atribución 4.0 Internacional