Testing Setup

Table of Contents

• Unit Tests
  • CI Pipeline
  • Nocompile
• Integration Tests

Unit Tests

CI Pipeline

Tests are executed automatically on each commit by using Github Actions. Two workflows are defined: one for non-mpi (serial) compilation and one for mpi (parallel). The two workflows are otherwise identical. Each CI workflow define a matrix on compilation flags (number of colors, representation, etc).

The script run_tests.sh automate the process of: creating the appropriate MkFlags; compiling the tests; and running all the test. Run:

./run_tests.sh --help

for a list of available options and their default values. For example:

./run_tests.sh -mpi -n 2 -r FUND

will write a new MkFlags file for 2 colors and fermions in the fundamental representation (for all other required flags default values will be used).

Tests on Github Actions are executed inside a Docker container. The Dockerfile for the container is in the TestProgram directory, and the image is hosted at docker://cpica/hr-tests.

To reproduce the execution in the docker container locally, run:

export GITHUB_WORKSPACE=/github/workspace
docker run --workdir /github/workspace -e GITHUB_WORKSPACE -v $(pwd)/HiRep:/github/workspace cpica/hr-tests ./TestProgram/run_tests.sh -mpi -n 2 -r FUND

e.g. for the case of 2 gauge colors and fermions in the fundamental representation.

Github Actions workflows are defined in the .github/ folder at the root of the HiRep repository.

Detailed debugging output can be found in the outfile (default out_0) in the respective directory.

Nocompile

The test suite has to be able to test a number of different setups generated by different choices in Make/MkFlags. Some of these choices, for example compiling with or without GPU, will also generate different namespaces. However, we still want to preserve the ability to test only for a single case of choosing the compilation flags.

The testing Makefiles have therefore been configured to only run certain tests for certain compilation flags, using a NOCOMPILE = XXX statement, where XXX is the corresponding compilation flag. If we, for example, want to write a test that works only if compiles without GPU acceleration, we can use this flag to configure the test this way in the preamble

/*******************************************************************************
*
* NOCOMPILE= WITH_GPU
*
* This test is only compiled if the WITH_GPU flag is inactive
*
*******************************************************************************/

Notice, that there is no space between NOCOMPILE and =. Conversely, we can configure tests that only test the GPU-version by negation

/*******************************************************************************
*
* NOCOMPILE= !WITH_GPU
*
* This tests only the GPU-version of HiRep
*
*******************************************************************************/

If we want to make sure that multiple flags are active, we can connect them over &&.

/*******************************************************************************
*
* NOCOMPILE= !WITH_GPU && !WITH_MPI
*
* This tests only the Multi-GPU compiled version of HiRep
*
*******************************************************************************/

If the test does test multiple, but not all possible setups, we can write them down using multiple lines.

/*******************************************************************************
*
* NOCOMPILE= WITH_GPU
* NOCOMPILE= WITH_MPI
*
* This test can be compiled either if MPI or GPU acceleration is disabled.
*
*******************************************************************************/

Integration Tests

The default input files for the scripts GaugeFix, HMC, ModeNumber, PureGauge, RenormalizationFactors, Reweight, Scattering, Spectrum, StaticPotential and WilsonFlow constitute good integration tests.