3.9. Development build options (CMake only)

The CMake build procedure of LAMMPS offers a few extra options which are useful during development, testing or debugging.


3.9.1. Monitor compilation flags

Sometimes it is necessary to verify the complete sequence of compilation flags generated by the CMake build. To enable a more verbose output during compilation you can use the following option.

-D CMAKE_VERBOSE_MAKEFILE=value    # value = no (default) or yes

Another way of doing this without reconfiguration is calling make with variable VERBOSE set to 1:

make VERBOSE=1

3.9.2. Address, Undefined Behavior, and Thread Sanitizer Support

Compilers such as GCC and Clang support generating instrumented binaries which use different sanitizer libraries to detect problems in the code during run-time. They can detect issues like:

Please note that this kind of instrumentation usually comes with a performance hit (but much less than using tools like Valgrind with a more low level approach). The to enable these features additional compiler flags need to be added to the compilation and linking stages. This is done through setting the ENABLE_SANITIZER variable during configuration. Examples:

-D ENABLE_SANITIZER=none       # no sanitizer active (default)
-D ENABLE_SANITIZER=address    # enable address sanitizer / memory leak checker
-D ENABLE_SANITIZER=leak       # enable memory leak checker (only)
-D ENABLE_SANITIZER=undefined  # enable undefined behavior sanitizer
-D ENABLE_SANITIZER=thread     # enable thread sanitizer

3.9.3. Code Coverage and Unit Testing

The LAMMPS code is subject to multiple levels of automated testing during development: integration testing (i.e. whether the code compiles on various platforms and with a variety of settings), unit testing (i.e. whether certain individual parts of the code produce the expected results for given inputs), run testing (whether selected complete input decks run without crashing for multiple configurations), and regression testing (i.e. whether selected input examples reproduce the same results over a given number of steps and operations within a given error margin). The status of this automated testing can be viewed on https://ci.lammps.org.

The unit testing facility is integrated into the CMake build process of the LAMMPS source code distribution itself. It can be enabled by setting -D ENABLE_TESTING=on during the CMake configuration step. It requires the YAML library and development headers to compile and will download and compile a recent version of the Googletest C++ test framework for implementing the tests.

After compilation is complete, the unit testing is started in the build folder using the ctest command, which is part of the CMake software. The output of this command will be looking something like this:

  [...]$ ctest
  Test project /home/akohlmey/compile/lammps/build-testing
        Start  1: MolPairStyle:hybrid-overlay
  1/109 Test  #1: MolPairStyle:hybrid-overlay .........   Passed    0.02 sec
        Start  2: MolPairStyle:hybrid
  2/109 Test  #2: MolPairStyle:hybrid .................   Passed    0.01 sec
        Start  3: MolPairStyle:lj_class2
   [...]
        Start 107: PotentialFileReader
107/109 Test #107: PotentialFileReader ................   Passed    0.04 sec
        Start 108: EIMPotentialFileReader
108/109 Test #108: EIMPotentialFileReader .............   Passed    0.03 sec
        Start 109: TestSimpleCommands
109/109 Test #109: TestSimpleCommands .................   Passed    0.02 sec

  100% tests passed, 0 tests failed out of 26

  Total Test time (real) =  25.57 sec

The ctest command has many options, the most important ones are:

Option

Function

-V

verbose output: display output of individual test runs

-j <num>

parallel run: run <num> tests in parallel

-R <regex>

run subset of tests matching the regular expression <regex>

-E <regex>

exclude subset of tests matching the regular expression <regex>

-N

dry-run: display list of tests without running them

-T memcheck

run tests with valgrind memory checker (if available)

In its full implementation, the unit test framework will consist of multiple kinds of tests implemented in different programming languages (C++, C, Python, Fortran) and testing different aspects of the LAMMPS software and its features. At the moment only tests for “force styles” are implemented. More on those in the next section.

Note

The unit test framework is new and still under development. The coverage is only minimal and will be expanded over time. Tests styles of the same kind of style (e.g. pair styles or bond styles) are performed with the same executable using different input files in YAML format. So to add a test for another pair style can be done by copying the YAML file and editing the style settings and then running the individual test program with a flag to update the computed reference data. Detailed documentation about how to add new test program and the contents of the YAML files for existing test programs will be provided in time as well.

Unit tests for force styles

A large part of LAMMPS are different “styles” for computing non-bonded and bonded interactions selected through the pair_style command, bond_style command, angle_style command, dihedral_style command, improper_style command, and kspace_style command. Since these all share common interfaces, it is possible to write generic test programs that will call those common interfaces for small test systems with less than 100 atoms and compare the results with pre-recorded reference results. A test run is then a a collection multiple individual test runs each with many comparisons to reference results based on template input files, individual command settings, relative error margins, and reference data stored in a YAML format file with .yaml suffix. Currently the programs pair_style, bond_style, and angle_style are implemented. They will compare forces, energies and (global) stress for all atoms after a run 0 calculation and after a few steps of MD with fix nve, each in multiple variants with different settings and also for multiple accelerated styles. If a prerequisite style or package is missing, the individual tests are skipped. All tests will be executed on a single MPI process, so using the CMake option -D BUILD_MPI=off can significantly speed up testing, since this will skip the MPI initialization for each test run. Below is an example command and output:

[tests]$ pair_style mol-pair-lj_cut.yaml
[==========] Running 6 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 6 tests from PairStyle
[ RUN      ] PairStyle.plain
[       OK ] PairStyle.plain (24 ms)
[ RUN      ] PairStyle.omp
[       OK ] PairStyle.omp (18 ms)
[ RUN      ] PairStyle.intel
[       OK ] PairStyle.intel (6 ms)
[ RUN      ] PairStyle.opt
[  SKIPPED ] PairStyle.opt (0 ms)
[ RUN      ] PairStyle.single
[       OK ] PairStyle.single (7 ms)
[ RUN      ] PairStyle.extract
[       OK ] PairStyle.extract (6 ms)
[----------] 6 tests from PairStyle (62 ms total)

[----------] Global test environment tear-down
[==========] 6 tests from 1 test suite ran. (63 ms total)
[  PASSED  ] 5 tests.
[  SKIPPED ] 1 test, listed below:
[  SKIPPED ] PairStyle.opt

In this particular case, 5 out of 6 sets of tests were conducted, the tests for the lj/cut/opt pair style was skipped, since the tests executable did not include it. To learn what individual tests are performed, you (currently) need to read the source code. You can use code coverage recording (see next section) to confirm how well the tests cover the code paths in the individual source files.

The force style test programs have a common set of options:

Option

Function

-g <newfile>

regenerate reference data in new YAML file

-u

update reference data in the original YAML file

-s

print error statistics for each group of comparisons

-v

verbose output: also print the executed LAMMPS commands

The ctest tool has no mechanism to directly pass flags to the individual test programs, but a workaround has been implemented where these flags can be set in an environment variable TEST_ARGS. Example:

env TEST_ARGS=-s ctest -V -R BondStyle

To add a test for a style that is not yet covered, it is usually best to copy a YAML file for a similar style to a new file, edit the details of the style (how to call it, how to set its coefficients) and then run test command with either the -g and the replace the initial test file with the regenerated one or the -u option. The -u option will destroy the original file, if the generation run does not complete, so using -g is recommended unless the YAML file is fully tested and working.

Recommendations and notes for YAML files

  • The reference results should be recorded without any code optimization or related compiler flags enabled.

  • The epsilon parameter defines the relative precision with which the reference results must be met. The test geometries often have high and low energy parts and thus a significant impact from floating-point math truncation errors is to be expected. Some functional forms and potentials are more noisy than others, so this parameter needs to be adjusted. Typically a value around 1.0e-13 can be used, but it may need to be as large as 1.0e-8 in some cases.

  • The tests for pair styles from OPT, USER-OMP and USER-INTEL are performed with automatically rescaled epsilon to account for additional loss of precision from code optimizations and different summation orders.

  • When compiling with (aggressive) compiler optimization, some tests are likely to fail. It is recommended to inspect the individual tests in detail to decide, whether the specific error for a specific property is acceptable (it often is), or this may be an indication of mis-compiled code (or an undesired large loss of precision due to significant reordering of operations and thus less error cancellation).

Tests for other components and utility functions

Additional tests that validate utility functions or specific components of LAMMPS are implemented as standalone executable which may, or may not require creating a suitable LAMMPS instance. These tests are more specific and do not require YAML format input files. To add a test, either an existing source file needs to be extended or a new file added, which in turn requires additions to the CMakeLists.txt file in the source folder.

Collect and visualize code coverage metrics

You can also collect code coverage metrics while running LAMMPS or the tests by enabling code coverage support during the CMake configuration:

-D ENABLE_COVERAGE=on  # enable coverage measurements (off by default)

This will instrument all object files to write information about which lines of code were accessed during execution in files next to the corresponding object files. These can be post-processed to visually show the degree of coverage and which code paths are accessed and which are not taken. When working on unit tests (see above), this can be extremely helpful to determine which parts of the code are not executed and thus what kind of tests are still missing. The coverage data is cumulative, i.e. new data is added with each new run.

Enabling code coverage will also add the following build targets to generate coverage reports after running the LAMMPS executable or the unit tests:

make gen_coverage_html   # generate coverage report in HTML format
make gen_coverage_xml    # generate coverage report in XML format
make clean_coverage_html # delete folder with HTML format coverage report
make reset_coverage      # delete all collected coverage data and HTML output

These reports require GCOVR to be installed. The easiest way to do this to install it via pip:

pip install git+https://github.com/gcovr/gcovr.git

After post-processing with gen_coverage_html the results are in a folder coverage_html and can be viewed with a web browser. The images below illustrate how the data is presented.

_images/coverage-overview-top.png

Top of the overview page

_images/coverage-overview-manybody.png

Styles with good coverage

_images/coverage-file-top.png

Top of individual source page

_images/coverage-file-branches.png

Source page with branches

3.9.4. Coding style utilities

To aid with enforcing some of the coding style conventions in LAMMPS some additional build targets have been added. These require Python 3.5 or later and will only work on Unix-like operating and file systems. The following options are available.

make check-whitespace    # generate coverage report in HTML format
make fix-whitespace      # generate coverage report in XML format
make check-permissions   # delete folder with HTML format coverage report
make fix-permissions     # delete all collected coverage data and HTML output

For the code in the unittest tree we are using the clang-format tool (Clang version 8.0 or later is required). If available, the source code files in the unittest tree can be updated to conform to the formatting settings using make format-tests.