Memory Checking

There are two commonly available memory checkers available to use with Axom on LC: AddressSanitizer and Valgrind.

AddressSanitizer

AddressSanitizer (aka ASan) is memory error detection tool that is a part of LLVM. It is very fast and easy to use but doesn’t seem as robust as Valgrind. It requires compile and link flags which are enabled via the CMake option AXOM_ENABLE_ASAN. Anything in our CMake system will get those flags after that is enabled but our third-party libraries (like MFEM) will not. After that just run your built executable and Asan will output a log to the screen after your program is done running. ASan’s behavior can be modified with a set of environment variables .

Note

ASan only works with the Clang and GCC compiler chains. Our build system will throw an error if you try to build with anything else while AXOM_ENABLE_ASAN is ON.

Here is a recommended ASan workflow:

# To configure and build the code with ASan enabled
./config-build.py -hc host-configs/rzwhippet-toss_4_x86_64_ib-clang@14.0.6.cmake -DAXOM_ENABLE_ASAN=ON
cd build-rzwhippet-toss_4_x86_64_ib-clang@14.0.6-debug
srun -N1 --exclusive --mpi-bind=off make -j

# To run a single unit test
LSAN_OPTIONS=suppressions=../suppressions.asan ASAN_OPTIONS=log_path=asan.out:log_exe_name=true srun -n2 <path to test>

# To run all the unit tests, requires absolute path to suppression file
LSAN_OPTIONS=suppressions=/absolute/path/to/suppressions.asan ASAN_OPTIONS=log_path=asan.out:log_exe_name=true ctest -VV

This will output files in the current directory for each process that follow the pattern: asan.out.<exe name>.<pid>. It also sets your return code to a non-zero value if there were any non-suppressed memory errors.

LSAN_OPTIONS and ASAN_OPTIONS are delimited by ‘:’.

Here is an explanation of the given options (all should be added to ASAN_OPTIONS unless noted):

  • suppressions: Location of memory leak suppression file (LSAN_OPTIONS)

  • log_path: Logs to the given file instead of to the screen. This is very helpful to avoid intermingled lines on the screen from every process

  • log_exe_name: Adds executable name to log_path

Helpful options:

  • fast_unwind_on_malloc=0: This improves ASan’s stack tracing ability but also greatly slows down the run

  • exitcode=0: This stops ASan from returning a non-zero exit code from your executable (defaults to 23) (LSAN_OPTIONS)

Note

Axom can be built to use both ASan and UBSan by enabling both via CMake, and then following the recommended ASan workflow above:

./config-build.py -hc host-configs/rzwhippet-toss_4_x86_64_ib-clang@14.0.6.cmake -DAXOM_ENABLE_ASAN=ON -DAXOM_ENABLE_UBSAN=ON

Valgrind

Valgrind is a very powerful set of tools that help with dynamic analysis tools. We will focus on memcheck which is a memory error detection tool.

Unlike ASan, valgrind does not need any special compiler flags. Just build your executable and run your executable with valgrind. Valgrind’s suppression files are easily generated by valgrind with --gen-suppressions=all and are more customizable than ASan’s.

Here is a recommended workflow:

./config-build.py -hc host-configs/rzgenie-toss_3_x86_64_ib-gcc@8.1.0.cmake
cd build-rzgenie-toss_3_x86_64_ib-gcc@8.1.0-debug
srun -N1 --exclusive --mpi-bind=off make -j
srun -n2 valgrind --tool=memcheck --log-file=valgrind.out --leak-check=yes --show-leak-kinds=all --num-callers=20 --suppressions=../suppressions.valgrind <path to test>

This will produce a file called valgrind.out in the current directory with a valgrind report.

Here is an explanation of the given options:

  • --tool=memcheck: valgrind is a tool-suite so this runs the memcheck tool

  • --log-file=valgrind.out: Logs report to the given file

  • --leak-check=yes: Enables memory leak checks

  • --show-leak-kinds=all`: Enables showing all memory leak kinds

  • --num-callers=20: Limits the size of the stack traces to 20

  • --suppressions=../suppressions.valgrind: Location of memory leak suppression file

Undefined Behavior Checking

Axom uses the UndefinedBehaviorSanitizer on LC. UndefinedBehaviorSanitizer (aka UBSan) is an undefined behavior detection tool that is a part of LLVM. UBSan requires compile and link flags which are enabled via the CMake option AXOM_ENABLE_UBSAN. After that just run your built executable and UBSan will detect and output undefined behavior errors.

Note

UBSan only works with the Clang and GCC compiler chains. Our build system will throw an error if you try to build with anything else while AXOM_ENABLE_UBSAN is ON.

Here is a workflow that enables UBSan and runs unit tests with the tool enabled:

./config-build.py -hc host-configs/rzwhippet-toss_4_x86_64_ib-clang@14.0.6.cmake -DAXOM_ENABLE_UBSAN=ON
cd build-rzwhippet-toss_4_x86_64_ib-clang@14.0.6-debug
srun -N1 --exclusive --mpi-bind=off make -j
make test