Third-party Libraries¶
Axom dependencies are grouped into four categories: Git submodules, built-in Third-party Libraries (TPLs) in the Axom source tree, system-level TPLs, and other TPL libraries. The following sections describe how to install and update these dependencies for Axom.
How does one add a new compiler or platform to the mix?
How does one build a new set of TPLs for a single platform or compiler for testing?
What is the procedure for changing versions of one or more TPLs?
How do we keep things straight when using different TPL versions for different branches?
How to use the scripts for team TPL support vs. local development and experimentation?
Others?
Determinism¶
We strive for as close to deterministic behavior in our builds as possible. By this, we mean that repeated builds should act the same in the following regards:
Set of libraries with their options and versions
Compilers, compiler flags, and versions
Installed file and directory structure with permissions
Build Scripts and Their Configuration Files¶
There are three levels of build scripts or programs that drive TPL builds. As you move up the levels, and away from Spack, the scripts require less configuration and even build multiple sets of TPLs and/or Axom configurations at a time.
Here is a brief description of what the levels are handling and what important configuration and input files they use, from lowest level to highest.
Level 1: Spack¶
Spack is a multi-platform package manager that builds and installs multiple versions and configurations of software packages. It has recipes on how to build each package with variants on each package to customize them to your needs. For example, Axom has variants for Fortran and MPI, among others. These recipes handle how to drive the individual packages build systems, as well as any packages they depend on. Spack also handles system level packages, so you can describe where they are on your system instead of building them from scratch. You will need to describe which compilers are available on your system as well.
Platform specific configuration files live under
scripts/spack/configs/<platform name>
. There is one file (spack.yaml
) per platform that handles the following:compilers
: This section contains the compiler specs that describe the location and any other required information about that compiler. For example, compiler or linker flags.packages
: This section describes the system level packages. For example, where they are located and what version they are. This file is very imporant due to its ability to drastically reduce the amount of packages that Spack builds.
Axom specific Spack package files live under
scripts/spack/packages
. These override the package files that live in Spack’s repository herevar/spack/repos/builtin/packages
. We try to minimize these but we have had to alter the existing packages to apply fixes before pushing them up to Spack proper or alterations to the recipes that are Axom specific. This overriding does not happen at the Spack level, but at the next level, Uberenv.
Note
Spack does not stop at the first error. It attempts to build as many packages
as possible. Due to this, finding the actual error can sometimes be hard but looking
through the log for a large indented section will help. The error will
be in that section and also a message with a path to the full log will be printed
by Spack afterwards. Searching for -build-out.txt
in your output should
help.
Level 1: Vcpkg¶
Vcpkg is an open-source C++ Library Manager for Windows, Linux, and MacOS by Microsoft. Axom only uses it for our Windows TPL builds.
Project specific package files live under
develop/scripts/vcpkg_ports
. There are two different files for each package:portfile.cmake
: This file is the recipe on how to build the package. Vcpkg has strict rules about how your project is laid out and you can do the conversion in this file as well.vcpkg.json
: This is the manifest file that describes information about the package. For example, dependencies, license information, and optional features.
Level 2: Uberenv¶
Uberenv simplifies the use of two level 1 package managers, Spack and Vcpkg. We rely on Uberenv for two major points: reducing multiple commands into one and adding as much determinism as possible. The basic workflow in Uberenv is the following:
Setup necessary paths and directories like the base directory where the package manager will be installed.
Clone the package manager to the specific Git commit.
Apply patches to package manager. For example, disabling extra config scopes in Spack.
Adds our repositories package repository to Spack, so our packages take precedence.
Clean previous temporary information from previous runs that may bleed into this run.
Optionally create a package source mirror.
Install packages via the selected package manager.
.uberenv_config.json
: This file describes project specific configurations, such as, where to download the package manager, what git commit to use, and the top level package to install.
Note
Uberenv’s warnings and errors are easy to find by searching the output for [ERROR:
or [Warning:
. Uberenv will stop at the first error.
Level 3: Build Scripts¶
There are three “build” scripts that live in scripts/llnl
that are designed
to handle suites of building TPLs via uberenv and Spack. They automatically
handle the platform differences and know the full list of compilers and package
specs required.
scripts/spack/specs.json
: This contains a list of all specs required per platform or machine name.build_tpls.py
: This script starts with building all TPLs listed inspecs.json
. It will copy the generated host-configs to the base of the Axom repository. After building all of the TPLs, it will test Axom against those built TPLs. As well, as testing the installedusing-with-cmake
example for correctness. This script stops at the first failed TPL build but attempts to build all host-configs against the Axom source with a summary at the end of which succeeded or failed.build_src.py
: This scripts takes the existing host-configs, or the specific one you point at, and builds and tests Axom against them. It also tests theusing-with-cmake
examples.build_devtools.py
: This script builds and installs the developer tools listed in theaxomdevtools
Spack package. It also uses a different set of Spack configs located inscripts/spack/devtools_config
, so that the regular Spack configs can reuse the seldom and previously built developer tools.
Note
Due to the large amount of information printed to the screen over a full build, the build scripts
redirect most build step output to log files. They will not only tell you what command is being run,
i.e., [exe: some/command --with-options]
, but it will tell you the log file being written
to before it redirects the output from the command, i.e., [[log file: /path/to/log
.
Updating TPLs¶
Git submodules¶
Currently, Axom uses three external packages that appear in the repo as Git submodules. These are the following, including the location of the package in the Axom source tree:
BLT, which is the CMake-based build system we use. Location:
axom/src/cmake/blt
.Axom Data, which is where we maintain data files used in testing Axom. Location:
axom/data
.Uberenv, which contains Python scripts we use to help automate building third-party dependencies for development and deployment. Location:
axom/scripts/uberenv
.
There is no software installation process for these dependencies in the traditional sense. To update one of these packages in Axom, simply go into its directory in Axom and check out a new version. If a version is intended to be changed in the Axom repo, make the version change on a branch and submit a GitHub pull request as you would do for other software changes. More info on Building and Installing Axom.
Built-in TPLs¶
Axom several lightweight header-only libraries that we use internally and expose for downstream customers to use if they wish.
CLI11, is a command line parser for C++11 and beyond that provides a rich feature set with a simple and intuitive interface.
fmt, is an open-source formatting library providing a fast and safe alternative to C stdio and C++ iostreams.
sol, is a C++ library binding to Lua.
Sparsehash, contains several hash-map implementations.
Note
Axom patches all built-in TPLs to be under the axom
namespace.
This is to prevent symbol collisions with other projects, either our
dependencies or downstream customers who wish their own versions. For
example, fmt::format("foo")
is axom::fmt::format("foo")
.
They can be found in the directory: src/thirdparty
. The basic
instructions on how to update a built-in TPL are as follows:
Download the new release and override the source that is already there. This can often involve removing files no-longer needed but most of the current ones are a single header file.
Review and apply the existing patch files. More than likely, you will not be able to directly apply the patch but it will give you the general idea on what needs to be applied. For example, the namespace update mentioned above.
Ensure that the build and tests still pass. More info on Tests and Examples.
Follow the normal pull request work flow. More info on Pull Requests and Code Reviews.
Local Third-party Library Installation¶
It is often useful to have a different set of TPLs during the development process. For example, you may want to try out a new library or version of an existing library.
From the top-level Axom directory, run the following script to build all TPLs for all existing compiler specs on the platform you are currently on:
$ ./scripts/llnl_scripts/build_tpls.py -d local/install/path
where local/install/path
is a directory location where you want the
libraries to be installed.
It will output whether the TPL install succeeded and, subsequently, whether an Axom build against the TPL install succeeded.
Running the script produces new host-config files (i.e., CMake cache files) that you can use to build and test Axom with the installation, if issues arise. The generated host-config files will be located in the top-level Axom directory of your local clone of the repo. If any changes to Axom code are needed to work with the TPL update(s), make the changes and test them.
Note
You can build a subset of TPLs for a platform, by passing a Spack
spec arguments to the build_tpls.py
script. For example,
--spec clang@10.0.0~cpp14+devtools+mfem+c2c
will build the TPLs for the clang 10.0.0 compiler. Please see the
scripts/spack/specs.json
file for a list of currently tested specs.