Execution Model¶

Mint provides a mesh-aware Execution Model, based on the RAJA programming model abstraction layer. The execution model supports on-node fine-grain parallelism for mesh traversals. Thereby, enable the implementation of computational kernels that are born parallel and portable across different processor architectures.

Note

To utilize NVIDIA GPUs, using the RAJA CUDA backend, Axom needs to be compiled with CUDA support and linked to a CUDA-enabled RAJA library. Consult the Axom Quick Start Guide for more information.

The execution model consists of a set of templated functions that accept two arguments:

A pointer to a mesh object corresponding to one of the supported Mesh Types.
The kernel that defines the operations on the supplied mesh, which is usually specified by a C++11 Lambda Expression.

Note

Instead of a C++11 Lambda Expression a C++ functor may also be used to encapsulate a kernel. However, in our experience, using C++11 functors, usually requires more boiler plate code, which reduces readability and may potentially have a negative impact on performance.

The Execution Model provides Node Traversal Functions, Cell Traversal Functions and Face Traversal Functions to iterate and operate on the constituent Nodes, Cells and Faces of the mesh respectively. The general form of these functions is shown in Fig. 21.

As shown in Fig. 21, the key elements of the functions that comprise the Execution Model are:

The Iteration Space: Indicated by the function suffix, used to specify the mesh entities to traverse and operate upon, e.g. the Nodes, Cells or Faces of the mesh.
The Execution Policy: Specified as as the first, required, template argument to the constituent functions of the Execution Model. The Execution Policy specifies where and how the kernel is executed.
The Execution Signature: Specified by a second, optional, template argument to the constituent functions of the Execution Model. The Execution Signature specifies the type of arguments supplied to a given kernel.
The Kernel: Supplied as an argument to the constituent functions of the Execution Model. It defines the body of operations performed on the supplied mesh.

See the Tutorial for code snippets that illustrate how to use the Node Traversal Functions, Cell Traversal Functions and Face Traversal Functions of the Execution Model.

Execution Policy¶

The Execution Policy is specifed as the first template argument and is required by all of the constituent functions of the Execution Model. Axom defines a set of high-level execution spaces, summarized in the table below.

Execution Policy	Requirements	Description
SEQ_EXEC	None.	Sequential execution on the CPU.
OMP_EXEC	RAJA + OpenMP	Parallel execution on the CPU using OpenMP.
CUDA_EXEC< BLOCKSIZE >	RAJA + CUDA + Umpire (memory management)	Parallel execution on CUDA-enabled GPUs.
CUDA_EXEC< BLOCKSIZE, ASYNC >	RAJA + CUDA + Umpire (memory management)	Asynchronous parallel execution on CUDA-enabled GPUs.

Internally, the implementation uses the axom::execution_space traits object to map each execution space to corresponding RAJA execution policies and bind the default memory space for a given execution space. For example, the default memory space for the axom::CUDA_EXEC execution space is unified memory, which can be accessed from both the host (CPU ) and device (GPU).

Execution Signature¶

The Execution Signature is specified as the second, optional template argument to the constituent functions of the Execution Model. The Execution Signature indicates the list of arguments that are supplied to the user-specified kernel.

Note

If not specified, the default Execution Signature is set to mint::xargs::index, which indicates that the supplied kernel takes a single argument that corresponds to the index of the corresponding iteration space, i.e, the loop index.

The list of currently available Execution Signature options is based on commonly employed access patterns found in various mesh processing and numerical kernels. However, the Execution Model is designed such that it can be extended to accomodate additional access patterns.

mint::xargs::index
- Default Execution Signature to all functions of the Execution Model
- Indicates that the supplied kernel takes a single argument that corresponds to the index of the iteration space, i.e. the loop index.
mint::xargs::ij/mint::xargs::ijk
- Applicable only with a Structured Mesh.
- Used with Node Traversal Functions (mint::for_all_nodes()) and Cell Traversal Functions (mint::for_all_cells()).
- Indicates that the supplied kernel takes the corresonding \((i,j)\) or \((i,j,k)\) indices, in 2D or 3D respectively, as additional arguments.
mint::xargs::x/mint::xargs::xy/mint::xargs::xyz
- Used with Node Traversal Functions (mint::for_all_nodes()).
- Indicates that the supplied kernel takes the corresponding nodal coordinates, \(x\) in 1D, \((x,y)\) in 2D and \((x,y,z)\) in 3D, in addition to the corresponding node index, nodeIdx.
mint::xargs::nodeids
- Used with Cell Traversal Functions (mint::for_all_cells()) and Face Traversal Functions (mint::for_all_faces()).
- Indicates that the specified kernel is supplied the constituent node IDs as an array argument to the kernel.
mint::xargs::coords
- Used with Cell Traversal Functions (mint::for_all_cells()). and Face Traversal Functions (mint::for_all_faces())
- Indicates that the specified kernel is supplied the constituent node IDs and corresponding coordinates as arguments to the kernel.
mint::xargs::faceids
- Used with the Cell Traversal Functions (mint::for_all_cells()).
- Indicates that the specified kernel is supplied an array consisting of the constituent cell face IDs as an additional argument.
mint::xargs::cellids
- Used with the Face Traversal Functions (mint::for_all_faces()).
- Indicates that the specified kernel is supplied the ID of the two abutting Cells to the given. By conventions, tor external boundary Faces, that are bound to a single cell, the second cell is set to \(-1\).

Warning

Calling a traversal function with an unsupported Execution Signature will result in a compile time error.