Using Sidre with MFEM

Note

The functionality described in this page is only available if Axom is configured with MFEM and if the CMake variable AXOM_ENABLE_MFEM_SIDRE_DATACOLLECTION is set to ON

The MFEMSidreDataCollection class implements MFEM’s DataCollection interface for recording simulation data. Specifically, it knows about fields (mfem::GridFunction) and the mesh on which these fields are defined.

The MFEMSidreDataCollection internally organizes its date according to the Mesh Blueprint, a hierarchical schema for describing mesh data.

See the Conduit page for more information on the Mesh Blueprint.

In this document, we first discuss Getting Started and how MFEM objects can be associated with the DataCollection. We then explain the process for and options available when Saving Data to a File. The workflow for reading saved data back in is discussed in Restarting a Simulation.

Getting Started

We begin to describe the data in a MFEMSidreDataCollection by “registering” a mesh with an instance of the class, e.g. at construction time:

  // Initialize the datacollection with the mesh
  // Note: all fields (added with RegisterField) must be on this mesh
  axom::sidre::MFEMSidreDataCollection dc("sidre_mfem_datacoll_vis_ex", mesh);

It can also be registered after construction:

dc.SetMesh(/* mfem::Mesh* */ mesh);

Note

There is a 1-1 relationship between DataCollection instances and meshes. That is, multiple meshes cannot be associated with a DataCollection.

Once a mesh has been registered, fields of interest can be association with the DataCollection:

  // Note: Any number of fields can be registered
  dc.RegisterField("solution", &soln);

Special kinds of fields (material-dependent fields, material set volume fraction fields, and species set values fields) require some setup before they can be registered - see Mixed-material Fields for more info.

Saving Data to a File

The data in an instance of the MFEMSidreDataCollection class can be saved to a file using a variety of protocols. These files can be visualized with tools like VisIt, or used to restart a simulation by loading them back into an instance of the class (see Restarting a Simulation).

Current options for output formats (protocols) include:

  • sidre_hdf5
  • sidre_conduit_json
  • sidre_json
  • conduit_hdf5

Note

The AXOM_USE_HDF5 build option must be enabled to use the HDF5-based formats.

Before saving, simulation state metadata should be updated. Currently, this metadata consists of:

  • cycle, the current iteration number for the simulation
  • time, the current simulation time
  • time_step, the current simulation time step (sometimes called dt)

Each of these variables has a corresponding “setter” function:

  // Save the initial state
  dc.SetCycle(0);      // Iteration counter
  dc.SetTime(0.0);     // Simulation time
  dc.SetTimeStep(dt);  // Time step

Note

There are also corresponding accessors for these state variables (GetCycle, GetTime, GetTimeStep)

Once state information has been updated, the complete simulation state can be written to a file:

  // Filename and protocol, both of which are optional
  dc.Save("sidre_mfem_datacoll_vis_ex", "sidre_hdf5");

Note

By default, save files will be written to the current directory. To write to/read from a different directory, use SetPrefixPath to change the DataCollection’s “working directory”.

See the sidre_mfem_datacollection_vis example for a more thorough example of the above functionality.

Restarting a Simulation

Experimental support for complete reconstruction of a simulation’s mesh, fields, and qfields is also provided by MFEMSidreDataCollection. That is, when an output file is read in using MFEMSidreDataCollection::Load, the data read in will be used to reconstruct MFEM objects than can be accessed with the GetField, GetQField, and GetMesh methods.

Warning

Currently, the MFEMSidreDataCollection must own the mesh and field data in order to completely reconstruct simulation state. Mesh data ownership can be configured with the owns_mesh_data constructor option (should be set to true), and field data ownership requires that the GridFunction be unallocated when passed to RegisterField, which performs the allocation within Sidre-owned memory. After registering, the GridFunction can be used normally. The same conditions apply for QuadratureFunction objects.

A complete demonstration of functionality is provided in the sidre_mfem_datacollection_restart example, which is a stripped-down example of how a simulation code might utilize the automatic reconstruction logic when loading in a datastore.

Note

The mesh/field reconstruction logic requires that the save file was created with the MFEMSidreDataCollection class. In Mesh Blueprint terms, the following constraints are imposed on the structure of the data:

  • There must be a coordinate set named coords
  • There must be a topology named mesh with corresponding attributes stored in a field named mesh_material_attribute
  • There must be a topology named boundary with corresponding attributes stored in a field named boundary_material_attribute

Mixed-material Fields

The Mesh Blueprint provides support for mixed-material simulations through its “material set” construct. Material metadata (stored in mfem::GridFunction s) can be registered in the DataCollection like any other field (with RegisterField), given that some additional setup is performed and the field names match a specific naming convention.

Currently, there are three kinds of “special” fields that can be associated with mixed-material metadata:

  • Volume fraction fields in a material set (the per-element proportion of a given material)
  • Material-dependent fields (a different set of values for each material)
  • Species set fields (a different set of values for each material and for each dimension)

Material sets are defined by associating a volume fraction field name with a material set name:

  // Inform the DataCollection that volume fraction information for a material set
  // called "matset" will be in fields called "volume_fraction"
  dc.AssociateMaterialSet("volume_fraction", "matset");

Once this association has been made, corresponding fields can be registered. Their names must satisfy <associated volume fraction field name>_<material id>, for example:

  dc.RegisterField("volume_fraction_001", &vol_frac_1);
  dc.RegisterField("volume_fraction_002", &vol_frac_2);

Material-dependent fields are defined by associating a field name with a material set name:

  // Inform the DataCollection of a material-dependent field "density" associated
  // with a material set called "matset"
  dc.AssociateMaterialDependentField("density", "matset");

Material-specific values can be registered with the name <associated dependent field name>_<material id>:

  dc.RegisterField("density", &density_indenpendent);
  dc.RegisterField("density_001", &density_dependent_1);
  dc.RegisterField("density_002", &density_dependent_2);

Species sets are defined by associating a field name with a species set name and corresponding material set name:

  // Inform the DataCollection that species set data for the species set "specset"
  // associated with material set "matset" will be in fields called "partial_density"
  const bool volume_dependent = false;
  dc.AssociateSpeciesSet("partial_density", "specset", "matset", volume_dependent);

Species set values can be registered with the name <associated field name>_<material id>_<component>:

  dc.RegisterField("partial_density_001_001", &partial_density_1_1);
  dc.RegisterField("partial_density_001_002", &partial_density_1_2);
  dc.RegisterField("partial_density_002_001", &partial_density_2_1);
  dc.RegisterField("partial_density_002_002", &partial_density_2_2);