Tutorial ¶

This short tutorial walks you through the basic usage of the Sina library. For more in-depth details, see the documentation for the individual classes, such as Record, Relationship, and Document.

Creating Documents and Records ¶

The basic working units in Sina are the Document, Record, and Relationship. A Document is a collection of Records and Relationships. A Record contains information about a particular entity, such as the run of an application, or a description of a uncertainty quantification (UQ) study. A Relationship describes how two records relate to each user (e.g. UQ studies contain runs).

This first example shows how to create a record:

void createRecord()
{
  axom::sina::ID id {"some_record_id", axom::sina::IDType::Local};
  std::unique_ptr<axom::sina::Record> record {new axom::sina::Record {id, "my_record_type"}};

  // Add the record to a document
  axom::sina::Document doc;
  doc.add(std::move(record));
}

The record has an ID “some_record_id”, which is unique to the enclosing document (it will be replaced by a global ID upon ingestion). The only required field for records is their type, which is “my_record_type” in this case. Once created, a record can be added to a Document.

We can create Runs. Runs are special types of records. They have the required fields of application (“My Sim Code”), version (“1.2.3”), and user (“jdoe”). The type is automatically set to “run”.

void createRun()
{
  axom::sina::ID id {"some_run_id", axom::sina::IDType::Local};
  std::unique_ptr<axom::sina::Record> run {new axom::sina::Run {id, "My Sim Code", "1.2.3", "jdoe"}};

  // Add the record to a document
  axom::sina::Document doc;
  doc.add(std::move(run));
}

Adding Data ¶

Once we have a Record, we can add different types of data to it. Any Datum object that is added will end up in the “data” section of the record in the output file.

void addData(axom::sina::Record &record)
{
  // Add a scalar named "my_scalar" with the value 123.456
  record.add("my_scalar", axom::sina::Datum {123.456});

  // Add a string named "my_string" with the value "abc"
  record.add("my_string", axom::sina::Datum {"abc"});

  // Add a list of scalars named "my_scalar_list"
  std::vector<double> scalarList = {1.2, -3.4, 5.6};
  record.add("my_scalar_list", axom::sina::Datum {scalarList});

  // Add a list of strings named "my_string_list"
  std::vector<std::string> stringList = {"hi", "hello", "howdy"};
  record.add("my_string_list", axom::sina::Datum {stringList});
}

Adding Curve Sets ¶

While you can add data items that are vectors of numbers, sometimes you want to express relationships between them. For example, you may want to express the fact that a timeplot captures the fact that there is an independent variable (e.g. “time”), and possibly multiple dependent variables (e.g. “temperature” and “energy”).

void addCurveSets(axom::sina::Record &record)
{
  axom::sina::CurveSet timePlots {"time_plots"};

  // Add the independent variable
  timePlots.addIndependentCurve(axom::sina::Curve {"time", {0.0, 0.1, 0.25, 0.3}});

  // Add some dependent variables.
  // The length of each must be the same as the length of the independent.
  timePlots.addDependentCurve(axom::sina::Curve {"temperature", {300.0, 310.0, 350.0, 400.0}});

  timePlots.addDependentCurve(axom::sina::Curve {"energy", {0.0, 10.0, 20.0, 30.0}});

  // Associate the curve sets with the record
  record.add(timePlots);
}

Adding Files ¶

It is also useful to add to a record a set of files that it relates to. For example your application generated some external data, or you want to point to a license file.

Conversely, at times it may be necessary to remove a file from the record’s file list. For example if the file was deleted or renamed.

void addAndRemoveFileToRecord(axom::sina::Record &run)
{
  axom::sina::File my_file {"some/path.txt"};
  // Adds the file to the record's file list
  run.add(my_file);
  // Removes the file from the record's file list
  run.remove(my_file);
}

Relationships Between Records ¶

Relationships between objects can be captured via the Relationship class. This relates two records via a user-defined predicate. In the example below, a new relashionship is created between two records: a UQ study and a run. The study is said to “contain” the run. As a best practice, predicates should be active verbs, such as “contains” in “the study contains the run”, rather than “is part of”, as in “the run is part of the study”.

void associateRunToStudy(axom::sina::Document &doc,
                         axom::sina::Record const &uqStudy,
                         axom::sina::Record const &run)
{
  doc.add(axom::sina::Relationship {uqStudy.getId(), "contains", run.getId()});
}

Library-Specific Data ¶

Oftentimes, simulation codes are composed of multiple libraries. If those offer a capability to collect data in a Sina document, you can leverage that to expose this additional data in your records.

For example, suppose you are using libraries named foo and bar. library foo defines foo_collectData() like this:

void foo_collectData(axom::sina::DataHolder &fooData)
{
  fooData.add("temperature", axom::sina::Datum {500});
  fooData.add("energy", axom::sina::Datum {1.2e10});
}

Library bar defines bar_gatherData() like this:

void bar_gatherData(axom::sina::DataHolder &barData)
{
  barData.add("temperature", axom::sina::Datum {400});
  barData.add("mass", axom::sina::Datum {15});
}

In your host application, you can define sections for foo and bar to add their own data.

void gatherAllData(axom::sina::Record &record)
{
  auto fooData = record.addLibraryData("foo");
  auto barData = record.addLibraryData("bar");

  foo_collectData(*fooData);
  bar_gatherData(*barData);

  record.add("temperature", axom::sina::Datum {450});
}

In the example above, once the record is ingested into a Sina datastore, users will be able to search for “temperature” (value = 450), “foo/temperature” (value = 500), and “bar/temperature” (value = 400).

Input and Output ¶

Once you have a document, it is easy to save it to a file. To save to a JSON, we run the saveDocument() with the optional argument Protocol set to JSON or set as nothing. Alternatively if you wish to save the document to an HDF5 file: Configure axom for HDF5 support then you can set saveDocument()’s optional Protocol parameter to HDF5. After executing the below, you will output a file named “my_output.json” and a file named “my_output.hdf5”, both of which you can ingest into a Sina datastore.

void save(axom::sina::Document const &doc)
{
  axom::sina::saveDocument(doc, "my_output.json");
#ifdef AXOM_USE_HDF5
  axom::sina::saveDocument(doc, "my_output.hdf5", axom::sina::Protocol::HDF5);
#endif
}

If needed, you can also load a document from a file. This can be useful, for example, if you wrote a document when writing a restart and you want to continue from where you left off. To load from a JSON file simply run loadDocument() with the optional argument Protocol set to JSON or set as nothing. If you’ve configured for, and wish to load from an HDF5 simply set the Protocol to HDF5.

Note that due to HDF5’s handling of ‘/’ as indicators for nested structures, parent nodes will have ‘/’ changed to the slashSubstitute variable located in axom/sina/core/Document.hpp as an HDF5 with saveDocument(). loadDocument() will restore them to normal:

void load()
{
  axom::sina::Document doc1 = axom::sina::loadDocument("my_output.json");
#ifdef AXOM_USE_HDF5
  axom::sina::Document doc2 = axom::sina::loadDocument("my_output.hdf5", axom::sina::Protocol::HDF5);
#endif
}

Non-Conforming, User-Defined Data ¶

While the Sina format is capable of expressing and indexing lots of different types of data, there may be special cases it doesn’t cover well. If you want to add extra data to a record but don’t care if it doesn’t get indexed, you can add it to the “user_defined” section of records (or libraries of a record). This is a JSON object that will be ignored by Sina for processing purposes, but will be brought back with your record if you retrieve it from a database.

Sina uses Conduit to convert to and from JSON. The user-defined section is exposed as a Conduit Node.

void addUserDefined(axom::sina::Record &record)
{
  conduit::Node &userDefined = record.getUserDefinedContent();
  userDefined["var_1"] = "a";
  userDefined["var_2"] = "b";

  conduit::Node subNode;
  subNode["sub_1"] = 10;
  subNode["sub_2"] = 20;
  userDefined["sub_structure"] = subNode;
}