Documents

Sina Document objects are a way to represent the top-level object of a JSON file that conforms to the Sina schema. When serialized, these documents can be ingested into a Sina database and used with the Sina tool.

Document objects follow a very basic JSON layout consisting of two entries: records and relationships. Each of these entries will store a list of their respective objects. An example of an empty document is shown below:

{
    "records": [],
    "relationships": []
}

The records list can contain Record objects and their inheritying types, such as Run objects. The relationships list can contain Relationship objects. For more information on these objects, see Records and Relationships.

Assembling Documents

Document objects can be assembled programatically. To accomplish this:

  1. Create a new instance of the Document class

  2. Create a Record

  3. Add the instance of the Record with the add method

On the Sina C++ User Guide page, you can see an example of this process. Below we will expand on this example to add a Relationship:

// Copyright (c) Lawrence Livermore National Security, LLC and other
// Axom Project Contributors. See top-level LICENSE and COPYRIGHT
// files for dates and other details.
//
// SPDX-License-Identifier: (BSD-3-Clause)

#include "axom/config.hpp"
#include "axom/sina.hpp"

int main(void)
{
  // Create a new document
  axom::sina::Document document;

  // Create a record of this specific study
  // This study has an ID of "study1", which has to be unique to this file
  axom::sina::ID studyID {"study1", axom::sina::IDType::Local};
  std::unique_ptr<axom::sina::Record> study {new axom::sina::Record {studyID, "UQ study"}};

  // Create a run of "My Sim Code" version "1.2.3", which was run by "jdoe".
  // The run has an ID of "run1", which has to be unique to this file.
  axom::sina::ID runID {"run1", axom::sina::IDType::Local};
  std::unique_ptr<axom::sina::Record> run {
    new axom::sina::Run {runID, "My Sim Code", "1.2.3", "jdoe"}};

  // Create a relationship between the study and the run
  // Here we're saying that the study contains the run
  axom::sina::Relationship relationship {studyID, "contains", runID};

  // Add the run, study record, and relationship to the document
  document.add(std::move(run));
  document.add(std::move(study));
  document.add(relationship);

  // Save the document directly to a file.
  // since we gave saveDocument no optional protocol parameter, it will default to JSON
  axom::sina::saveDocument(document, "MySinaData.json");

  // You can also add a document's contents to an already-dumped one, useful for capturing
  // snapshots from a code. Appending has quite a bit of functionality, see the documentation
  // for documents for more info.
  axom::sina::Document document2;
  axom::sina::ID studyID2 {"study2", axom::sina::IDType::Local};
  std::unique_ptr<axom::sina::Record> study2 {new axom::sina::Record {studyID2, "UQ study"}};
  document2.add(std::move(study2));
  axom::sina::appendDocumentToJson("MySinaData.json", document2, 1, false);

#ifdef AXOM_USE_HDF5
  // We will also save a copy of the document as an HDF5 file
  // which can be done by passing the protocol as HDF5
  axom::sina::saveDocument(document, "MySinaData.hdf5", axom::sina::Protocol::HDF5);
#endif
}

After executing the above code, the resulting MySinaData.json file will look like so:

{
    "records": [
        {
            "type": "run",
            "local_id": "run1",
            "application": "My Sim Code",
            "version": "1.2.3",
            "user": "jdoe"
        },
        {
            "type": "UQ study",
            "local_id": "study1"
        }
    ],
    "relationships": [
        {
            "predicate": "contains",
            "local_subject": "study1",
            "local_object": "run1"
        }
    ]
}

Generating Documents From JSON

Alternatively to assembling Document instances programatically, it is also possible to generate Document objects from existing JSON files or JSON strings.

Using our same example from the previous section, if we instead had the MySinaData.json file prior to executing our code, we could generate the document using Sina’s loadDocument() function:

#include "axom/sina.hpp"

int main (void) {
    axom::sina::Document myDocument = axom::sina::loadDocument("MySinaData.json");
}

Similarly, if we had JSON in string format we could also load an instance of the Document that way:

#include "axom/sina.hpp"

int main (void) {
    std::string my_json = "{\"records\":[{\"type\":\"run\",\"id\":\"test\"}],\"relationships\":[]}";
    axom::sina::Document myDocument = axom::sina::Document(my_json, axom::sina::createRecordLoaderWithAllKnownTypes());
    std::cout << myDocument.toJson() << std::endl;
}

Generating Documents From HDF5

In addition to assembling Document instances from existing JSON files, it is possible to generate Document objects from existing HDF5 files using Conduit.

When Axom is configured with HDF5 support, Sina’s saveDocument() and loadDocument() functions support HDF5 assembly through the Protocol::HDF5 argument. The functions will throw a runtime error with the list of available types in Axom configurations if Protocol::HDF5 is attempted without HDF5 support.

#include "axom/sina.hpp"

int main (void) {
    axom::sina::Document myDocument = axom::sina::loadDocument("MySinaData.hdf5", axom::sina::Protocol::HDF5);
}

Obtaining Records & Relationships from Existing Documents

Sina provides an easy way to query for both Record and Relationship objects that are associated with a Document. The getRecords() and getRelationships() methods will handle this respectively.

Below is an example showcasing their usage:

// Copyright (c) Lawrence Livermore National Security, LLC and other
// Axom Project Contributors. See top-level LICENSE and COPYRIGHT
// files for dates and other details.
//
// SPDX-License-Identifier: (BSD-3-Clause)

#include "axom/sina.hpp"
#include "axom/slic.hpp"

int main(void)
{
  // Initialize slic
  axom::slic::initialize();

  // Create a new document
  axom::sina::Document document;

  // Create a record of this specific study
  // This study has an ID of "study1", which has to be unique to this file
  axom::sina::ID studyID {"study1", axom::sina::IDType::Local};
  std::unique_ptr<axom::sina::Record> study {new axom::sina::Record {studyID, "UQ study"}};

  // Create a run of "My Sim Code" version "1.2.3", which was run by "jdoe".
  // The run has an ID of "run1", which has to be unique to this file.
  axom::sina::ID runID {"run1", axom::sina::IDType::Local};
  std::unique_ptr<axom::sina::Record> run {
    new axom::sina::Run {runID, "My Sim Code", "1.2.3", "jdoe"}};

  // Create a relationship between the study and the run
  // Here we're saying that the study contains the run
  axom::sina::Relationship relationship {studyID, "contains", runID};

  // Add the run, study record, and relationship to the document
  document.add(std::move(run));
  document.add(std::move(study));
  document.add(relationship);

  // Query for a list of records and relationships
  auto &records = document.getRecords();
  auto &relationships = document.getRelationships();

  SLIC_ASSERT_MSG(records.size() == 2, "Unexpected number of records found.");
  std::cout << "Number of Records: " << records.size() << std::endl;
  SLIC_ASSERT_MSG(relationships.size() == 1, "Unexpected number of relationships found.");
  std::cout << "Number of Relationships: " << relationships.size() << std::endl;

  // Finalize slic
  axom::slic::finalize();
}

Running this will show that both records and the one relationship were properly queried:

Number of Records: 2
Number of Relationships: 1

Appending Documents to Existing Sina Files

It’s normal during a simulation to need to dump data multiple times, for example, to collect quantities at certain milestones, write timeseries once they reach a certain length, or add new sets of curves. The append methods such as appendDocument() cover this case. Simply write your first document to the filesystem, and when you reach a point where you would like to write another, use an append function matching the filetype (JSON is recommended for small files, namely anything not involving timeseries, and HDF5 for anything beefier). There’s some nuance to how appending works:

  • If your new document contains records not present in the file on disk, it will add them. This is generally how you’ll want to do snapshots; it determines whether records match by checking the ID, so as long as each snapshot has a unique ID, they’ll accumulate in the target file.

  • If your new document contains records that ARE present in the file on disk, it will attempt to merge the data they contain. This is most useful when you have a longer-running simulation where you want to accumulate timeseries values. Simply ensure that the record containing new values has the same ID as the one containing what you have so far.

  • For data, files, user defined, and anything else that isn’t curve sets or libraries, it will go through field-by-field. If there’s a field not already present, it will be added. If it IS already present, an optional argument allows you to define the behavior. By default, newest wins, but you can also have oldest win or refuse the write.

  • For curve sets, it will append new values to the existing dependents/independents. There is some checking to ensure all timeseries WITHIN A CURVE SET end up the same length, so while it is possible to add new dependents (or independents) to a set of curves while a simulation is running, ensure that you’re backfilling the required number of values (ex: you dump every 10 cycles, you’ve dumped 20 times, each curve has 200 values. If on your next dump you also add brand_new_curve, it MUST have 210 values.) You’ll almost always want to define curves up front and add in new values.

  • For library_data, append will recurse, following the above rules.

In general, appending is very powerful, but a bit complicated; if Sina encounters any issues, it will write them to the returned conduit node for troubleshooting.

Filetype Comparisons