We now explain the general evolution of data. This evolution spans the duration of events from the moment the data is sent to the computing resources for the execution of simulation tasks, to the moment it is stored as objects or structured data in the database to be read by the Web Interface. We refer to this evolution flow as the Data Lifecycle.
In the image below, we present a flowchart summarizing the steps involved in the data lifecycle.
The Data Lifecycle is composed primarily of the following three steps. The reader is referred to the same number labelling cited in the above image.
1. Entity Structured Data Creation¶
The actions performed by the user in the Web Interface lead to the creation of entity-related structured data in the corresponding account-owned collections, stored within the database. Jobs corresponding to the simulation tasks provide one such example.
2. Execution of Simulation¶
The structured data describing such Entities is then sent from the database to the computing clusters, the core component of the overall computational infrastructure. Here, the relevant simulations are executed, which creates non-structured data in the form of output files.
3. Data Retrieval¶
Following simulation completion, the output data produced by the simulation engine is stored on disk as unstructured files.
3a. Extraction of Properties as Structured Data¶
3b. Storage of Output Files as Objects¶
All simulation output files are additionally stored in the Object storage, in order to make them accessible to the Web Interface. This can be accomplished under the Files Tab of Job Viewer, for example.