Relating simulation studies by provenance—Developing a family of Wnt signaling models

For many biological systems, a variety of simulation models exist. A new simulation model is rarely developed from scratch, but rather revises and extends an existing one. A key challenge, however, is to decide which model might be an appropriate starting point for a particular problem and why. To answer this question, we need to identify entities and activities that contributed to the development of a simulation model. Therefore, we exploit the provenance data model, PROV-DM, of the World Wide Web Consortium and, building on previous work, continue developing a PROV ontology for simulation studies. Based on a case study of 19 Wnt/β-catenin signaling models, we identify crucial entities and activities as well as useful metadata to both capture the provenance information from individual simulation studies and relate these forming a family of models. The approach is implemented in WebProv, a web application for inserting and querying provenance information. Our specialization of PROV-DM contains the entities Research Question, Assumption, Requirement, Qualitative Model, Simulation Model, Simulation Experiment, Simulation Data, and Wet-lab Data as well as activities referring to building, calibrating, validating, and analyzing a simulation model. We show that most Wnt simulation models are connected to other Wnt models by using (parts of) these models. However, the overlap, especially regarding the Wet-lab Data used for calibration or validation of the models is small. Making these aspects of developing a model explicit and queryable is an important step for assessing and reusing simulation models more effectively. Exposing this information helps to integrate a new simulation model within a family of existing ones and may lead to the development of more robust and valid simulation models. We hope that our approach becomes part of a standardization effort and that modelers adopt the benefits of provenance when considering or creating simulation models.

Mechanistic, biochemical models are implemented and questioned to deepen the 2 understanding of biological systems. These models are usually the results of simulation 3 studies that include phases of refinement and extension of simulation models together 4 with the execution of diverse in silico (simulation) experiments. 5 A plethora of work has emerged over the last two decades to support the execution 6 and documentation of simulation studies (e.g., modeling and simulation life cycles [1], 7 workflows [2], conceptual models [3]). Depending on the application domain, different 8 modeling approaches have their own documentation guidelines [4][5][6]. In the case of 9 systems biology, the "Minimum Information Requested in the Annotation of 10 Biochemical Models (MIRIAM)" [7] and the "Minimum Information About a 11 Simulation Experiment (MIASE)" [8] are two community standards used for 12 documenting simulation models and corresponding simulation experiments. A recent 13 perspective by Porubsky et al. (2020) [9] looks at all stages of a biochemical simulation 14 study and at tools supporting their reproducibility. When looking at an entire 15 simulation study and at the generation process of the included simulation model, these 16 guidelines provide some indication about what information might be useful for 17 documenting a complete simulation study as well as for establishing relationships 18 between different simulation models. 19 This is of particular interest when several simulation models for a system under 20 consideration exist, offering different perspectives on the system, answering different 21 questions, or reflecting the data and information available at the time of generation. 22 Model repositories such as BioModels [10,11], JWS Online [12], or the Physiome Model 23 Repository 2 (PMR2) [13] provide different means to retrieve and use simulation models. 24 For example, querying the BioModels database for biochemical and cellular simulation 25 models that contain proteins such as Wnt, Janus kinase (Jak), or mitogen-activated 26 protein kinase (MAPK), which are associated with corresponding signaling pathways, 27 returns 22 simulation models for Wnt, 12 simulation models for Jak and 139 simulation 28 models for MAPK (as of January 2021). This already shows that MAPK is an 29 intensively studied signaling pathway. However, there is no way to easily compare these 30 simulation models or examine their relationships to each other. Sometimes these 31 relationships are represented in a model relationship map, such as the one created by 32 data has been used both by calibration and validation activities, or the option to reuse 93 validation experiments among model descendants to check consistency)" [20]. The 94 adaptation and application of this ontology for capturing the essential information of 95 our case study is presented in the Results and discussion section. 96 Collecting provenance information 97 In order to gather all relevant information, the publications as well as the supporting 98 materials-as they often contain model and experiment descriptions-were read 99 thoroughly. Referenced publications were checked, as well, whenever they appeared to 100 be important for the development of the simulation model. All information that 101 resembled provenance entities were marked. While reading a study, a first sketch of a 102 possible provenance graph was made. Afterwards, a revision of all markings helped to 103 finalize the graph and to remove duplicate entities. Often, authors described their 104 simulation study chronologically, which made it easy to determine the path of its 105 development, but sometimes, the connections of the entities had to be inferred from the 106 context. In general, tracing provenance information from an entire simulation study in 107 retrospective involved some interpretation of the results presented in the publication. 108 Implementing the PROV-DM ontology: WebProv 109 We have developed WebProv , a web-based tool that can be used to store, access, and 110 display provenance information from simulation studies. It allows one to insert and 111 query provenance information based on a web interface as frontend and a graph-based 112 database as backend. The frontend uses Vue, a popular JavaScript reactivity system, 113 along with D3.js, a JavaScript visualization library, to create the front-end 114 visualizations and power the node/relationship editor. As scalability was not a concern 115 when designing the tool, all graph data is sent to the frontend when the website is first 116 opened, allowing the frontend to perform approximate string matching and explore the 117 entire graph without additional queries to the database. Although this reduces the 118 responsibilities of the back-end system, the backend still provides an interface for 119 loading different types of nodes, updating data and importing/exporting JSON data 120 from Neo4j. Furthermore, the backend allows one to load in a set of nodes and 121 relationships from JSON into Neo4j on startup to initialize the database. 122 The tool can also be installed locally for testing and replicability purposes. Details 123 about its installation, as well as the code, can be found on GitHub. 124 Provenance nodes 125 The main concept of WebProv is the Neo4j Provenance Node and the dependency graph 126 created from related Provenance Nodes using Neo4j relationships. A Provenance Node 127 represents an entity or activity and, therefore, must have a classification (e.g.,

128
Simulation Model or Building Simulation Model) which defines the types of 129  [32]) to 30 (in [54]). The dimension of a system 163 may be higher if a model contains compounds or different states of the species. A  In order to provide useful information about a set of simulation models as a kind of 168 family, we need to answer the questions about which information regarding these 169 models and their development processes are needed and how to describe them. Based 170 on our earlier work on provenance of simulation models, we refine a specialization of the 171 PROV Data Model (PROV-DM) and, thus, define a PROV ontology that is capable of 172 both relating simulation models and reporting their generation processes. We also 173 examine the level of detail, or granularity, that is necessary to capture relevant 174 information of the provenance of simulation studies.

175
Rule   Depicted are the components (species, compartments, and reactions) of the central canonical Wnt signaling pathway (a) and its crosstalk with other signaling pathways (b). Note that the overview is a simplified and condensed representation. Interactions are simplified and some components of the submodels that do not directly affect the Wnt signaling pathway are omitted. Activated/phosphorylated proteins are indicated by (*). Inactive/unphosphorylated states of proteins have been omitted when possible. Submodels involving membrane-mediated processes, such as receptor/ligand interactions, destruction complex recruitment and endocytosis [32,46,52,57], or cadherin-mediated cell adhesion [35,40,50] are incorporated in (a). Submodels involving crosstalk with ERK/FGF/PI3K/Akt [33,37,54], Notch [34,37,49], and ROS/Dvl-mediated pathways [52,58] are shown in the lower panels of (b), respectively.
First, we will introduce and discuss our specialization of PROV-DM for cellular  Table 2 provide 178 overviews of these entity and activity types and should be consulted when skipping the 179 first section.

180
Second, we will discuss our findings, applying our specialization of PROV-DM and 181 demonstrate the relationships as well as specific features of the provenance information 182 from the 19 Wnt simulation studies covered in this publication.

183
Further steps towards a PROV-DM ontology for cellular 184 biochemical simulation models 185 We have revised and refined the specialization of PROV-DM, which was introduced by 186 Ruscheinski et al. (2018) [20]. For capturing provenance information from simulation 187 studies of cellular biochemical simulation models and relating these, we are defining and 188 using a) specific types of entities and activities and b) specific relations with their roles 189 and constraints. During the process of collecting provenance information from the 190 studies, we identified the types and relations as well as information that was useful for 191 describing them. Our final set of entities, activities, and relations is shown in Table 2. 192 Each entity and activity has already been mentioned for provenance, modeling or 193 documentation purposes, or experiment design of simulation 194 studies [1, 3-5, 8, 20, 21, 60-63], but they have not all been used together.

195
In the following section, we will describe these entities, activities, and relations and 196 discuss the information that should be included in WebProv . For each entity and   [1,67] and is key to 211 interpreting its outputs such as simulation data or a simulation model.   Table 2. Entities, activities and allowed relations in our PROV-DM specialization.
Entity wasGeneratedBy (Activity) In order to facilitate the analysis of assumptions, we adopted the Systems Biology 225 Ontology (SBO) [68] to categorize the assumptions. SBO provides "structured 226 controlled vocabularies, comprised of commonly used modeling terms and concepts" [69] 227 and is primarily used to "describe the entities used in computational modeling (in the 228 domain of systems biology)" [68]. By using SBO, we are trying to answer, which part of 229 the model contains assumptions rather than what was assumed.

230
In the provenance example, three assumptions with three different categories could 231 be identified. A1, for instance, reads "Dsh, TCF, and GSK3β are degraded very slowly, 232 we assume that their concentrations remain constant throughout the timecourse of a 233 Wnt signaling event" [29] and was matched to ID 362 (Concentration conservation law) 234 of SBO.  These may be used for the purpose of calibrating or validating a simulation model.

238
They also direct the modeler towards adaptation of a model if the requirements are not 239 met. We do not consider other kinds of requirements (e.g., the need of using specific  [29]. Besides the entities and activities that make up the provenance information from the study (see legend), additional entities from three other studies [64][65][66], which were used by Lee et al. (2003), are shown. The colors of the ellipses show different entity types, the borders of the rectangles visualize different activity types. The gray areas separate the individual studies. The graph displays, for example, that the Building Simulation Model activity BSM1 used, among others, the entity WD1 of type Wet-lab Data from Lee et al. 2001 [64]. This activity then generated the Simulation Model SM1. tools or approaches in performing a simulation study).

241
Typically, simulation data needs to be compared with real-world data-in our case 242 wet-lab data. These real-world measurements determine the species of interest which 243 should be part of the Requirement entity. Therefore, we record the main species 244 considered by the requirement as well as its type (either qualitative or quantitative) and 245 connect the Requirement to the wet-lab data it relates to. The list of main species will 246 make it easier to compare, interrelate and reuse simulation models as they determine 247 the focus of the model. 248 We were able to identify one requirement R1 in the provenance example of Lee et al. 249 (2003). The quantitative requirement that "Axin stimulates the phosphorylation of 250 β-catenin by GSK3β at least 24,000-fold" [29] actually refers to the wet-lab data WD1 251 obtained in another study by Dajani et al. (2003) [65]. Its main species are Axin, 252 β-catenin, and GSK3β.

253
Qualitative Model [QM]: 254 We define the qualitative model to be a network diagram, such as a reaction scheme 255 (chemical reaction network diagram), which contains the entities of the system (e.g., 256 species) and their interactions. This diagram may be presented in a formal (e.g., using 257 the Systems Biology Graphical Notation (SBGN) [70]   Furthermore, entity references should ideally consist of a digital object identifier (DOI) to make the artifact associated with the particular entity unambiguously accessible. Additional information can always be entered in the "Further Information" part of WebProv .
also be part of the qualitative model. It should be noted that the qualitative model is 261 also called conceptual model [72] sometimes, whereas in other publications, the 262 qualitative model forms part of the conceptual model [3]. 263 We record a reference to the qualitative model, which, for example, could be a The simulation model is the actual mathematical or computational model [74] that can 275 be executed by a suitable tool. In most cases of our domain, the simulation model 276 contains equations (for ODE/PDE systems) or, in some cases, reaction rules (for 277 rule-based systems). An integral part of these quantitative simulation models are the 278 parameter values as well as the initial condition. The simulation model could also be 279 described in another form (e.g., in a quantitative process algebra [75,76] or with a 280 combination of multiple formalisms [77]). Formal approaches to describe a system 281 through qualitative models (e.g., Boolean models [78] or Petri nets [79]) come with their 282 own means of analysis and are assigned to the Simulation Model entity as they are   Again, we are relying on a reference of the simulation model for accessing it. It 291 should be a link to the simulation model in Biomodels or a DOI to the description of 292 the simulation model. Ideally, it is presented in a structured and widely accepted format 293 such as SBML [80] or CellML [81].

294
As for the provenance example, the calibrated simulation model of Lee et al. (2003), 295 SM2, can be found in BioModels.

297
The simulation experiment is an execution of the simulation model. Ideally, it can be 298 linked to a complete experiment specification (e.g., as a SED-ML [82] or SESSL [83] file 299 or simply as the execution code in a general purpose programming language) and to 300 documentation in a standard format that applies reporting guidelines such as MIASE 301 for simulation experiments [8]. Different simulation experiments might be used for the 302 analysis, calibration, and validation of a simulation model.

303
To further structure the set of applied simulation experiments, we distinguish 304 simulation experiments by whether they are used for optimization, sensitivity analysis, 305 perturbation, parameter scan, steady-state analysis, or time course analysis. This list is 306 neither complete nor are the categories disjoint, and, given a different set of simulation 307 studies, they will likely be subject to renaming, extension, and refinement.  Eventually, an ontology about the various experiment types and analysis methods 320 and their use in simulation studies will be crucial as simulation experiments play a 321 central role in the provenance of simulation models. This would also help to exploit the 322 provenance information effectively, for example, for automatically generating simulation 323 experiments [84].

324
In the case of Lee et al. (2003), different simulation experiments have been executed. 325 For example, SE1 contains a parameter scan in order to validate the simulation model. 326 However, no further details are given in the paper, therefore, no reference could be 327 included in the entity (the reference is "not available"). referencing, for example, a research protocol on Protocols.io [85]. The type of the 342 wet-lab experiment (in vitro or in vivo) as well as the used organism and organ/ tissue/ 343 cell line should be recorded.

344
In the provenance example, Lee et al. (2003) have executed in vitro wet-lab 345 experiments with an egg extract of Xenopus and have shown in WD1 that the "turnover 346 of GSK3β, Dsh, and TCF is relatively slow" [29]. The data from this wet-lab 347 experiment is not shown in the publication. The simulation data SD2 contains the 348 results of the successful validation of the simulation model SM2. The simulation data is 349 presented in Figure 2 of their publication. The way the provenance graph and the 350 metadata of SD2 is visualized in WebProv can be seen in    [29] with additional entities from three other studies [64][65][66], which are automatically colored differently. The node SD2 has been clicked on, which opens a box on the right with the stored and editable metadata.

Provenance activities and relations 352
The provenance graph is formed by explicitly relating entities and activities. This is 353 done by declaring which entities are being used or which entities are being generated by 354 which activities. We currently distinguish four activities: building, calibrating, 355 validating, and analyzing the simulation model. Wet-lab Data for calibration, and Requirements to confirm the calibration results. All 362 connections that we currently distinguish are shown in Table 2.

363
It should be noted that provenance activities in simulation studies can be defined at 364 various levels of granularity. We have opted for a rather coarse-grained approach 365 identifying only crucial activities of a simulation study without explicitly denoting how 366 an activity has used a specific entity. Thus, we aggregate activities as much as possible 367 and leave out intermediate steps, focusing on the entities and not on the activities.
From the moment provenance information is recorded automatically during the course 369 of a simulation study, a higher level of detail could be achieved and an 370 abstraction-based filter could be applied to zoom out to reach our granularity [2].

372
The Building Simulation Model activity, also called model derivation [86], can use all 373 entity types as any entity described above can contribute to the model building process, 374 but it needs to have at least one link to a Research Question or Simulation Model. The 375 only result of the building simulation activity is a Simulation Model entity. Not every 376 update of a simulation model within a simulation study will be reflected in the 377 provenance graph-only those changes to the model that are considered essential by the 378 authors.

379
In our provenance graph of the study of Lee et al. (2003), two Building Simulation 380 Model activities are shown. BSM1 is using wet-lab data, the research question, the 381 qualitative model, a requirement and assumptions to develop a "provisional reference 382 state model" [29], which forms the not yet calibrated simulation model in the study.

383
The Building Simulation Model activity BSM2 extends the simulation model SM2.  Sometimes, switching parts of a model on or off (e.g., individual rules or model 387 components) or choosing between entire models can also be interpreted as a discrete 388 parameter value to be determined using methods of model selection [87]. This activity 389 uses a Simulation Model and typically needs reference data (Wet-lab Data or Simulation 390 Data) for the parameter estimation procedure and produces a specification or 391 documentation of a Simulation Experiment as well as a Simulation Data entity. If the 392 calibration is successful, the result of this activity will always be a (calibrated) 393 Simulation Model. Ideally, it also takes an explicit requirement into account, which, in 394 some cases, if formally defined, can also be used for calibrating the simulation 395 model [88,89]. It may also use an Assumption. The validation of a simulation model is used to test its validity (with regard to some 403 requirements). Unlike calibration activities, here, the result is typically a binary answer, 404 yes or no, which may be determined based on a specific distance measure and error of [19]. This would be of particular relevance if models are not only validated but also 439 accredited, which typically involves a different group of people other than those who 440 have developed the simulation model [92]. 441 We have also not included the direct connection between two activities or two 442 entities, such as the possibility to have a model being derived from another model.

443
Thus, we have not included the following relations: a) WasInformedBy, which relates an 444 activity to another one and b) WasDerivedFrom, which describes a direct 445 transformation (update) of an entity into a new one. However, these relations can partly 446 be inferred via the existing relations. For example, a simulation model that has been 447 generated by a Building Simulation Model activity that used another simulation model 448 indicates that the former has been derived from the latter. Additionally, a validation 449 activity that failed and that is followed by a Building Simulation Model activity 450 obviously holds some information for the latter.

451
In our experience, it is best to capture provenance information manually or 452 (semi-)automatically during the modeling (and simulation) process. This could be done, 453 for example, within an artifact-based workflow system [2]. However, this would rely on 454 a fixed life cycle definition (i.e., constraints regarding the allowed activities). Other 455 approaches are based on automatically analyzing scripts, either with [93] or without [94] 456 the help of user annotations. The latter has potential for a fully automatic and 457 transparent provenance capture, however, it is difficult to implement, highly 458 system-dependent, and only accounts for provenance information contained explicitly in 459 the scripts, thus leaving out important information, such as research question, 460 assumptions, or qualitative models. WebProv , on the contrary, is a standalone tool that 461 works system-independently. Although it requires user input or a valid JSON input file, 462 it also provides great flexibility regarding the information to be captured. Based on the entities and activities that were identified and defined above, we have 466 recorded the provenance information from the 19 studies shown in bold in Table 1 as presentations of the provenance information can also be found on GitHub. We will now 471 discuss the observations we have made during the process of capturing the provenance 472 information and later show how the studies and simulation models relate to each other. 473

Provenance of individual Wnt simulation models 474
It is important to remember that we have manually collected all provenance information 475 (entities, activities and relations), as described in the Materials and methods section.

476
Collecting this information based on publications only is a demanding task and requires 477 some interpretation, as natural language descriptions tend to be ambiguous. Also, the 478 nonlinearity of the text-it is not a lab protocol after all-makes it hard to identify the 479 relations between the entities and activities as well as the order of their execution. This 480 would likely hamper an effective use of text mining or machine learning methods to 481 complement or replace the manual work. Therefore, provenance information should be 482 collected during the simulation study and ideally without an intervention of the modeler. 483 The Research Question was usually repeated within the abstract and throughout the 484 introduction and discussion sections. Sometimes, there was more than one research 485 question to be answered. In this case, we have combined these into a single entity.

486
Many Assumptions were introduced by the word "assume" or its derivatives. Other 487 expressions such as "hypothesis", "is believed", "consider", "approximate", "simplify", 488 "suggest", "suppose", or "propose" were also used by the authors to mark an 489 assumption. However, not every sentence containing one of these words was identified to 490 be an assumption of the simulation model. Occasionally, there were also assumptions 491 which did not use one of the key words from above. Furthermore, two out of 19 studies 492 did not explicitly state assumptions ( [30,54]). Generally, identifying assumptions  In order to further analyze the assumptions, we categorized them using the Systems 499 Biology Ontology (SBO) [68]. However, the assumptions could not always be 500 unambiguously matched to an SBO vocabulary and some assumptions dealing with 501 biological mechanisms are not covered by SBO. For example, an autocrine signaling 502 assumed by Mazemondet et al. (2012) [47] cannot be expressed by SBO. Some 503 assumptions also include more than one detail which is reflected by multiple SBO 504 categories per assumption. In this case, the assumption entity is duplicated and every 505 assumption entity will receive its own category. The categorization of 106 collected 506 assumptions shows that the three most used categories of assumptions deal with kinetic 507 constants (13 times), transport (9 times), and equivalence (8 times). The result of the 508 categorization can be found in S1 Table. 509 In many cases, Requirements were not given explicitly in a formal way or even as

512
(2020) [57]. The lack of Requirements was especially obvious when calibration or 513 validation experiments were carried out without explicitly explaining the objective 514 function.

515
In the surveyed publications, it was common to include a reaction scheme of the 516 simulation model, which we referred to in the Qualitative Model entities. When 517 recording all species, we disregarded di-or multimeric compounds established by 518 monomers already mentioned. We also ignored different states of the species (e.g., 519 phosphorylation states). In all provenance graphs but the one by Mirams et al.

523
The Simulation Models were either part of the manuscript or, more often, part of the 524 supporting material. In 13 out of 19 cases, it was a system of ordinary differential 525 equations. There were two simulation studies using PDEs and four using a rule-based 526 approach (see Table 1). Although the Wnt signaling pathway has also been used to 527 illustrate features of rule-based modeling [95,96], only few simulation models have been 528 developed based on a rule-based approach. The reason for this might be partly because 529 support for thorough experimentation with rule-based models including calibration and 530 validation has only become available during the last decade [97][98][99]. 531 We categorized all 145 Simulation Experiments that we found depending on which 532 experiment type they served (see UML class diagram shown in Fig 3). The results of 533 the categorization are shown in Fig 5 and  Experiments were parameter scans, followed by time course analyses and perturbations. 535 None of the 19 simulation studies have used steady-state analysis alone without 536 applying another type of experiment at the same time, which we then recorded because 537 it was more specific. The detection of steady states is typically part of an optimization, 538 parameter scan, perturbation, and sensitivity analysis, because steady-state values are 539 often the starting and end point of each simulation and are used for calculations.  accessible, interoperable and reusable [101].

551
In the case of Simulation Data, the focus lies on FAIR simulation models and 552 experiments as it should be possible to easily regenerate the data. This could be 553 achieved, for example, by publishing a COMBINE archive [61], which is a "single file 554 that aggregates all data files and information necessary to reproduce a simulation study 555 in computational biology" [102]. However, the publications we have analyzed date back 556 up to 17 years, so most data has not been published in a FAIR way.

578
Some studies develop a story line where a simulation model has been successively 579 refined, extended, or composed ( [29,34,35,37,42,47,49,50,52,57]). These studies are  Some simulation studies resulted in the development of multiple simulation models 583 that are neither extensions nor compositions but rather form a revision or alternative to 584 other simulation models developed in the same study: [34,40,42,47,49,50]. This can be 585 seen in the provenance graph of a single study when the last simulation model is not 586 connected by a directed path to other simulation models of that study or when the 587 simulation models are part of disjoint branches of the provenance graph. For example, 588 in Mazemondet et al. (2012) [47], the core model of the Wnt/β-catenin signaling 589 pathway has been calibrated with wet-lab data from Lee et al. (2003) [29], but this 590 calibrated simulation model was not used and instead, a new calibration with wet-lab 591 data from another study took place. Two simulation studies ( [36,42]) show 592 disconnected graphs. This shows that these researchers considered, built, and analyzed 593 multiple simulation models independently of each other.

594
Beyond individuals: A family of Wnt simulation models 595 Whereas before we have looked at the properties of individual simulation studies, we are 596 now going to investigate the interrelations between the 19 Wnt/β-catenin signaling 597 simulation models. We will identify features that transform a set of simulation models 598 into a family.   [29] x [30] x [31] x [32] x [33] o x [34] x x [35] x [36] x x x [37] x x [40] o [42] x [46] x x [47] x x [49] o x [50] x x x [52] x [58] x The top row shows the origin of the cells used by the 19 studies. Not applicable means that the parameters values were obtained from theoretical calculations/considerations from a textbooks or without concrete reference to a wet-lab study. The colors denote the organisms of the cell line/organ used, with Xenopus, mouse, human, hamster, kangaroo rat, and rat. The "x" denotes directly used, the "o" denotes indirectly used by using (parts of) a referenced simulation model. pathway. 613 We have also included the cell lines/tissue used for wet-lab experiments within each 614 study. As already seen in Table 3, we observe that different simulation studies use data 615 or models obtained using other cell lines. This may be valid as the Wnt/β-catenin 616 signaling pathway is evolutionarily conserved [24], which means that data can be shared. 617 Still, care must always be taken when using, for instance, parameter values determined 618 with one cell line in a study that uses another cell line.

619
When looking at the same graph using a circular layout, we observe four clusters of 620 two or more studies, as shown in S1 Figure. We have also colored the studies according 621 to additional pathways they include and observe that the clusters separate the studies 622 depending on these additional cellular mechanisms. The central cluster includes the 623 Wnt model by Lee et al. (2003) [29] as well as the studies [31,35,36,42]. A second 624 cluster forms around the simulation studies [46,47,52,57,58] and either includes the 625 same wet-lab data from [104,105], the Cell cycle [47] or ROS [52,58]. A third cluster 626 includes the pathways of Notch [34,49] and Notch + MAPK/ERK [37]. Even though 627 the algorithms locate [30] in the same cluster, it is content-wise rather part of the 628 central cluster. A forth cluster forms around studies that include MAPK/ERK [33] or 629 MAPK/ERK + PI3K/Akt [54]. All other models are not part of a cluster and are 630 either completely disconnected from the other studies [32] or include E-cadherin and the 631 cell cycle [40] or just E-cadherin [50]. Model, Simulation Experiment, Simulation Data, and Wet-lab Data have been identified 646 as crucial entities of simulation studies [20]. Additionally, we have taken knowledge of 647 modeling and simulation life cycles [1] into account and identified the Research Question, 648 Assumptions, Requirements, the Qualitative Model to be important ingredients of the 649 provenance of simulation models. We also distinguish between Building Simulation Simulation Model activities and connect the entities and activities by using the relations 652 wasGeneratedBy and used. In our definitions of the entities and activities, we aimed at 653 achieving the minimal level of detail, or granularity, of the provenance graph to 654 understand the course of a simulation study. We also kept the necessary metadata of 655 the entities and activities to a minimum to convey both the main idea of the simulation 656 study and the content of each entity and activity. For storing, visualizing, and querying 657 the provenance information, we have created the web-based tool WebProv that allows 658 for each entity and activity to store (customized) metadata and references.

659
In order to examine our specialization of PROV-DM, the extensive analysis of 19 660 Fig 6. Provenance graph of all Wnt/β-catenin simulation studies considered here (black outlines) as well as additionally used studies (gray outlines). The colors indicate the cell lines used in wet-lab experiments of a study. Gray boxes represent pure Wnt/β-catenin simulation studies without acquiring wet-lab data. White boxes display publications used by some of the Wnt/β-catenin simulation studies that are either text books or simulation studies without published wet-lab data. survey. Thus, a family of Wnt signaling models could be revealed.

666
In conclusion, provenance information provides added value to the existing list of 667 documentation requirements and could complement and enrich the effort of 668 "harmonizing semantic annotations for computational models in biology" [107].

669
Together with the exploitation of community standards and ontologies, provenance 670 information opens up further possibilities of reusing and analyzing simulation models, 671 for example, to help with model selection, model merging, or model difference detection. 672 Of course, to be fully accepted, our specialization of PROV-DM should be subject to a 673 standardization initiative. We think that WebProv , or a similar tool, would be a