You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
These standards promote the reuse of model code and reproducibility of model results.
12
+
These standards promote the reuse of model code and reusability of model results.
13
13
{{% /standards-preamble %}}
14
14
15
-
## Overview of Reproducibility Standards
15
+
## Overview of Reusability Standards
16
16
17
-
In this document we are adopting the [National Academies of Sciences, Engineering and Medicine 2019 report on Reproducibility and Replicability in Science](https://doi.org/10.17226/25303) definitions of reproducibility and replicability.
17
+
In this document we are adopting the [National Academies of Sciences, Engineering and Medicine 2019 report on Reusability and Replicability in Science](https://doi.org/10.17226/25303) definitions of Reusability and replicability.
18
18
19
-
> Reproducibility means obtaining consistent computational results using the same input data, computational steps, methods, code, and conditions of analysis. Replicability means obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data.
19
+
> Reusability means obtaining consistent computational results using the same input data, computational steps, methods, code, and conditions of analysis. Replicability means obtaining consistent results across studies aimed at answering the same scientific question, each of which has obtained its own data.
20
20
21
-
Reproducibility is a cornerstone of scientific research, especially for transparent evaluation of research claims. Equally important is the potential for scientific procedures to be reused and modified in order to carry out new research. While clear documentation is essential to reusability, new technologies also offer the potential to increase the reproducibility and reusability of model code.
21
+
Reusability is a cornerstone of scientific research, especially for transparent evaluation of research claims. Equally important is the potential for scientific procedures to be reused and modified in order to carry out new research. While clear documentation is essential to reusability, new technologies also offer the potential to increase the Reusability and reusability of model code.
22
22
23
-
A critical metric of reproducibility is whether a model can consistently reproduce the published results that claim to have been generated by the model. Reproducibility and reusability can also be enhanced if model code is accompanied by explicit workflows that demonstrate how the model generates and transforms its output data into its published findings.
23
+
A critical metric of Reusability is whether a model can consistently reproduce the published results that claim to have been generated by the model. Reusability and reusability can also be enhanced if model code is accompanied by explicit workflows that demonstrate how the model generates and transforms its output data into its published findings.
24
24
25
-
## Goals for Reproducibility Standards
25
+
## Goals for Reusability Standards
26
26
27
-
An independent reviewer should be able to easily download, build, and execute a computational model and verify that it meets the reproducibility claims stated by the model author(s).
27
+
An independent reviewer should be able to easily download, build, and execute a computational model and verify that it meets the Reusability claims stated by the model author(s).
28
28
29
-
## Minimal Reproducibility Standards
29
+
## Minimal Reusability Standards
30
30
31
-
This set of minimal standards also can be adopted by journals to ensure that submitted publications meet baseline reproducibility and reusability requirements.
31
+
This set of minimal standards also can be adopted by journals to ensure that submitted publications meet baseline Reusability and reusability requirements.
32
32
33
33
Computational models must:
34
34
@@ -41,14 +41,14 @@ Computational models must:
41
41
- software with large compute or data requirements can be problematic:
42
42
- provide representative input data samples along with sampling methodology
43
43
- provide durable link to wholetale or other provenance-tracked computation
44
-
- clearly describe what reproducibility measures or metrics are applicable to the given software
44
+
- clearly describe what Reusability measures or metrics are applicable to the given software
45
45
- provide range of acceptable outcomes for software with stochastic components
46
46
- provide descriptions of possible input parameters and expected outputs with units and format (e.g., shape, data type, etc.)
47
47
- include output analyses and workflows
48
48
- scripts to transform raw data -> intermediate -> figures
49
49
- subsets of intermediate data if prohibitive to generate
50
50
51
-
## Ideal Reproducibility Standards
51
+
## Ideal Reusability Standards
52
52
53
53
In order to meet the ideal standards, computational models should:
54
54
@@ -58,7 +58,7 @@ In order to meet the ideal standards, computational models should:
58
58
- include metadata on related research outputs (publications, other software, relationship)
59
59
- additional domain specific standards if any (examples?)
60
60
61
-
## Cyberinfrastructure and Tools for Implementation of Reproducibility Standards
61
+
## Cyberinfrastructure and Tools for Implementation of Reusability Standards
OMF may consider building some github template repositories or scaffolding for common modeling frameworks that reduce friction of adoption.
75
75
- e.g., https://github.com/uwescience/shablona and https://github.com/geodynamics/software_template
76
76
- a GitHub bot that submits PRs against a GitHub repository to improve compliance with minimal / ideal standards
77
-
- e.g. provide a cookiecutter project structure that supports best practices for reproducibility and reusability like [Cookiecutter Data Science](http://drivendata.github.io/cookiecutter-data-science/)
77
+
- e.g. provide a cookiecutter project structure that supports best practices for Reusability and reusability like [Cookiecutter Data Science](http://drivendata.github.io/cookiecutter-data-science/)
78
78
79
-
## Examples and References for Reproducibility
79
+
## Examples and References for Reusability
80
80
81
81
-[Lorena Barba's reproducible workflow for computational fluid dynamics](https://doi.org/10.5281/zenodo.2642710)https://github.com/barbagroup/cloud-repro
0 commit comments