ENCORE: a practical implementation to improve reproducibility and transparency of computational research
Antoine H. C. van KampenUtkarsh MahamuneAldo JongejanBarbera D. C. van SchaikDaria BalashovaDanial LashgariMia L. Pras‐RavesEric WeverAdriaan D. DaneRodrigo García-ValientePerry D. Moerland
1
Citation
86
Reference
10
Related Paper
Citation Trend
Abstract:
Reproducibility of computational research is often challenging despite established guidelines and best practices. Translating these guidelines into practical applications remains difficult. Here, we present ENCORE, an approach to enhance transparency and reproducibility by guiding researchers in how to structure and document a computational project. ENCORE builds on previous efforts in computational reproducibility and integrates all project components into a standardized file system structure. It utilizes pre-defined files as documentation templates, leverages GitHub for software versioning, and includes an HTML-based navigator. ENCORE is designed to be agnostic to the type of computational project, data, programming language, and ICT infrastructure, and does not rely on specific software tools. We also share our group's experience using ENCORE, highlighting that the most significant challenge to the routine adoption of approaches like ours is the lack of incentives to motivate researchers to dedicate sufficient time and effort to ensure reproducibility.Keywords:
Best practice
Abstract Background Accurate repeat assessment of the diameter of an abdominal aortic aneurysm (AAA) is important. This study investigated the reproducibility of different methods of measuring AAA diameter from ultrasound images. Methods Fifty AAA patients were assessed by ultrasound. Maximum AAA diameter was measured independently by three trained observers on two separate occasions using a standardised protocol. Five diameters were measured from each scan, three in the anterior–posterior (AP) and two in the transverse (TV) plane, including inner-to-inner (ITI), outer-to-outer (OTO) and leading edge-to-leading edge (LETLE). Intra- and inter-observer reproducibility were reported as reproducibility coefficients. Statistical comparison of methods was performed using linear mixed effects models. Results Intra-observer reproducibility coefficients (AP LETLE 2.2 mm; AP ITI 2.4 mm; AP OTO 2.6 mm) were smaller than inter-observer reproducibility coefficients (AP LETLE 4.6 mm: AP ITI 4.5; and AP OTO 4.8 mm). There was no statistically significant difference in intra-observer reproducibility of three types of measurements performed in the AP plane. Measurements obtained in the TV plane had statistically significant worse intra-observer reproducibility than those performed in the AP plane. Conclusions This study suggests that the comparison of maximum AAA diameter between repeat images is most reproducibly performed by a single trained observer measuring diameters in the AP plane.
Cite
Citations (14)
Abstract We checked on reproducibility of parameters of a multichannel electrogastrogram in adults after intake of typical, applied in electrogastrography, test meals. Recordings of multichannel electrogastrograms were accomplished in four blocks comprising 18 subjects (nine healthy volunteers and nine patients with functional GI disorders) each. Every subject had two examinations taken 1‐2 days apart, and a third one was accomplished at least 2 weeks before or after the two other sessions. The registration involved a 30‐min fasted and a 2‐h postprandial period after one of the meal stimuli tested within a given block: 400 mL water, 400 g yoghurt (378 kcal), a scrambled eggs sandwich (370 kcal), a pancake (355 kcal). From among the parameters reflecting the propagation of the gastric slow waves, the average percentage of slow wave coupling (APSWC) exhibited a good (coefficient of variation for paired examinations CV p ≤ 10%) to moderate (10< CV p ≤30%) reproducibility. On the other hand, the reproducibility of the maximum dominant frequency difference and the spatial dominant power difference was found to be unsatisfactory. The reproducibility of the multichannel electrogastrographic parameters did not differ between healthy volunteers and patients with functional GI disorders. Gender or the kind of a test meal did not affect the reproducibility of the electrogastrographic parameters either. The medium‐term reproducibility was not any worse than the short‐term one. From among the parameters of a multichannel electrogastrogram intended to quantify the propagation of slow waves, only the APSWC offers a reproducibility potentially good enough for clinical applications.
Electrogastrogram
Coefficient of variation
Cite
Citations (24)
Being transparent about data sharing can help alleviate student concerns, said officials at the Department of Education's Privacy Technical Assistance Center. They presented a webinar on transparency best practices and released guidance in a document titled “Transparency Best Practices for Schools and Districts.” Although the guidance was directed mainly at K–12 officials, much of it applies to higher education, said Baron Rodriguez, PTAC director, speaking during the webinar. FERPA requires institutions to provide an annual notification to students. In addition, PTAC recommends that, as a best practice, institutions provide the following information about their privacy policies.
Best practice
Data Sharing
Cite
Citations (0)
Coefficient of variation
Cite
Citations (20)
Cite
Citations (0)
Objective To compare the reproducibility of probing depth (PD) measurements using the Florida probe and the conventional periodontal probe.Methods Before and after initial periodontal treatment, five patients with chronic periodontitis received repeated PD measurements using the Florida probe and the conventional periodontal probe, respectively. The reproducibility of PD measurements was compared between the two different probes. The reproducibility of PD measurements using the same probes was also compared before and after the treatment.Results The reproducibility of PD measurements with both of these probes was good (P~0.05 , r=~0.77 ～~0.85 ) for both before and after initial treatment. Before the treatment, the reproducibility of these two probes was similar. After treatment, the reproducibility of the Florida probe was better than that of conventional periodontal probe (P~0.05 ).Conclusion The reproducibility of PD measurements with the Florida probes is reliable, suggesting that it can be applied in clinical research.
Periodontal probe
Cite
Citations (0)
Finally, we are at the last chapter of this book. Terraform offers multiple ways to implement configurations. Technically, they are all valid solutions, and Terraform will not complain about which one you use. However, it is important to evaluate whether the chosen technique for configuration implementation is generic, promotes reusability, is extensible, is secure, and promotes collaboration. This chapter is about implementing Terraform configurations using best practices. This chapter will cover the practices to employ when using various features of Terraform. It will provide best practices related to providers, versioning, variables, outputs, authentication, and many other aspects.
Best practice
Reusability
Extensibility
Cite
Citations (0)
The purpose of this paper was to assess the reproducibility of three simulations presented on videotape: echocardiograms, ventriculograms, and arteriograms. The results indicate that roughly comparable numbers of cases, but different amounts of testing time, are required to achieve specified levels of reproducibility; score interpretation affected reproducibility in the expected ways.
Cite
Citations (1)
This three‐part study investigated the reproducibility of natural head posture (NHP) using radiographs and photographs. In part 1, reproducibility of cephalograms 1 year after the introduction of NHP was investigated and found to be less favourable (Dahlberg coefficient of 2.99 degrees) than most other previous investigations. In order to minimize radiation exposure of patients, reproducibility of photographs and method agreement between photographs and cephalograms were investigated in part 2. Reproducibility of the two photographs was poor (2.71 degrees). However, method agreement between cephalograms and the photographs taken at the same time was good (1.39 degrees). Replacement of the radiographic method with the photographic method for assessing NHP reproducibility appeared justified. Changing the protocol for achieving NHP in part 3 of the study improved reproducibility substantially (1.41 degrees).
Repeatability
Coefficient of variation
Cite
Citations (69)