Reproducibility and Transparent Analysis

Overview

A reproducible analysis allows another knowledgeable analyst to use the same data, code and computational environment to obtain the same reported results.

Transparent analysis also makes clear:

• what was planned;
• what changed;
• how data were transformed;
• which software was used;
• how tables and figures were generated;
• what can and cannot be shared.

Reproducibility is not proof that the study is valid. A biased analysis can be reproduced exactly. Transparency makes the evidence easier to verify and critique.

Why this matters

Non-reproducible workflows commonly rely on:

• manually edited spreadsheets;
• undocumented recoding;
• copied values between software and manuscripts;
• overwritten raw data;
• unknown package versions;
• unrecorded random seeds;
• analysis files named final_final_revised2;
• tables edited after export.

These practices create errors that may not be visible in the final paper.

A script-based workflow creates an auditable path:

raw data → cleaned data → analysis → tables and figures → manuscript

When to use this framework

Use the framework for every quantitative project, including:

• student research;
• clinical trials;
• observational analyses;
• laboratory studies;
• systematic reviews;
• prediction models;
• routine reports.

The level of public sharing depends on consent, ethics, law, contracts, privacy and intellectual-property requirements.

Reproducibility, robustness and replication

Computational reproducibility

Same data and analysis procedures produce the same numerical results.

Robustness

The conclusion remains similar under reasonable alternative specifications or assumptions.

Replication

Independent data and procedures provide evidence relevant to the original claim.

Terminology varies across disciplines, so authors should define what they mean rather than using “reproducible” vaguely.

Project structure

telemonitoring-project/
├── README.md
├── LICENSE
├── CITATION.cff
├── data/
│   ├── raw/
│   │   └── trial_export_2026-07-01.csv
│   ├── metadata/
│   │   ├── data_dictionary.csv
│   │   └── missing_codes.md
│   └── derived/
├── R/
│   ├── 01_import.R
│   ├── 02_clean.R
│   ├── 03_describe.R
│   ├── 04_primary_analysis.R
│   ├── 05_sensitivity_analysis.R
│   └── 06_tables_figures.R
├── output/
│   ├── tables/
│   └── figures/
├── manuscript/
│   └── article.qmd
├── renv.lock
└── session-info.txt

Immutable raw data

Recommended practices:

• preserve the original export;
• store its acquisition date;
• record a checksum;
• restrict write access;
• never add manual correction columns to the raw file;
• document source-system updates.

If a source error is corrected, retain the original and record the correction in code or a formal query log.

Data dictionary

A data dictionary should contain:

• variable name;
• label;
• type;
• unit;
• allowed values;
• missing codes;
• derivation;
• collection time point;
• source form;
• confidentiality classification.

A code such as 9 can mean missing, severe disease or site 9. Meaning must not be inferred from the value alone.

Scripted cleaning

raw <- read.csv(
  "data/raw/trial_export_2026-07-01.csv",
  na.strings = c("", "NA", "999")
)

clean <- transform(
  raw,
  group = factor(
    group_code,
    levels = c(0, 1),
    labels = c("Usual care", "Telemonitoring")
  ),
  controlled = as.integer(
    sbp_week12 < 140 & dbp_week12 < 90
  )
)

The validation checks stop the workflow when an assumption is violated.

One source of truth

The manuscript should not contain manually typed statistical values when they can be generated from analysis objects.

For example, a Quarto document can calculate:

• sample size;
• effect estimate;
• confidence interval;
• p-value;
• table;
• figure.

Updating the data then updates the outputs consistently.

Computational environment

Record:

• R version;
• operating system;
• package versions;
• locale;
• random-number generator;
• external dependencies.

In R:

sessionInfo()

A project library manager such as renv can record package versions in renv.lock.

Containers can capture a broader environment but add complexity and maintenance requirements.

Random seeds

Set and record a seed for stochastic procedures:

set.seed(20260803)

Examples include:

• bootstrap resampling;
• multiple imputation;
• cross-validation folds;
• simulation;
• random forests;
• stochastic optimisation.

A seed alone is not enough if software versions or algorithms change.

Version control

Git records:

• file changes;
• authorship;
• timestamps;
• branches;
• tagged releases.

Useful practices include:

• small meaningful commits;
• descriptive messages;
• no identifiable data in the repository;
• tagged manuscript releases;
• issue tracking for analysis decisions.

Version control does not replace secure backup.

Analysis-decision log

Document decisions such as:

• exclusion rules;
• outcome corrections;
• transformation changes;
• model deviations;
• additional sensitivity analyses;
• reasons and dates;
• who approved them;
• whether made before or after outcome review.

This helps distinguish prespecified from data-responsive analysis.

Data and code sharing

The TOP Guidelines provide a current policy framework for transparency practices such as registration, data sharing, code sharing and verification [1].

Sharing options include:

• public de-identified data;
• controlled-access repository;
• synthetic data plus code;
• metadata and data dictionary only;
• code without restricted data;
• formal data-use application.

“Data available on reasonable request” should explain:

• who can request;
• criteria;
• governance;
• restrictions;
• expected process.

FAIR principles

The FAIR principles encourage digital research objects to be:

• findable;
• accessible;
• interoperable;
• reusable [2].

FAIR does not necessarily mean open. Sensitive data can be findable through metadata and accessible through controlled procedures.

Worked example

Automated table example

The statistical calculations producing these fixed example values are not shown; the code demonstrates automated formatting and export.

estimate <- -5.9
lower <- -8.5
upper <- -3.3
p_value <- 0.000012

report_row <- data.frame(
  Outcome = "12-week systolic blood pressure",
  Effect = sprintf(
    "%.1f mmHg (95%% CI %.1f to %.1f)",
    estimate,
    lower,
    upper
  ),
  P = ifelse(
    p_value < 0.001,
    "<0.001",
    sprintf("%.3f", p_value)
  )
)

write.csv(
  report_row,
  "output/tables/primary-result.csv",
  row.names = FALSE
)

The table is generated from variables rather than typed independently.

Quality checks

Artificial intelligence tools

Current ICMJE recommendations state that AI technology used to conduct the study should be described in sufficient detail to enable replication, including tool, version and prompts where applicable [3].

AI-generated code requires:

• human verification;
• test cases;
• source checking;
• documentation;
• privacy review;
• version tracking.

Do not upload identifiable or contract-restricted data to unapproved systems.

Robustness and reproducibility

A reproducible result can remain sensitive to:

• model form;
• exclusion rule;
• missing-data assumption;
• outcome definition;
• seed;
• preprocessing.

Include prespecified sensitivity analyses and make their code available where feasible.

Interpreting transparency

Presenting reproducibility practices

Suggested reporting

Transparent-analysis statements should identify what is available, where it is available, which version matches the article and why any material cannot be shared. Computational-research guidance recommends recording every analysis step and avoiding manual manipulation [4]. The R software version used for the archived analysis should also be stated explicitly [5].

Common mistakes

Knowledge check

Key take-home messages

References

1. Center for Open Science. Transparency and Openness Promotion Guidelines. TOP Guidelines 2025. Charlottesville (VA): Center for Open Science; 2025. https://www.cos.io/initiatives/top-guidelines
2. Wilkinson MD, Dumontier M, Aalbersberg IJJ, Appleton G, Axton M, Baak A, et al. The FAIR guiding principles for scientific data management and stewardship. Sci Data. 2016;3:160018. https://doi.org/10.1038/sdata.2016.18
3. International Committee of Medical Journal Editors. Recommendations for the conduct, reporting, editing, and publication of scholarly work in medical journals: preparing a manuscript for submission. Updated 2026. https://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-for-submission.html
4. Sandve GK, Nekrutenko A, Taylor J, Hovig E. Ten simple rules for reproducible computational research. PLoS Comput Biol. 2013;9(10):e1003285. https://doi.org/10.1371/journal.pcbi.1003285
5. R Core Team. R: a language and environment for statistical computing. Version 4.6.1. Vienna: R Foundation for Statistical Computing; 2026. https://www.R-project.org/