Would your analysis survive new data?

Most research is felled at harvest. The paper goes out, and the code, data and decisions that grew it are left to go cold. When new data arrive, the next version starts from seed.

Eight questions place your analysis on the ladder from finished paper to living pipeline. Pick one that matters, a paper, a recurring report or a model, and answer for how it actually is. Nothing is recorded until you send it.

1. Q1Could you recover exactly the code behind your published results?

   I think so. The scripts are in a folder somewhere.Yes. I kept a copy of the folder as it was on submission.Yes. It is a commit in version control, and I know which one.Yes. It is tagged as the published version.
2. Q2Where do the code and data live?

   On my laptop, and in a few email attachments.On a shared drive, with dated copies.The code is in git. The data sit beside it.Code in git, and each data version recorded with where it came from.
3. Q3Pick one figure in your report. What produced it?

   I would have to open a few scripts to find out.A named script, run after the others in the right order.One command, which runs everything in order.The manifest says, from the raw data to that figure.
4. Q4A source revises three years of its history. Which outputs are now stale?

   All of them, to be safe. I would redo everything.I would know the main ones from memory.I rerun everything and compare old outputs with new.The pipeline tells me which are out of date, and rebuilds only those.
5. Q5A colleague must rerun it from scratch next week, without you. How does it go?

   They would need to ring me, more than once.They could manage with the README and a free afternoon.They clone it, run one command, and get the same figures.Someone else already has, and the numbers matched.
6. Q6Which package versions produced the published results?

   Whatever I had installed at the time.I could list the packages, but not the versions.An environment file records the versions.A lockfile or container, rebuilt regularly to prove it still works.
7. Q7Before results go out, what checks them?

   I look over the tables and figures.A second person reads the numbers against the tables.Scripted checks on counts, totals and ranges, which I run before sending.Checks run on every change, and a failure stops the build.
8. Q8The paper is out. What happens to the analysis now?

   It is done. The next project starts fresh.It is archived with the paper, in case reviewers ask.It could be rerun if someone asked for an update.It is rerun when new data arrive, and the paper can follow.

The button opens an email to us, copied to Rabbie, our robot, who keeps the diary. Your result travels in it so we can prepare. You can change or delete it before you send, and nothing leaves your browser until you do. Science Supercharged rebuilds one analysis to run again from source, with its manifest, and the knowledge tree explains the idea in four figures.

Back to Science Supercharged →