This is the environment file as it stands today, not a record of this run. The run predates environment capture, so what it actually had on hand was never recorded and cannot be recovered.

# Find brief — prior art, the Institute for Replication

**Environment:** `night-shift-default`. The settings are on this mission's page.

This run was not a broad sweep. It was one question, asked because the answer
changes what the board should build:

> I4R is publicly building an "AI Replication Engine" — three agents that
> re-execute research code and compare against published results. Does it exist
> in a form Night Shift can adopt, instead of building a parallel one? And is
> there anything else in their public work worth taking?

Two rules carried over from FIND-002 and FIND-003, because they are what make a
row trustworthy:

- **Never invent a paper, a title, or a package URL.**
- **`package_read: yes` only if the package was actually opened in this run.**
  Found-by-search is not opened.

Report what could not be reached, by name. A blocked host is a finding, not an
omission to be quietly rounded away.

(2026). A randomised study of how team composition affects reproducibility
assessment: teams were assigned to **Human-Only**, **AI-Assisted**, or **AI-Led**
branches and set to work reproducing quantitative social-science papers.

**Package.** https://github.com/I4Replication/AI-Games — **opened.** R, MIT
licence for code and CC-BY-4.0 for data. The repo carries `data/` (including
`AI games.xlsx`, prompts information, comparison papers, sentiment data),
`code/` (19 scripts plus `master.R`), `output/`, and a `focus groups/` folder of
qualitative material. The README gives a single entry point — `source("code/master.R")`
from the repository root — and the master script installs packages, creates
folders, cleans, and runs the analysis. Requires R >= 4.4.0 and 22 packages via
`pacman::p_load()`. No runtime is stated.

**Why it is worth reproducing.** All input data ships in the repo, so nothing is
gated: this is a complete package on public data with one command to run it,
which is the profile the board most wants. It is also the corpus I4R has said it
will benchmark its own engine against, so a reproduction here is legible to the
people building the comparison. And the subject matter is the board's own:
reproducing the paper about whether AI can assess reproducibility.

**Completeness.** `verified_complete: yes` — from the README, the file tree, and
reading `code/cleaning.R`, not from executing anything. The usual caveat holds:
pages were read through a fetch-and-summarise tool rather than a clone.

---

# Prior art — the Institute for Replication

**Prepared:** 2026-08-19.
**Method in one line:** open I4R's public GitHub organisation, read every repo
that could plausibly be tooling, and report what is actually there.

> **Read this before you trust the schema section below.** The reproducibility
> assessment form this run went looking for lives on **osf.io**, which the
> egress proxy refused. The form's field structure was recovered indirectly —
> from the variable names, factor levels and recodes in the study's own
> `code/cleaning.R`, which reads the teams' responses. That is a faithful
> reconstruction of *what the form captured*, not a reading of the form itself:
> exact question wording and scale anchors are unverified. Anyone confirming it
> should open `osf.io/sz2g8/` from a network that can reach it.

---

## 1. AI, Humans, and Reproducibility (Brodeur et al., PNAS 2026)

**Paper.** Brodeur et al., *Proceedings of the National Academy of Sciences*
(2026). A randomised study of how team composition affects reproducibility
assessment: teams were assigned to **Human-Only**, **AI-Assisted**, or **AI-Led**
branches and set to work reproducing quantitative social-science papers.

**Package.** https://github.com/I4Replication/AI-Games — **opened.** R, MIT
licence for code and CC-BY-4.0 for data. The repo carries `data/` (including
`AI games.xlsx`, prompts information, comparison papers, sentiment data),
`code/` (19 scripts plus `master.R`), `output/`, and a `focus groups/` folder of
qualitative material. The README gives a single entry point — `source("code/master.R")`
from the repository root — and the master script installs packages, creates
folders, cleans, and runs the analysis. Requires R >= 4.4.0 and 22 packages via
`pacman::p_load()`. No runtime is stated.

**Why it is worth reproducing.** All input data ships in the repo, so nothing is
gated: this is a complete package on public data with one command to run it,
which is the profile the board most wants. It is also the corpus I4R has said it
will benchmark its own engine against, so a reproduction here is legible to the
people building the comparison. And the subject matter is the board's own:
reproducing the paper about whether AI can assess reproducibility.

**Completeness.** `verified_complete: yes` — from the README, the file tree, and
reading `code/cleaning.R`, not from executing anything. The usual caveat holds:
pages were read through a fetch-and-summarise tool rather than a clone.

---

## Notes on this run

**The engine is not public.** I4R's organisation holds four public repos:
`AI-Games` (R, the package above), `utilities` (Python, "several utility
functions to help our processes"), `AI-vertical` (HTML), and
`I4R-First-Meta-Paper` (Stata). There is **no AI Replication Engine, no agent
framework, no reproduction toolkit**. Their public GitHub supports
human-conducted replication; the engine described in their blog posts and grant
applications is not released. Build-vs-adopt therefore stays open rather than
resolved — with the note that they have said a toolkit is anticipated later in
2026.

**The study's own result is a finding for this board.** Brodeur et al. report
that **AI-assisted teams outperform AI-led teams, but not human-only teams** —
AI works best as a collaborative tool rather than an autonomous agent. That is
direct evidence about the thing Night Shift is deciding how far to automate, and
it favours the architecture already in place: blind two-reader extraction,
reconciliation by script, verdicts assigned maintainer-side, a human merge. The
line to hold is that an agent may produce numbers but must not certify its own
verdict.

**I4R's error taxonomy — the most useful thing found.** `cleaning.R` reads the
teams' responses under these fields, which together are the classification the
form captured:

- `x1_major` — major errors
- `x2_minor` — minor errors
- `x3_pre_regression` — errors before the regression stage
- `x4_regression` — errors in the regression itself
- `x5_transcription_post_regression` — transcription errors after the regression
- `x6_false_positive` — a reported error that turned out not to be one
- `x7_error_known_by_i4r` — the error was already known

alongside the reproduction outcome (`reproduction`, `time_reproduction`) and
robustness (`ran_robustness`, `good_robustness`).

This is an axis the board does not currently have. Night Shift's
`inclusion_rule` classes (target / restatement / parameter / excluded) answer
*which numbers count*; its three-state schema answers *did it reproduce*. Neither
answers *what kind of error this was*. Two of I4R's categories are worth adopting
regardless of the rest: `x6_false_positive`, which is exactly the harm the
second-run gate exists to prevent and is currently tracked nowhere, and
`x7_error_known_by_i4r`, which separates a novel finding from a rediscovery.

**Reached:** github.com, raw.githubusercontent.com, i4replication.org.
**Refused by the proxy:** osf.io (the template form and AI training materials),
arxiv.org (the ReproRepo and ETH papers), www.builder.io. The OSF block is the
one that cost something, and the workaround is stated at the top of this file
rather than hidden.