The point of an audit is not applause; it is to find what breaks. We would rather publish “a reviewer showed claim X is weaker than stated, here is the correction” than leave it unchecked. Findings — confirmations, corrections, or refutations — will be published on this site with attribution (or anonymously, your choice), alongside what changed as a result.
The three claims most worth attacking
1 · The origin gap: “the top exporter is not the top miner” (17 of 32)
Where: findings. Data: reconciled CEPII BACI trade vs USGS mine shares. Reproduce: python build_origin_gap_units.py. Robustness already run: the gap holds measured tonnes-vs-tonnes (17/17 survive), and in every case the top exporter is a genuine refiner, not a re-export hub (0/17). Best attack: we bound the transshipment artefact but do not fully trace ore-to-refined product-form differences; quantify how much of a given gap is refining vs. a customs product-code change.
2 · The mechanism taxonomy: “only 4 of 58 chokepoints are geological”
Where: Chokepoint Map + Report 04. What it is: a classification of each chain’s binding stage into seven mechanisms from process physics. Best attack: these are assigned judgments, not statistical inferences, and carry no uncertainty. Challenge specific assignments, or propose a mechanism the taxonomy misses; the classifier is a single JSON field per chain, so a re-classification is a one-line change to test.
3 · The nowcast: 84% out-of-sample persistence, engine forward-test pending
Where: nowcast backtest. Reproduce: python build_nowcast_backtest.py. Known limit (stated): 84% is a high base rate (top exporters rarely change), and the model has ~no skill at year-over-year direction (42%). The reconciliation engine’s skill over persistence is not yet validated — a 2025 forward test is pre-registered. Best attack: is persistence the right baseline; are the material-years independent; what are the confidence intervals; does the engine’s ~1.8× level bias vs official BACI bias any share-based claim?
Weaknesses we already know about
How to contribute
Everything is public and reproducible. The whole reconciliation engine runs in CI without an API key.
- Open an issue or pull request on the public repository: github.com/materials-atlas/critical-materials-atlas — the fastest path for a specific correction.
- Prefer to write it up? A short adversarial memo (what you checked, what held, what didn’t) is exactly what would raise this from “careful solo project” to reviewed research. It will be published here with your name or anonymously, alongside the resulting changes.
- Already looked and found nothing wrong? Say so — a named “checked X, it holds” is worth as much as a correction.