BENCHMARK · SNAPSHOT ri-20261025-1526
Does the judgment hold up?
If a judgment is the product, its error rate is part of the product. These are all 28 cases from one run against the live endpoint — including the ones it abstained on and the one control we could find.
01 METHOD
The labels are not our opinion; they are computed from the snapshot by construction:
- A — the paper itself carries an A-tier notice (a retraction registered against its DOI). Expected verdict: CONTESTED.
- B — one of its references is retracted. Expected: CONTESTED, or an abstention. Calling it SUPPORTED is a false negative.
- C — no retracted work within two citation hops. Only SUPPORTED counts as correct.
Every case is a real call to the production retraction_propagation tool on 2026-10-28 against snapshot ri-20261025-1526. Latency is client wall-clock time including the public network round trip.
02 THE CASES (click for the graph)
| Group | DOI | Verdict | Conf. | Cov. | Roots | ms |
|---|---|---|---|---|---|---|
| DEPTH0 | 10.1016/j.saa.2015.02.011 | CONTESTED | 0.50 | 0.00 | 1 | 1817 |
RETRACTED: Green synthesis of zinc oxide nanoparticles using Moringa oleifera leaf extract and evaluation of its antimicrobial activity (2015) · DOI 10.1016/j.saa.2015.02.011 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 1817 ms | ||||||
| DEPTH0 | 10.1083/jcb.200501040 | CONTESTED | 0.50 | 0.00 | 1 | 2050 |
RETRACTED: FOXO1a acts as a selective tumor suppressor in alveolar rhabdomyosarcoma (2005) · DOI 10.1083/jcb.200501040 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2050 ms | ||||||
| DEPTH0 | 10.1182/blood-2008-01-133710 | CONTESTED | 0.50 | 0.00 | 1 | 2019 |
Osteolineage niche cells initiate hematopoietic stem cell mobilization (2008) · DOI 10.1182/blood-2008-01-133710 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2019 ms | ||||||
| DEPTH0 | 10.1016/s0092-8674(03)00436-7 | CONTESTED | 0.50 | 0.00 | 1 | 2024 |
RETRACTED: The Chromatin-Remodeling Complex WINAC Targets a Nuclear Receptor to Promoters and Is Impaired in Williams Syndrome (2003) · DOI 10.1016/s0092-8674(03)00436-7 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2024 ms | ||||||
| DEPTH0 | 10.1016/j.jacc.2011.12.043 | CONTESTED | 0.50 | 0.00 | 1 | 1921 |
RETRACTED: Plasma PCSK9 Levels and Clinical Outcomes in the TNT (Treating to New Targets) Trial (2012) · DOI 10.1016/j.jacc.2011.12.043 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 1921 ms | ||||||
| DEPTH0 | 10.2337/db13-1526 | CONTESTED | 0.50 | 0.00 | 1 | 2289 |
Replication of Obesity and Associated Signaling Pathways Through Transfer of Microbiota From Obese-Prone Rats (2014) · DOI 10.2337/db13-1526 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2289 ms | ||||||
| DEPTH0 | 10.1074/jbc.m611728200 | CONTESTED | 0.50 | 0.00 | 1 | 2125 |
Arabidopsis Phosphatidylinositol Phosphate Kinase 1 Binds F-actin and Recruits Phosphatidylinositol 4-Kinase β1 to the Actin Cytoskeleton (2007) · DOI 10.1074/jbc.m611728200 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2125 ms | ||||||
| DEPTH0 | 10.1073/pnas.0610590104 | CONTESTED | 0.50 | 0.00 | 1 | 1781 |
RETRACTED: Metabolic regulation of SIRT1 transcription via a HIC1:CtBP corepressor complex (2007) · DOI 10.1073/pnas.0610590104 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 1781 ms | ||||||
| DEPTH0 | 10.1038/nature26142 | CONTESTED | 0.50 | 0.00 | 1 | 2026 |
RETRACTED ARTICLE: Quantized Majorana conductance (2018) · DOI 10.1038/nature26142 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2026 ms | ||||||
| DEPTH0 | 10.1038/s41417-022-00501-1 | CONTESTED | 0.50 | 0.00 | 1 | 1736 |
RETRACTED ARTICLE: Nrf2 overexpression increases the resistance of acute myeloid leukemia to cytarabine by inhibiting replication factor C4 (2022) · DOI 10.1038/s41417-022-00501-1 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 1736 ms | ||||||
| DEPTH0 | 10.1016/j.jcrysgro.2005.05.065 | CONTESTED | 0.50 | 0.00 | 1 | 2060 |
RETRACTED: Nucleation and growth behavior of well-aligned ZnO nanorods on organic substrates in aqueous solutions (2005) · DOI 10.1016/j.jcrysgro.2005.05.065 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2060 ms | ||||||
| DEPTH0 | 10.1001/jama.293.9.1082 | CONTESTED | 0.50 | 0.00 | 1 | 24704 |
Effect of folate and mecobalamin on hip fractures in patients with stroke: a randomized controlled trial. (2005) · DOI 10.1001/jama.293.9.1082 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 24704 ms | ||||||
| DEPTH0 | 10.1128/jvi.76.13.6815-6824.2002 | CONTESTED | 0.50 | 0.00 | 1 | 1657 |
P0 of Beet Western Yellows Virus Is a Suppressor of Posttranscriptional Gene Silencing (2002) · DOI 10.1128/jvi.76.13.6815-6824.2002 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 1657 ms | ||||||
| DEPTH0 | 10.1038/ncb942 | CONTESTED | 0.50 | 0.00 | 1 | 2197 |
Cytokines suppress adipogenesis and PPAR-γ function through the TAK1/TAB1/NIK cascade (2003) · DOI 10.1038/ncb942 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 2197 ms | ||||||
| DEPTH0 | 10.1038/35099527 | CONTESTED | 0.50 | 0.00 | 1 | 1922 |
Magnetic carbon (2001) · DOI 10.1038/35099527 verdict CONTESTED · confidence 0.50 · coverage 0.00 (0/0 references resolved) · 1922 ms | ||||||
| DEPTH1 | 10.1161/cir.0000000000001052 | ABSTAINED | 0.50 | 1.00 | 10 | 2542 |
Heart Disease and Stroke Statistics—2022 Update: A Report From the American Heart Association (2022) · DOI 10.1161/cir.0000000000001052 verdict INSUFFICIENT_EVIDENCE · confidence 0.50 · coverage 1.00 (3191/3195 references resolved) · abstained: single_aggregator_source · 2542 ms | ||||||
| DEPTH1 | 10.1161/cir.0000000000000558 | ABSTAINED | 0.50 | 1.00 | 2 | 2327 |
Heart Disease and Stroke Statistics—2018 Update: A Report From the American Heart Association (2018) · DOI 10.1161/cir.0000000000000558 verdict INSUFFICIENT_EVIDENCE · confidence 0.50 · coverage 1.00 (2260/2269 references resolved) · abstained: single_aggregator_source · 2327 ms | ||||||
| DEPTH1 | 10.1161/cir.0000000000000485 | ABSTAINED | 0.50 | 1.00 | 3 | 2430 |
Heart Disease and Stroke Statistics—2017 Update: A Report From the American Heart Association (2017) · DOI 10.1161/cir.0000000000000485 verdict INSUFFICIENT_EVIDENCE · confidence 0.50 · coverage 1.00 (2094/2101 references resolved) · abstained: single_aggregator_source · 2430 ms | ||||||
| DEPTH1 | 10.4158/ep161365.gl | CONTESTED | 0.99 | 0.99 | 3 | 2368 |
American Association of Clinical Endocrinologists and American College of Endocrinology Comprehensive Clinical Practice Guidelines For Medical Care of Patients with Obesity (2016) · DOI 10.4158/ep161365.gl verdict CONTESTED · confidence 0.99 · coverage 0.99 (1801/1818 references resolved) · 2368 ms | ||||||
| DEPTH1 | 10.1161/cir.0000000000000350 | ABSTAINED | 0.50 | 1.00 | 3 | 2244 |
Heart Disease and Stroke Statistics—2016 Update (2015) · DOI 10.1161/cir.0000000000000350 verdict INSUFFICIENT_EVIDENCE · confidence 0.50 · coverage 1.00 (1743/1748 references resolved) · abstained: single_aggregator_source · 2244 ms | ||||||
| DEPTH1 | 10.1161/cir.0000000000000152 | CONTESTED | 1.00 | 1.00 | 2 | 24640 |
Heart Disease and Stroke Statistics—2015 Update (2014) · DOI 10.1161/cir.0000000000000152 verdict CONTESTED · confidence 1.00 · coverage 1.00 (1521/1525 references resolved) · 24640 ms | ||||||
| DEPTH1 | 10.1161/cir.0000000000000152 | CONTESTED | 1.00 | 1.00 | 2 | 2987 |
Heart disease and stroke statistics--2015 update: a report from the American Heart Association. (2015) · DOI 10.1161/cir.0000000000000152 verdict CONTESTED · confidence 1.00 · coverage 1.00 (1521/1525 references resolved) · 2987 ms | ||||||
| DEPTH1 | 10.1161/01.cir.0000441139.02102.80 | ABSTAINED | 0.50 | 0.99 | 2 | 1979 |
Heart Disease and Stroke Statistics—2014 Update (2013) · DOI 10.1161/01.cir.0000441139.02102.80 verdict INSUFFICIENT_EVIDENCE · confidence 0.50 · coverage 0.99 (1452/1460 references resolved) · abstained: single_aggregator_source · 1979 ms | ||||||
| DEPTH1 | 10.1097/01.hjh.0000431740.32696.cc | CONTESTED | 1.00 | 0.99 | 4 | 2226 |
2013 ESH/ESC Guidelines for the management of arterial hypertension (2013) · DOI 10.1097/01.hjh.0000431740.32696.cc verdict CONTESTED · confidence 1.00 · coverage 0.99 (1292/1299 references resolved) · 2226 ms | ||||||
| DEPTH1 | 10.1161/str.0000000000000046 | ABSTAINED | 0.50 | 1.00 | 4 | 2530 |
Guidelines for the Primary Prevention of Stroke (2014) · DOI 10.1161/str.0000000000000046 verdict INSUFFICIENT_EVIDENCE · confidence 0.50 · coverage 1.00 (1221/1227 references resolved) · abstained: single_aggregator_source · 2530 ms | ||||||
| DEPTH1 | 10.36660/abc.20201238 | ABSTAINED | 0.48 | 0.97 | 8 | 2370 |
Diretrizes Brasileiras de Hipertensão Arterial – 2020 (2021) · DOI 10.36660/abc.20201238 verdict INSUFFICIENT_EVIDENCE · confidence 0.48 · coverage 0.97 (1110/1144 references resolved) · abstained: single_aggregator_source · 2370 ms | ||||||
| DEPTH1 | 10.1161/str.0000000000000024 | ABSTAINED | 0.50 | 1.00 | 4 | 2192 |
Guidelines for the Prevention of Stroke in Patients With Stroke and Transient Ischemic Attack (2014) · DOI 10.1161/str.0000000000000024 verdict INSUFFICIENT_EVIDENCE · confidence 0.50 · coverage 1.00 (1091/1095 references resolved) · abstained: single_aggregator_source · 2192 ms | ||||||
| CLEAN | 10.3748/wjg.11.7461 | SUPPORTED | 1.00 | 1.00 | 0 | 1595 |
Fibroblast growth factor-4 and hepatocyte growth factor induce differentiation of human umbilical cord blood-derived mesenchymal stem cells into hepatocytes (2005) · DOI 10.3748/wjg.11.7461 verdict SUPPORTED · confidence 1.00 · coverage 1.00 (13/13 references resolved) · 1595 ms | ||||||
03 WHAT IT CANNOT DO
There is no clean control group. The snapshot is a retraction neighbourhood: harvesting started from retracted seeds and walked their citations, so almost everything in it is inside the contamination radius. Only one true control survived. This benchmark therefore shows recall and non-fabrication, not precision on clean literature. Fixing that needs a control harvest (about a day of API budget) and is the next version of this page.
Abstaining is a feature. Eight of the twelve citing-retracted cases abstained, all with single_aggregator_source: only the C-tier aggregator claimed a retraction and no A-tier notice backed it. We refuse to promote an aggregator flag to a fact.
Coverage is the ceiling. Mean reference coverage is 0.46; below 0.5 the tool abstains outright. More conclusions require more data, not a different model.
04 REPRODUCE IT
The judgment is a deterministic graph query: the same DOI on the same snapshot always returns the same answer.
Anonymous MCP, no key needed, 20 requests/minute per IP:
curl -s -X POST https://mcp.ashesborn.cloud/mcp \
-H 'content-type: application/json' \
-H 'accept: application/json, text/event-stream' \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"bench","version":"1"}}}'
Free status lookup, no key: ashesborn.cloud/datasets ·
GET https://pay.ashesborn.cloud/api/v1/judge/paper?doi=<doi>
Raw run output — 28 records, verdicts, roots, coverage and latency as measured.
Full judgments with the per-hop evidence path are metered at $0.02 / $0.10, no signup: POST https://pay.ashesborn.cloud/api/v1/judge