Citation Integrity in Incoming Research — Real Finding, Wrong Paper (Three Instances, One Shape)
Open questionA research brief on 9/11 was checked source by source - 55 of its 69 links opened, plus ten original documents it named but never linked. Most of what it said held up: the 1962 Northwoods memo, the government's own collapse figures, the building's tenants, the insurance dates, the freedom-of-information clause. What did not hold up was its referencing. Three times a real finding was attached to the wrong paper - including a sentence quoted as one economist's that belongs to three others studying a different kind of contract entirely. And three times a source was given a grander label than it deserved: a hobbyist website presented as a congressional record, an academic essay as a government transcript, a personal blog as an official response. The open question is whether that is ordinary sloppiness or something structural - because a document whose citations look solid is precisely what a reader trusts instead of checking. What leans it toward the second reading is that all three errors happen to strengthen the argument rather than weaken it. Three cases is a small sample and the engine does not close it. The rule it sets: a report's substance and its sourcing get scored separately, and the map takes in the original documents a report points at, never the report's description of them.
The engine's record — word for word
A commissioned structural brief on the 9/11 institutional surround was verified source by source: 55 of its 69 cited URLs opened, plus ten primaries it named but never linked. Its SUBSTANCE largely survived contact with the primaries — every Operation Northwoods claim, every NIST kinematic figure, all four WTC 7 tenants, both Hulsey conclusions, every Harrit figure, every Silverstein date, 10 U.S.C. 135, the Homeland Security Act facts, and the entire FOIA Exemption 3 analysis all checked out verbatim. Its CITATION APPARATUS did not, and the failures share one shape.
THREE MISATTRIBUTIONS, each a real finding welded to the wrong document:
(1) A claim of abnormal pre-attack trading across airline and reinsurance sectors was cited to Chesney, Reshetar & Karaman (2011, Journal of Banking & Finance) — a 25-country study of terrorism's general market impact in which the word 'abnormal' does not appear. The paper that makes the claim is Chesney, Crameri & Mancini (2015, Journal of Empirical Finance). Decisive test: the 2015 paper's own related-literature section does not cite the 2011 paper.
(2) A sentence quoted as Poteshman's — 'reject the null hypotheses that there was no abnormal trading in these contracts prior to the September 11 attacks' — belongs to Wong, Thompson & Teh, together with the OTM/ATM/ITM methodology and the SPX instrument class attached to it. Wong et al. state they studied index options INSTEAD of airlines precisely because 'the airline data have been well studied by Poteshman (2006)'. Their paper appears nowhere in the brief's 69 sources.
(3) A Bažant paper on gravitational collapse energetics was described, but the only resolving link is Le & Bažant's separate technical note on motion smoothness, in which 'kinetic energy' appears zero times.
PLUS TIER INFLATION: a 9/11-chronology page was presented as 'Tier: Congressional Record'; an academic essay as 'Tier: Agency Transcript'; a personal blog as 'Tier: Agency FOIA Response'. And a NIST quotation inside quotation marks is a paraphrase; a girder designation attributed to NIST ('A2001') appears zero times across 1.78 million characters of NIST text.
THE DIVERGENCE. Reading A — ordinary sloppiness: fast synthesis over a large corpus, citations attached from memory, no intent, and the surviving substance is the measure of the work. Reading B — structural: an apparatus that generates confident, well-formed, heavily-cited prose whose citations do not bear inspection is a distinct failure mode from error, because the citation apparatus is precisely what a reader uses INSTEAD of checking. Under B the tier labels matter more than the misattributions, since they upgrade source authority in the reader's eye rather than merely misplacing it.
WHAT DECIDES BETWEEN THEM, and it is checkable: whether the misattributions are randomly distributed or directional. Here all three land on the side that strengthens the brief's argument — upgrading a forum post to an agency record, a blog to a FOIA response, and attaching a real quantitative finding to a more citable author. That is a directional pattern, which leans B. But three instances is a small sample and the engine does not close it on three.
THE STANDING RULE THIS SETS: an incoming report's substance and its sourcing are scored separately. Canon ingests the primaries the report points at, never the report's characterisation of them, and where a primary declines to characterise something, canon declines too. Applied in this ripple: the audit's own three-way split is held, the brief's reading of it is not.
Walk this on the live map →