{ "claims": [ { "claim": "The book tells a story in which causal inference was in a sorry state until Pearl's 'Causal Revolution' and is now essentially solved, with near-universal consensus on causal diagrams; this presentation may lead newcomers, especially policy-makers, to unrealistically optimistic conclusions and overlooks alternative models.", "claim_id": "P050-H1", "related_corrections": [ "P050-N006" ], "tied_to": "No numbered result. Opening paragraphs and the statement of the review's two goals, pp. 482-483 (PDF pp. 2-3); quotation from the book p. 19; block quotation from the book p. 141" }, { "claim": "The central problem scientists face is choosing a valid or reasonable causal model, not expressing or analysing one. The book is about causal analysis given a known structure, not causal discovery, so Pearl's approach can clarify where a disagreement lies but cannot adjudicate it; the 1950s smoking-lung cancer debate (Fisher versus Cornfield) illustrates this.", "claim_id": "P050-H2", "related_corrections": [], "tied_to": "No numbered result. Section 'Unfounded Optimism About Causal Models', p. 483 (PDF p. 3); block quotation from the book pp. 79-80; 'Diagrams 5.1 and 5.2 in the book (p. 176)'" }, { "claim": "The modes of inquiry that actually settle debates, such as Cornfield's sensitivity analysis and randomized experiments, work because they let scientists stay largely agnostic about the causal structure; the book's treatment of randomized controlled trials (no need to treat them as a gold standard) misses this. Their usefulness comes with limits on what they identify or cover.", "claim_id": "P050-H3", "related_corrections": [], "tied_to": "No numbered result. p. 483 right column to p. 484 top left (PDF pp. 3-4); block quotation from the book p. 140" }, { "claim": "Agnostic approaches have limited applicability, but situations where Pearl's framework can be applied are rare, and even with consensus on the structure the calculus will often say inference is impossible: Fisher's genetic confounding was real, a graph cannot encode that it is weak, and 'Pearl's calculus would have told us that progress was not possible until these genes were identified and measured'. Causal inquiry cannot be reduced to a mathematical exercise.", "claim_id": "P050-H4", "related_corrections": [], "tied_to": "No numbered result. p. 484 left column, second and third paragraphs (PDF p. 4); the smoking-gene example ('as noted in the book')" }, { "claim": "The book's history of causal inference is selective and narrow. No consensus, not even an emerging one, exists about the superiority of DAGs; several frameworks thrive because scientists face different problems and can defend different assumptions, and Pearl reduces this pluralism to 'cultural resistance'. Notable omissions are the contributions of a large group of scholars and the problem of estimation.", "claim_id": "P050-H5", "related_corrections": [], "tied_to": "No numbered result. Section 'Pluralism in Causal Inference', p. 484 left column, and footnote 1 (PDF p. 4); 'Whig history'; quotation 'cultural resistance' (p. 394)" }, { "claim": "Robins (1986) developed a closely related causal model that predates Pearl's. Both are nonparametric structural equation models; the sole distinction is that Pearl's adds independence assumptions, which make it more powerful (natural mediation effects become identified) but have no testable implications, even through experiments. Robins also introduced g-notation and g-computation, which parallel the do-operator and do-calculus.", "claim_id": "P050-H6", "related_corrections": [ "P050-E001" ], "tied_to": "No numbered result. p. 484, bottom of left column and top of right column, and footnote 2 (PDF p. 4); Robins (1986); quotations from the book p. 20 and p. 334" }, { "claim": "The design-based approach (Neyman, Fisher) treats random assignment as the sole source of randomness and is agnostic about laws of nature and superpopulations; the cost is a circumscribed task: effects of a well-defined intervention for a fixed population in one setting.", "claim_id": "P050-H7", "related_corrections": [], "tied_to": "No numbered result. p. 484 right column, paragraph beginning 'The next model' (PDF p. 4); cross-reference 'As we noted above'" }, { "claim": "Pearl's calculus cannot use information about the form of causal relations. The local average treatment effect interpretation of instrumental variables rests on monotonicity, which a DAG cannot encode; instrumental variables, regression discontinuity (continuity) and difference-in-differences (additivity) rely on such assumptions, are immensely popular in the social sciences, and the last two are omitted from the book. Readers who knew this might doubt that causal diagrams give a complete and systematic solution to confounding.", "claim_id": "P050-H8", "related_corrections": [ "P050-N006" ], "tied_to": "No numbered result. p. 484 right column, last two paragraphs, to the end of the review on p. 485 (PDF pp. 4-5); block quotation from the book p. 257; closing re-quotation of the p. 141 passage" } ], "date_published": "2026-10-25", "findings": [ { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of five lines at the top of the right column of page 484. The third line reads: have testable implications. Pearl is quick to point the first part. A red outline surrounds the two words point the. The fourth line begins: while neglecting the second.", "caption": "Page 484 (PDF p. 4), top of the right column. The outline marks “point the” in the third line, where the word “out” is missing between the two words.", "url": "https://pmaronow.github.io/llm-audit/works/P050/exhibits/P050-E001.png" } ], "formal_release": null, "formal_statements": [], "headline": "The word “out” is missing: the sentence should read “Pearl is quick to point out the first part while neglecting the second.”", "id": "P050-E001", "location": "p. 484 (PDF p. 4), right column, first (continued) paragraph, third line", "pdf_page": 4, "printed_page": "484", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "Pearl is quick to \\hl{point the} first part while neglecting the second.", "corrected": "Pearl is quick to point \\hl{out} the first part while neglecting the second.", "evidence": "I checked the printed page.", "problem": "The phrase ``point out'' has lost its second word. The preceding sentence supplies the two parts: the additional independence assumptions make Pearl's model more powerful, and they have no testable implications.", "what_survives": "The meaning is clear from the preceding sentence, and nothing in the review depends on the missing word." }, "url": "https://pmaronow.github.io/llm-audit/works/P050/#P050-E001", "work": { "arxiv_id": "2003.11635", "doi": "10.1080/01621459.2020.1721245", "title": "Review of The Book of Why: The New Science of Cause and Effect", "url": "https://pmaronow.github.io/llm-audit/works/P050/" }, "work_id": "P050" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of the four lines that end the review on page 485. The last two lines read: question whether, opening quotation mark, causal diagrams provide a complete and systematic way of finding a solution [to confounding], full stop, closing quotation mark. A red outline surrounds the single word a, between finding and solution.", "caption": "Page 485 (PDF p. 5), left column, the last sentence of the review. The outline marks the word “a” in “finding a solution [to confounding]”.", "url": "https://pmaronow.github.io/llm-audit/works/P050/exhibits/P050-N006.png" }, { "alt": "Crop of the five-line block quotation on page 483. It reads: There is now an almost universal consensus, at least among epidemiologists, philosophers and social scientists, that (1) confounding needs and has a causal solution, and (2) causal diagrams provide a complete and systematic way of finding that solution. (p. 141). A red outline surrounds the word that, at the start of the last line.", "caption": "Page 483 (PDF p. 3), left column, the block quotation cited to p. 141 of the book, for comparison. The outline marks the word “that” in “finding that solution”.", "url": "https://pmaronow.github.io/llm-audit/works/P050/exhibits/P050-N006-b.png" } ], "formal_release": null, "formal_statements": [], "headline": "The closing sentence quotes p. 141 of the book as “finding a solution”, while the block quotation on p. 483 has “finding that solution”; the change of word is not marked.", "id": "P050-N006", "location": "p. 485 (PDF p. 5), left column, last sentence of the review; compare the block quotation on p. 483 (PDF p. 3), left column", "pdf_page": 5, "printed_page": "485", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\emph{Block quotation, p.~483.} There is now an almost universal consensus, at least among epidemiologists, philosophers and social scientists, that (1) confounding needs and has a causal solution, and (2) causal diagrams provide a complete and systematic way of finding \\hl{that} solution. (p.~141)\n\n\\emph{Last sentence, p.~485.} \\ldots\\ they might question whether ``causal diagrams provide a complete and systematic way of finding \\hl{a} solution [to confounding].''", "corrected": "\\ldots\\ they might question whether ``causal diagrams provide a complete and systematic way of finding \\hl{[a]} solution [to confounding].''", "evidence": "I compared the two printed pages, which show that the quotations differ. I did not read p.~141 of the book, so I cannot establish which quotation matches it.", "problem": "The review quotes the same sentence of the book twice, and the two quotations differ in one word. In the closing sentence the square brackets mark ``to confounding'' as an insertion, but ``a'' stands outside the brackets where the block quotation has ``that''. The two quotations cannot both reproduce the wording of the book.", "what_survives": "The sense of the quotation is unchanged, and the argument of the closing paragraph does not depend on the word. I use the block quotation on p.~483 as the reference wording below and mark the substitution with brackets; the final wording should be compared with p.~141 of the book." }, "url": "https://pmaronow.github.io/llm-audit/works/P050/#P050-N006", "work": { "arxiv_id": "2003.11635", "doi": "10.1080/01621459.2020.1721245", "title": "Review of The Book of Why: The New Science of Cause and Effect", "url": "https://pmaronow.github.io/llm-audit/works/P050/" }, "work_id": "P050" } ], "lean": { "applicable": false, "guide": null, "reason": "A prose book review (Journal of the American Statistical Association 115:529, printed pp. 482-485, PDF pp. 2-5) with no theorem, formula, identity, bound or proof on any rendered page: its eight headline claims (P050-H1 to P050-H8) are interpretive remarks tied to no numbered result, and its only two confirmed errors, both Minor (P050-E001, the word 'out' missing from 'Pearl is quick to point the first part while neglecting the second', p. 484, PDF p. 4; P050-N006, the closing re-quotation 'finding a solution [to confounding]', p. 485, PDF p. 5, against 'finding that solution', p. 483, PDF p. 3), are a wording misprint and a quotation slip with no mathematical statement to correct.", "repository": null, "targets": [] }, "missing_sources": [], "reproduction": { "agreement": "not_applicable", "status": "not_applicable", "targets": [] }, "summary": "The review assesses \\emph{The Book of Why} by Judea Pearl and Dana Mackenzie. It argues that the book is too optimistic about what causal models can settle, since the central difficulty is the choice of a model, and that its history neglects other approaches to causal inference. The review has no formulas, tables or data. 2 entries are classified as errors (2 Minor): a missing word, and an unmarked change of one word in a repeated quotation. Eight possible errors were not confirmed, and two of them cannot be settled without the book.", "what_was_checked": "I read the whole text of the review, both footnotes and the single reference on the published pages. I checked the following and found no error in them. The examples are computed in exact arithmetic.\n\\begin{itemize}\n\\item The limits of identification from a graph (p.~484, left column). On the graph in which an unobserved gene causes both smoking and lung cancer, the effect of smoking is not identified. There exist two models on this graph that have the same observed law and average causal effects $1/8$ and $-1/8$.\n\\item The comparison of Robins' model with Pearl's (p.~484). There exist three laws of the counterfactuals that have the same observed law and the same law under every static intervention, and whose natural direct effects are $0$, $1$ and $1/2$. The cross-world independence that Pearl's model adds is what identifies the effect, and it has no implication for the observed law or for the law under a static intervention.\n\\item Footnote 2, last sentence. A static treatment regime is a fixed intervention, so the do-operator is a special case of g-notation.\n\\item Monotonicity and the local average treatment effect (pp.~484--485). With a randomized binary instrument, exclusion, monotonicity and a nonzero first stage, the Wald ratio equals the mean effect among compliers; both are $1/2$ in an example. Without monotonicity the ratio need not be an average effect for any group: with compliers of mass $1/2$ and effect $1$, and defiers of mass $1/4$ and effect $3$, it equals $-1$. A graph records which variables depend on which, and not the sign, shape or strength of the dependence.\n\\item Regression discontinuity and difference-in-differences (p.~485). The descriptions of these designs as relying on continuity and on additivity agree with their usual identification conditions. In an example with two groups, two periods and additive untreated means, the difference-in-differences equals the average effect on the treated, $4$.\n\\item Sensitivity analysis (p.~483). In the checked model, outcome risk depends only on a risk-increasing binary confounder, and exposure has no effect. If its prevalences among the unexposed and exposed satisfy $0