{
 "claims": [
  {
   "claim": "Agnosticism in statistics is better understood as assumption-skeptical than assumption-free: even standard analyses of randomized experiments rest on assumptions about the structure of the world (SUTVA: no interference, no hidden versions of treatment), so a definition of agnosticism that excludes assumptions would exclude the study of causal effects, and needing assumptions for temporal validity is not by itself disqualifying.",
   "claim_id": "P048-H1",
   "related_corrections": [
    "P048-E003",
    "P048-N001",
    "P048-N003"
   ],
   "tied_to": "No numbered result. Abstract and Introduction, p. 1; section 'Agnosticism and causality', p. 2; Notes 3, 4 and 5 (pp. 4–5)"
  },
  {
   "claim": "Munger's 'unsolved contradiction' (external validity needs knowledge of the target; the target is in the future; the future is unknown) is troubling only if guaranteed transport of a causal parameter is the sole criterion of success. Causal knowledge whose estimated parameter does not literally travel can still be useful, by informing heuristic 'laws of qualitative structure' (Simon) and by establishing local particulars (Boas).",
   "claim_id": "P048-H2",
   "related_corrections": [
    "P048-E005",
    "P048-N003",
    "P048-N007"
   ],
   "tied_to": "No numbered result. Abstract, p. 1; section 'How knowledge travels', pp. 2–3; Notes 6, 7 and 9"
  },
  {
   "claim": "Knowledge decay in social science often comes from adaptation and learning by the people studied, which is not opaque to researchers. Lucas's critique and Goodhart's law give constructive, microfoundational accounts of parameter drift, with examples in hospital report cards, machine-learning benchmarks and strategic classification; these dynamics can themselves be studied, and changing parameters need not make the underlying process wholly opaque.",
   "claim_id": "P048-H3",
   "related_corrections": [
    "P048-E001",
    "P048-E002",
    "P048-N009"
   ],
   "tied_to": "No numbered result. Abstract and Introduction, p. 1; section 'Why knowledge decays', pp. 3–4; Note 10"
  },
  {
   "claim": "Fast-decaying contexts are not 'too expensive for rigorous causal knowledge': rapid knowledge decay alone does not make rigorous causal research unworthy, because researchers may value results other than stable effect estimates (local dynamics, later uses), scientists differ in what they want to study, and the field cannot know in advance which results will matter later.",
   "claim_id": "P048-H4",
   "related_corrections": [
    "P048-N008"
   ],
   "tied_to": "No numbered result. Section 'How knowledge travels', p. 3 (paragraphs beginning 'Munger argues that' and 'Munger also asks us'); closing paragraph of 'Why knowledge decays', p. 4; Notes 8 and 9"
  },
  {
   "claim": "Munger's conclusions may be too pessimistic for practice. The paper finds no contradiction among existing methods for external validity, the institutions of social science and a predictive paradigm; with explicit model beliefs ('a bit of gnosis') useful and reasonable claims can be made about the conditions under which findings generalize, and technical tools exist for studying changing systems (extensions of sensitivity analysis, designs for non-stationary multi-arm problems, distributionally and adversarially robust learning).",
   "claim_id": "P048-H5",
   "related_corrections": [],
   "tied_to": "No numbered result. Abstract, last sentence; section 'Concluding remarks', p. 4; definition of gnosis in Note 2; rests on the preceding three sections"
  }
 ],
 "date_published": "2026-10-25",
 "findings": [
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of the four lines of the reference entry Hendrycks D, Burns C, Basart S, et al. (2021) Measuring massive Multitask Language Understanding. In: International Conference on Learning Representations, Singapore, Thu Apr 24 - Mon Apr 28th, 2025. A red outline surrounds Singapore, Thu Apr at the end of the third line and another surrounds 24 - Mon Apr 28th, 2025. in the fourth line.",
     "caption": "Reference list, p. 5 (PDF p. 5), right column, entry for Hendrycks et al. (2021). The outlines mark the place and date line ending in 2025.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-E001.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The reference dated 2021 carries a place and date line for 2025, which should be deleted.",
   "id": "P048-E001",
   "location": "Reference list, p. 5 (PDF p. 5), right column, entry “Hendrycks D, Burns C, Basart S, et al. (2021)”",
   "pdf_page": 5,
   "printed_page": "5",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "Hendrycks D, Burns C, Basart S, et al. (2021) Measuring massive Multitask Language Understanding. In: International Conference on Learning Representations, \\hl{Singapore, Thu Apr 24 -- Mon Apr 28th, 2025}.",
    "corrected": "Hendrycks D, Burns C, Basart S, et al. (2021) Measuring massive Multitask Language Understanding. In: International Conference on Learning Representations\\del{, Singapore, Thu Apr 24 -- Mon Apr 28th, 2025}.",
    "evidence": "The evidence is a check of the page and of the calendar: 24 and 28 April fall on a Thursday and a Monday in 2025, and on a Saturday and a Wednesday in 2021. I conclude that the printed days are those of 2025. I did not check the place and dates of the 2021 meeting against its proceedings, and the correction omits them.",
    "problem": "The entry is dated 2021, and the text on p.~4 cites it as ``Hendrycks et al., 2021'', but its last line has the place and dates of a meeting in 2025.",
    "what_survives": "The benchmark is identified by its title, authors and year, and the argument on p.~4 about adaptation to benchmarks is unaffected."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-E001",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of the three lines of the reference entry Björkegren D, Blumenstock JE and Samsun K (2020) Training machine learning to anticipate manipulation. arXiv preprint arXiv:2004.03865. A red outline surrounds the name Samsun K.",
     "caption": "Reference list, p. 5 (PDF p. 5), left column, last entry. The outline marks “Samsun K”.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-E002.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The third author of the reference is Samsun Knight, and “Samsun K” should read “Knight S”.",
   "id": "P048-E002",
   "location": "Reference list, p. 5 (PDF p. 5), left column, entry “Björkegren D, Blumenstock JE and Samsun K (2020)”",
   "pdf_page": 5,
   "printed_page": "5",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "Björkegren D, Blumenstock JE and \\hl{Samsun K} (2020) ``Training machine learning to anticipate manipulation''. arXiv preprint arXiv:2004.03865.",
    "corrected": "Björkegren D, Blumenstock JE and \\hl{Knight S} (2020) ``Training machine learning to anticipate manipulation''. arXiv preprint arXiv:2004.03865.",
    "evidence": "I base this correction on the arXiv listing for arXiv:2004.03865, whose author line reads ``Daniel Björkegren, Joshua E. Blumenstock, Samsun Knight''.",
    "problem": "The entry lists its authors by surname and initials, and it prints the third author's given name as the surname and the initial of the surname as the given name.",
    "what_survives": "The arXiv identifier locates the work, and the discussion of strategic adaptation on p.~4 does not depend on the entry."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-E002",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of the first four lines of note 4 on page 5. The note begins: Following Lin (2013)'s, then in quotation marks Agnostic notes on regression adjustments to experimental, then literal belief in assumptions is not requisite to learning from settings in which they are applied. A red outline surrounds the word experimental and the closing quotation mark that follows it directly. No comma follows the quotation mark.",
     "caption": "Note 4, p. 5 (PDF p. 5), left column, first four lines. The outline marks the end of the quoted title, which stops at “experimental”.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-E003.png"
    },
    {
     "alt": "Crop of the three lines of the reference entry Lin W (2013) Agnostic notes on regression adjustments to experimental data: reexamining Freedman's critique. Annals of Applied Statistics 7(1): 295-318. A red outline surrounds the words data: reexamining Freedman's critique.",
     "caption": "Reference list, p. 5 (PDF p. 5), right column, entry for Lin (2013). The outline marks the part of the title that note 4 omits.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-E003-reference.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "Note 4 quotes the title of Lin (2013) only as far as “experimental” and has no comma after it; the note should cite Lin (2013) without the title.",
   "id": "P048-E003",
   "location": "Note 4, p. 5 (PDF p. 5), left column, first sentence",
   "pdf_page": 5,
   "printed_page": "5",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "Following Lin (2013)\\hl{’s ``Agnostic notes on regression adjustments to experimental''} literal belief in assumptions is not requisite to learning from settings in which they are applied:",
    "corrected": "Following Lin (2013)\\hl{,} literal belief in assumptions is not requisite to learning from settings in which they are applied:",
    "evidence": "I compared note 4 with the entry for Lin (2013) in the reference list on the same page.",
    "problem": "The quoted title ends at the adjective ``experimental''. The reference list on the same page prints the title as ``Agnostic notes on regression adjustments to experimental data: reexamining Freedman's critique''. The sentence also has no comma between the introductory phrase and ``literal belief''.",
    "what_survives": "The quotation about OLS adjustment that follows in the note is unaffected, as is the passage on p.~2 that the note supports."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-E003",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of the three lines of the reference entry Hastie T, Tibshirani R, Friedman JH, et al. (2009) The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer, Vol. 2. A red outline surrounds et al. in the first line and another surrounds Vol. 2. at the end of the third line.",
     "caption": "Reference list, p. 5 (PDF p. 5), right column, entry for Hastie, Tibshirani and Friedman (2009). The outlines mark “et al.” and “Vol. 2.”",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-E004.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The book has three authors and is a second edition: “et al.” should be deleted, and “Vol. 2” should read “2nd ed.”",
   "id": "P048-E004",
   "location": "Reference list, p. 5 (PDF p. 5), right column, entry “Hastie T, Tibshirani R, Friedman JH, et al. (2009)”",
   "pdf_page": 5,
   "printed_page": "5",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "Hastie T, Tibshirani R, Friedman JH, \\hl{et al.} (2009) \\emph{The Elements of Statistical Learning: Data Mining, Inference, and Prediction}. New York: Springer, \\hl{Vol. 2}.",
    "corrected": "Hastie T, Tibshirani R \\hl{and} Friedman JH (2009) \\emph{The Elements of Statistical Learning: Data Mining, Inference, and Prediction}. \\hl{2nd ed.} New York: Springer.",
    "evidence": "The evidence is a public source. The title page of the book, as the publisher hosts it, names Trevor Hastie, Robert Tibshirani and Jerome Friedman and reads ``Second Edition'', and the publisher's bibliographic information for the book gives ``Edition number: 2''.",
    "problem": "The entry adds ``et al.'' after the names of all three authors of the book, and it describes the second edition of the book as a second volume.",
    "what_survives": "The book is identifiable by its title, authors and year, and the definition of prediction in note 6 does not depend on the entry."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-E004",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of four lines of the right column of page 2: item 3 of the numbered list, We cannot have knowledge of the future, and the first three lines of the next paragraph, which begins We agree that if the only criteria for a successful research program is the guaranteed transportability of a causal parameter. A red outline surrounds the word criteria.",
     "caption": "Section “How knowledge travels”, p. 2 (PDF p. 2), right column, after the numbered list. The outline marks “criteria”.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-E005.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "“The only criteria … is” should read “the only criterion … is”.",
   "id": "P048-E005",
   "location": "Section “How knowledge travels”, p. 2 (PDF p. 2), right column, first sentence after the numbered list",
   "pdf_page": 2,
   "printed_page": "2",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "We agree that if the only \\hl{criteria} for a successful research program is the guaranteed transportability of a causal parameter, then this contradiction would be troubling.",
    "corrected": "We agree that if the only \\hl{criterion} for a successful research program is the guaranteed transportability of a causal parameter, then this contradiction would be troubling.",
    "evidence": "I checked the printed page.",
    "problem": "The sentence names one condition, the guaranteed transportability of a causal parameter, and its verb is the singular ``is'', but its noun is the plural ``criteria''.",
    "what_survives": "The conditional argument of the sentence is unchanged, and nothing else depends on the word."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-E005",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of five lines of the left column of page 2. The paragraph that begins To contextualize this stance says that standard analyses of RCTs do not make zero-modeling assumptions, with the last two words in quotation marks and followed by the note number 5. A red outline surrounds the hyphenated word zero-modeling, which stands inside one line.",
     "caption": "Section “Agnosticism and causality”, p. 2 (PDF p. 2), left column. The outline marks “zero-modeling” in the short quotation from Munger (2023).",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-N001.png"
    },
    {
     "alt": "Crop of the first five lines of the left column of page 2. The quotation reads: fully agnostic: they ensure unconfoundedness and positivity (internal validity) through research design, with zero modeling assumptions about the structure of the world, followed by the citation Munger 2023: p. 3. A red outline surrounds the two words zero modeling, which have a space between them and no hyphen.",
     "caption": "The same column, lines 1–5 of the page. The outline marks “zero modeling”, printed without a hyphen in the long quotation from Munger (2023: p. 3).",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-N001-quotation.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The short quotation from Munger (2023) should read “zero modeling assumptions”, without a hyphen, as the longer quotation above it does.",
   "id": "P048-N001",
   "location": "Section “Agnosticism and causality”, p. 2 (PDF p. 2), left column, first sentence of the paragraph beginning “To contextualize this stance”",
   "pdf_page": 2,
   "printed_page": "2",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "To contextualize this stance, we first note that standard analyses of RCTs do not make ``\\hl{zero-modeling} assumptions.''$^{5}$",
    "corrected": "To contextualize this stance, we first note that standard analyses of RCTs do not make ``\\hl{zero modeling} assumptions.''$^{5}$",
    "evidence": "I base this correction on the page and the published text of Munger (2023), which prints ``zero modeling'' on its p.~3.",
    "problem": "The article quotes the same words of Munger (2023, p.~3) twice in this column. The long quotation at the top of the column has ``zero modeling assumptions''. The short quotation has ``zero-modeling assumptions'', and the hyphen is not a line-break hyphen, since it falls inside a line.",
    "what_survives": "Both forms are read as ``no modeling assumptions'', and the argument of the section is unchanged."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-N001",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of the heading Why knowledge decays and the first five lines of the section in the right column of page 3. The first sentence says that Munger provides the concept of a decay rate r, which, even in a best case scenario, is both positive and unpredictable, with the citation p 6. A red outline surrounds the word best at the end of the second line and another surrounds the word case at the start of the third line. No hyphen follows best.",
     "caption": "Section “Why knowledge decays”, p. 3 (PDF p. 3), right column, first lines. The outlines mark “best” and “case”, printed without a hyphen at the end of one line and the start of the next.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-N003.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The quotation from Munger (2023, p. 6) should read “best-case scenario”, with a hyphen.",
   "id": "P048-N003",
   "location": "Section “Why knowledge decays”, p. 3 (PDF p. 3), right column, first sentence",
   "pdf_page": 3,
   "printed_page": "3",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "In formalizing the idea of knowledge decay, Munger provides the concept of a decay rate $r$, which, even in a ``\\hl{best case} scenario\\ldots\\ is both positive and unpredictable'' (p 6).",
    "corrected": "In formalizing the idea of knowledge decay, Munger provides the concept of a decay rate $r$, which, even in a ``\\hl{best-case} scenario\\ldots\\ is both positive and unpredictable'' (p 6).",
    "evidence": "I base this correction on a comparison with the published text of Munger (2023), \\emph{Research \\& Politics} 10(3).",
    "problem": "Munger (2023, p.~6) prints ``best-case scenario'', and the quotation drops the hyphen. The other passages that were compared with that article agree with it, apart from the hyphen in \\fid{P048-N001}: the three premises of the contradiction on p.~2, the words ``assumption-free'', the symbol $r$, the long quotation from its p.~3, the sentence quoted in note 7, and the page numbers 2, 3 and 6.",
    "what_survives": "The quoted words keep their meaning, and the article's account of Munger's argument is unchanged."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-N003",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of five lines of the notes on page 5: the last line of note 6, the three lines of note 7 and the first line of note 8. Note 7 reads: Munger acknowledges that beyond prediction, There are other goals, of course, and social science is no stranger to methodological pluralism (p 2). The quotation opens before There and closes after the full stop that follows (p 2). A red outline surrounds (p 2), the full stop and the closing quotation mark.",
     "caption": "Note 7, p. 5 (PDF p. 5), left column. The outline marks the page reference “(p 2).” and the closing quotation mark printed after it.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-N007.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "In note 7 the closing quotation mark should come before the page reference “(p 2)”.",
   "id": "P048-N007",
   "location": "Note 7, p. 5 (PDF p. 5), left column",
   "pdf_page": 5,
   "printed_page": "5",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "Munger acknowledges that beyond prediction, ``There are other goals, of course, and social science is no stranger to methodological pluralism \\hl{(p 2).''}",
    "corrected": "Munger acknowledges that beyond prediction, ``There are other goals, of course, and social science is no stranger to methodological pluralism\\hl{'' (p 2).}",
    "evidence": "I checked the printed page.",
    "problem": "The closing quotation mark follows the page reference and the full stop, and the reference is printed inside the quotation. On p.~3 the same construction closes the quotation first: ``\\ldots\\ is both positive and unpredictable'' (p 6).",
    "what_survives": "The quotation is read as ending at ``pluralism'', and nothing else changes."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-N007",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of six lines of the notes on page 5: the five lines of note 8 and the first line of note 9. The last sentence of note 8 reads: Einstein himself was emphatically skeptical of the eventual possibility of using the atom to generate energy Moszkowski (1921/2014). A red outline surrounds Moszkowski (1921/2014) and the full stop after it.",
     "caption": "Note 8, p. 5 (PDF p. 5), left column. The outline marks the citation “Moszkowski (1921/2014)”, which follows the sentence without enclosing parentheses.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-N008.png"
    },
    {
     "alt": "Crop of the three lines of the reference entry Moszkowski A (2014). Einstein the searcher: his work explained from dialogues with einstein. In: Translated from the Original German by Henry L. Brose. Oxfordshire: Routledge. The title is in roman type and the words Translated from the Original German by Henry L. Brose are in italics. A red outline surrounds the passage from einstein, with a lower-case e, through In: to Brose.",
     "caption": "Reference list, p. 5 (PDF p. 5), right column, entry for Moszkowski. The outline marks the lower-case “einstein”, the “In:” and the translator credit set in italics.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-N008-reference.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The citation of Moszkowski in note 8 lacks its parentheses, and the reference entry prints the translator credit as a containing work after “In:” and the second “Einstein” of the title in lower case.",
   "id": "P048-N008",
   "location": "Note 8, last sentence, p. 5 (PDF p. 5), left column; and the entry “Moszkowski A (2014)” in the reference list, p. 5, right column",
   "pdf_page": 5,
   "printed_page": "5",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "\\emph{Note 8.} Einstein himself was emphatically skeptical of the eventual possibility of using the atom to generate energy \\hl{Moszkowski (1921/2014)}.\n\n\\emph{Reference list.} Moszkowski A (2014). Einstein the searcher: his work explained from dialogues with \\hl{einstein}. \\hl{In: \\emph{Translated from the Original German by Henry L. Brose}}. Oxfordshire: Routledge.",
    "corrected": "\\emph{Note 8.} Einstein himself was emphatically skeptical of the eventual possibility of using the atom to generate energy \\hl{(Moszkowski, 1921/2014)}.\n\n\\emph{Reference list.} Moszkowski A (2014). \\emph{Einstein the searcher: his work explained from dialogues with \\hl{Einstein}}. \\del{In:} \\hl{Translated from the Original German by Henry L. Brose}. Oxfordshire: Routledge.\n\nIn the entry the title of the book is set in italics and the translator credit in roman.",
    "evidence": "I checked the printed page.",
    "problem": "In note 8 the citation is appended to the sentence without the parentheses that a citation outside the grammar of the sentence takes. In the reference entry the translator credit follows ``In:'' and is set in italics, as the title of a containing work would be, while the title of the book is set in roman, and the second ``Einstein'' of the title is in lower case.",
    "what_survives": "The book remains identifiable, and the dates are compatible with an original year paired with the year of the cited edition. The argument about basic science and later innovation does not depend on the form of the citation."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-N008",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of the three lines of the reference entry Goodhart CAE (1984) Problems of monetary management: the UK experience. In: Monetary Theory and Practice. Berlin: Springer, 91-121. A red outline surrounds Berlin: at the end of the second line and another surrounds Springer, with its comma, at the start of the third line.",
     "caption": "Reference list, p. 5 (PDF p. 5), right column, entry for Goodhart (1984). The outlines mark “Berlin:” and “Springer”.",
     "url": "https://pmaronow.github.io/llm-audit/works/P048/exhibits/P048-N009.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The 1984 book Monetary Theory and Practice was published by Macmillan Press in London, not by Springer in Berlin.",
   "id": "P048-N009",
   "location": "Reference list, p. 5 (PDF p. 5), right column, entry “Goodhart CAE (1984)”",
   "pdf_page": 5,
   "printed_page": "5",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "Goodhart CAE (1984) Problems of monetary management: the UK experience. In: \\emph{Monetary Theory and Practice}. \\hl{Berlin: Springer}, 91--121.",
    "corrected": "Goodhart CAE (1984) Problems of monetary management: the UK experience. In: \\emph{Monetary Theory and Practice}. \\hl{London: Macmillan Press}, 91--121.",
    "evidence": "I base this correction on the front matter of a reprint of the first edition: its title page reads ``MACMILLAN PRESS'' and ``LONDON'', and its copyright page reads ``First published 1984 by'' and ``London and Basingstoke''.",
    "problem": "The entry gives Berlin and Springer as the place and publisher of the 1984 book, which was published by Macmillan Press in London. Springer hosts a later reprint.",
    "what_survives": "The entry identifies the chapter and its pages 91--121 correctly, and the discussion of Goodhart's law on pp.~3--4 does not depend on the imprint."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P048/#P048-N009",
   "work": {
    "arxiv_id": null,
    "doi": "10.1177/20531680241307942",
    "title": "Gnostic Notes on Temporal Validity",
    "url": "https://pmaronow.github.io/llm-audit/works/P048/"
   },
   "work_id": "P048"
  }
 ],
 "lean": {
  "applicable": false,
  "guide": null,
  "reason": "A six-page conceptual research note with no theorem, proposition, lemma, displayed formula, proof, estimator definition, table or computation: none of the five headline claims (P048-H1 to P048-H5) is tied to a numbered or formal result, and all ten confirmed errors (all Minor) are bibliographic, quotation-typography or grammatical misprints whose corrections change no mathematical statement, formula or proof step.",
  "repository": null,
  "targets": []
 },
 "missing_sources": [],
 "reproduction": {
  "agreement": "not_applicable",
  "status": "not_applicable",
  "targets": []
 },
 "summary": "This article, a research note, replies to Munger (2023), who argues that an agnostic analyst cannot produce findings that generalize to the future. I argue in the article that agnosticism is skeptical of assumptions and not free of them, that causal knowledge can be useful when a parameter does not transport, and that decay through adaptation can be studied. The article has no theorem, formula, table or data. 10 entries are classified as errors (10 Minor): slips in quotations, citations and reference entries, and one of grammar. None affects the argument.",
 "what_was_checked": "I read all six pages, with the ten notes and the reference list. The article has no theorem, formula, table, figure, data or code, and it reports no number that could be recomputed. The following were checked and held up.\n\\begin{itemize}\n\\item The statement on p.~2 that covariates are balanced in expectation under uniform random assignment holds in exact enumerations of complete randomization with four units and with five units.\n\\item The statement of note 5 that a sharp null hypothesis can be tested in a randomized experiment without structural assumptions holds in exact enumerations with four units and with six units, three of them treated: the randomization test rejects with probability at most its level at every attainable level.\n\\item All 29 entries of the reference list are cited in the text, every citation in the text has an entry, and notes 1 to 10 are called out in order.\n\\item The passages compared with the published text of Munger (2023) agree with it, apart from the hyphens in \\fid{P048-N001} and \\fid{P048-N003}: the three premises of the contradiction on p.~2, the words ``assumption-free'', the symbol $r$, the long quotation from its p.~3, the sentence quoted in note 7, and the page numbers 2, 3 and 6.\n\\item Stocking (1966, p.~875) calls Boas's posture ``agnostic'', as note 9 says. Lucas (1976) links parameter drift in part to agents who adapt their decision rules, which supports the description of Lucas's view on p.~3 when ``learning'' is read broadly.\n\\item Holland (1986, p.~948) treats the assumptions of his scientific solutions under the headings ``Temporal Stability and Causal Transience'' and ``Unit Homogeneity''. The ``temporal consistency'' of p.~2 is the article's own label, given without quotation marks in a list of examples.\n\\end{itemize}\nThe following were not checked. The quotations from Simon (2001) and their page numbers could not be compared with the published chapter; an archived draft of the chapter contains several of the quoted phrases, with ``scientist's hunting license'' for the printed ``scientists’ hunting license'' and with different pagination. The other quotations from Munger (2023) were not compared with that article. The quotations from the other cited works and the descriptions of them, among them Lin (2013), Rubin (1974), Boas (1920), Goodhart (1984), Dranove et al. (2003) and Recht et al. (2019), were not compared with their sources, and neither were the reference entries other than those for Björkegren et al. (2020), Goodhart (1984), Hastie et al. (2009), Hendrycks et al. (2021) and Moszkowski (2014). The statement about Einstein in note 8 was not checked. The concluding remarks name three technical approaches without citations, and there was nothing to check them against.\n\nThe article has no supplement, published correction or replication files. I used public copies for the source comparisons described here. The absence of further corrections is not a finding that the article is correct.",
 "work": {
  "arxiv_id": null,
  "audit_complete": true,
  "authors": [
   "Austin Jang",
   "Molly Offer-Westort",
   "Serena Wang",
   "P. M. Aronow"
  ],
  "category": "Essays, Commentary, Reports",
  "citation": "Austin Jang, Molly Offer-Westort, Serena Wang, and P. M. Aronow (2024). Gnostic Notes on Temporal Validity. Research and Politics 11(4).",
  "doi": "10.1177/20531680241307942",
  "edition": {
   "pages_in_audited_file": 6,
   "sha256": "1f7fe1854e51663255476027b28d3573b19dc2b697fba56cbb4597b4eee5fdd4",
   "source_url": "https://doi.org/10.1177/20531680241307942",
   "venue": "Research and Politics",
   "year": "2024"
  },
  "id": "P048",
  "later_materials": null,
  "title": "Gnostic Notes on Temporal Validity",
  "type": "Commentary/article",
  "url": "https://pmaronow.github.io/llm-audit/works/P048/",
  "venue": "Research and Politics",
  "year": "2024"
 }
}