LLM-generated corrections to the published research of P. M. Aronow

LLM-generated corrections to the published research of P. M. Aronow

This site lists corrections to my published research, as generated by large language models and as supervised by me (P. M. Aronow). The works are mine and those of my coauthors. The record shows no consultation with my coauthors, and nothing here speaks for them. The corrections began with an audit that multiple large language models carried out at my request.

This site is still a work in progress. It is subject to change as investigations continue and as models improve.

Almost all text on this website is machine-generated. I have personally reviewed all errors classified as moderate, major or critical, and a portion of those classified as minor or as clarity notes. My review is ongoing. Apart from earlier official corrections that an entry restates, no correction listed here is recorded as officially accepted by my coauthors or by a journal. This is an unofficial resource that is being provided without warranty as a service to the academic community.

The page of each work names the edition that was audited, says what was checked and what was not, and lists the corrections, the most serious first. A correction has a permanent identifier and a severity. It gives the passage as printed, the problem, the evidence, what survives and the corrected passage. Clarity notes are listed separately, and the About page defines them.

Two cautions apply to every page. First, the audit cannot say how many errors it missed. An estimate of its recall—the proportion of all errors it found—would need a benchmark of errors established independently of the models used in the audit, and it has no such benchmark. Second, where no correction is listed, the audit does not find that the work is correct. Where the audit reports no error, at least two reads and the recorded checks raised no error that was confirmed. As part of this verification effort, I have also used large language models to formalize and prove in Lean, a proof assistant, selected mathematical results of my work and the corrections to its mathematical statements, with their counterexamples. The current formal statements and proofs are maintained in llm-audit-lean; each formal target links to its statement and proof. I have not examined these proofs personally. These formalizations were built as part of the verification effort, and are being provided without warranty as a potentially helpful resource. They have not been vetted to the standards of other posted formalizations, e.g., on Palomar. A proof establishes its Lean statement under the stated assumptions. Whether that statement captures the printed or corrected passage depends on a separate comparison with the text.

The audit in numbers

Counts for the whole audit
Works audited (the planned checks were completed for every work; my review continues)52
Possible errors examined1314
Classified as errors796
Critical1
Major7
Moderate109
Minor679
Clarity notes158
Not confirmed212
Undetermined87
Outside the scope61

An entry is classified as an error when independent model checks confirmed it against the published text. Most are Minor (679 of 796: real but inconsequential). Three restate corrections the authors had already published. Neither the number of errors the audit missed nor the number of entries wrongly classified as errors is known.

Listed on this site: 796 corrections, 158 clarity notes, and 2 entries examined and not confirmed, in 52 works.

Works

Books (1)

Articles: theory and methodology (31)

Articles: empirics (12)

Book chapters (2)

Essays, commentary and reports (6)

Formal results: 805 whole; 110 restricted; 73 refuted; 33 omitted.

Coverage concerns the identified claim under its stated assumptions. A checked proof establishes the formal statement; correspondence to the published text was reviewed by AI. Repository access is currently required to open the source links.