{ "claims": [ { "claim": "The letter reports a nationally representative survey fielded through the 2014 Cooperative Congressional Election Study, with $n=1384$.", "claim_id": "P054-H1", "related_corrections": [ "P054-N001" ], "tied_to": "No numbered result (the letter has no tables, figures or numbered results). In-text statement of the sample, p. 223 (PDF p. 1), column 1, second paragraph; codebook counts for YAL401-YAL405." }, { "claim": "77% (95% CI 75-79) of Americans favoured universal background checks, yet only 53% (51-56) reported preferring stricter gun control laws: support for universal checks is substantially greater than preference for stricter laws.", "claim_id": "P054-H2", "related_corrections": [ "P054-N001" ], "tied_to": "No numbered result (the letter has no tables, figures or numbered results). In-text estimates, p. 223 (PDF p. 1), column 1, second paragraph; survey items YAL402 and YAL401 (or YAL405)." }, { "claim": "41% (38-43) of Americans believed universal background checks are already required by federal law, 47% (44-50) correctly said a check is required only for some purchases, and 12% (10-14) believed no federal check is required.", "claim_id": "P054-H3", "related_corrections": [ "P054-N001" ], "tied_to": "No numbered result (the letter has no tables, figures or numbered results). In-text estimates, p. 223 (PDF p. 1), column 1, second paragraph; survey item YAL403." }, { "claim": "Among people who strongly supported universal background checks, correct knowledge of the law predicted support for stricter gun laws: 74% (69-79) of those who wrongly believed checks are universal supported stricter laws against 89% (86-92) of those who knew some sales are exempt. No such relation was recorded among those who did not strongly favour universal background checks.", "claim_id": "P054-H4", "related_corrections": [ "P054-N001" ], "tied_to": "No numbered result (the letter has no tables, figures or numbered results). In-text estimates, p. 223 (PDF p. 1), end of column 1 and top of column 2; cross-tabulation of YAL402, YAL403 and YAL401 (or YAL405)." }, { "claim": "There is an engagement gap: among voters who thought gun laws should be less strict, 71% (65-78) said they would never vote for a candidate who did not share their position on gun control, compared with 34% (30-37) of those who support stricter gun laws.", "claim_id": "P054-H5", "related_corrections": [ "P054-N001", "P054-N005" ], "tied_to": "No numbered result (the letter has no tables, figures or numbered results). In-text estimates, p. 223 (PDF p. 1), column 2, second paragraph; survey items YAL404 and YAL401." }, { "claim": "The letter notes that many people mistakenly believe strong gun laws are already enacted and do not support new legislation. It suggests that poor awareness of existing law might contribute to the difference between 77% support for universal checks and 53% preference for stricter laws, and that public education could help mobilise support.", "claim_id": "P054-H6", "related_corrections": [], "tied_to": "No numbered result (the letter has no tables, figures or numbered results). Interpretation of the in-text $77\\%$ versus $53\\%$ contrast, the $41\\%$ misperception estimate and the $74\\%$ (interval $[69\\%,79\\%]$) versus $89\\%$ (interval $[86\\%,92\\%]$) comparison among strong supporters; p. 223 (PDF p. 1), column 1, second paragraph, and the closing sentences of column 2. No education intervention is reported." } ], "date_published": "2026-10-25", "findings": [ { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "recalculation_from_source", "exhibits": [ { "alt": "Crop of the second paragraph of the first column of page 223, 23 lines, from 'To answer this question, we undertook a nationally representative survey through the 2014 Cooperative Congressional Election Study (n=1384)' to '12 per cent (10-14) believed that no federal background checks are required.' Red outlines surround five ranges: 75-79, which follows '77 per cent (95 per cent CI', and the ranges in parentheses (51-56), (38-43), (44-50) and (10-14), which follow 53, 41, 47 and 12 per cent.", "caption": "Column 1, second paragraph, p. 223 (PDF p. 1). The outlines mark the first five printed intervals: 75–79, 51–56, 38–43, 44–50 and 10–14.", "url": "https://pmaronow.github.io/llm-audit/works/P054/exhibits/P054-N001.png" }, { "alt": "Crop of the first 22 lines of the second column of page 223, from 'existing laws predicted support for stricter gun laws' to 'laws. However, the role of policy'. Red outlines surround four ranges: 69-79, which follows '74 per cent (95 per cent CI'; (86-92), which follows 89 per cent; 65-78, which follows '71 per cent (95 per cent CI'; and (30-37), which follows 34 per cent. Between them the text reads 'We recorded no such relation among those who did not strongly favour universal background checks' and 'We also identified evidence of an engagement gap: among voters who thought gun laws should be less strict than at present'.", "caption": "Column 2, first 22 lines, p. 223 (PDF p. 1). The outlines mark the other four printed intervals: 69–79, 86–92, 65–78 and 30–37.", "url": "https://pmaronow.github.io/llm-audit/works/P054/exhibits/P054-N001-b.png" }, { "alt": "Chart titled 'Allowing for the survey weights widens all nine 95 per cent intervals', with the subtitle 'Standard errors are 1.28 to 1.47 times larger; the nine percentages do not change'. The horizontal axis is percentage points below or above the estimate, from -9 to +9. Each of nine rows has an orange interval, which rounds to the printed one, above a wider blue interval, which allows for the survey weights; both are centred at zero. To the right of each row are the group size, the printed estimate and interval, and the corrected one. All respondents, n = 1384: favour universal background checks, 77 (75-79) and 77 (74-80); prefer stricter gun laws, 53 (51-56) and 53 (50-57); believe checks are already universal, 41 (38-43) and 41 (37-45); say checks cover only some purchases, 47 (44-50) and 47 (43-51); believe no checks are required, 12 (10-14) and 12 (10-15). Strong supporters of universal checks, share preferring stricter gun laws: believe checks are already universal, n = 306, 74 (69-79) and 74 (67-81); know some sales are exempt, n = 334, 89 (86-92) and 89 (85-93). Share who would never vote for someone with a different position on gun control: want less strict gun laws, n = 191, 71 (65-78) and 71 (63-80); want stricter gun laws, n = 746, 34 (30-37) and 34 (29-39). The widest blue interval, for n = 191, reaches about 8.6 points on each side of the estimate, against about 6.4 for the orange one.", "caption": "The nine 95% intervals, each drawn relative to its estimate. Orange: $\\hat p\\pm1.96\\sqrt{V_0}$, which rounds to the printed interval in every row. Blue: $\\hat p\\pm1.96\\sqrt{V}$, the stated independent-respondent survey linearisation allowing for the weights. Orange reconstructs the printed arithmetic without identifying the historical method; in the stricter-law row, the unrounded blue lower endpoint is $49.51388\\%$. The columns at the right give the size of the group, the estimate and interval as printed, and the corrected interval, in per cent.", "url": "https://pmaronow.github.io/llm-audit/works/P054/exhibits/P054-N001-intervals.png" } ], "formal_release": null, "formal_statements": [ { "id": "P054-T1", "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-T1" }, { "id": "P054-T2", "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-T2" } ], "headline": "Under the stated independent-respondent survey linearisation, all nine intervals widen and the 53% stricter-law interval includes 50%; the nine estimates and the three main positive comparisons remain.", "id": "P054-N001", "location": "Columns 1 and 2, the nine percentages printed with 95% confidence intervals, from “77% (95% CI 75–79)” to “34% (30–37)”, p. 223 (PDF p. 1)", "pdf_page": 1, "printed_page": "223", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Moderate", "subject": "paper", "target_ids": [ "P054-T1", "P054-T2" ], "text": { "as_printed": "To answer this question, we undertook a nationally representative survey through the 2014 Cooperative Congressional Election Study (n=1384). [\\ldots] 77\\% (95\\% CI \\hl{75--79}) of Americans favoured universal background checks, yet only 53\\% (\\hl{51--56}) reported preferring stricter gun control laws. [\\ldots] We found that 41\\% (\\hl{38--43}) of Americans believed that universal background checks are already required by federal law, whereas 47\\% (\\hl{44--50}) correctly reported that a federal background check is required only for some gun purchases, and 12\\% (\\hl{10--14}) believed that no federal background checks are required.\n\n[\\ldots] Whereas 74\\% (95\\% CI \\hl{69--79}) of those who incorrectly believed that the USA has universal background checks supported stricter gun laws, 89\\% (\\hl{86--92}) of those who knew that some gun sales do not require background checks supported stricter laws. [\\ldots] 71\\% (95\\% CI \\hl{65--78}) said they would never vote for a political candidate who did not share their position on gun control, compared with just 34\\% (\\hl{30--37}) of those who support stricter gun laws.", "corrected": "77\\% (95\\% CI \\hl{74--80}) of Americans favoured universal background checks, yet only 53\\% (\\hl{50--57}) reported preferring stricter gun control laws. [\\ldots] An estimated 41\\% (\\hl{37--45}) of Americans believed that universal background checks are already required by federal law, whereas 47\\% (\\hl{43--51}) correctly reported that a federal background check is required only for some gun purchases, and 12\\% (\\hl{10--15}) believed that no federal background checks are required.\n\n[\\ldots] \\hl{Among those who strongly supported universal background checks,} 74\\% (95\\% CI \\hl{67--81}) of those who incorrectly believed that the USA has universal background checks supported stricter gun laws, \\hl{whereas} 89\\% (\\hl{85--93}) of those who knew that some gun sales do not require background checks supported stricter laws. [\\ldots] \\hl{Among respondents who thought gun laws should be less strict than at present,} 71\\% (95\\% CI \\hl{63--80}) said they would never vote for a political candidate who did not share their position on gun control, compared with just 34\\% (\\hl{29--39}) of those who support stricter gun laws.\n\nThe letter should also state the release of the data, the respondents selected, the weights and the interval method that were used. If they are those of the recalculation, a sentence such as the following would serve: \\hl{Estimates use the team weight of the February 2015 release of the survey data, on the 1384 respondents with valid answers to the three questions on gun laws, on universal background checks and on existing federal law. The 95\\% confidence intervals use a linearised variance that allows for the weights and treats respondents as independent.}\n\nThese replacement intervals are conditional on the following approximation; they are not independently established full-design national intervals. The corrected intervals are $\\hat p\\pm1.96\\sqrt V$, with $\\hat p$ and $V$ as defined under Evidence. They are approximate 95\\% intervals under three conditions: respondents can be treated as independent draws with the released weights; the linear and normal approximations are adequate; and the released weights identify the national population, with the retained respondents, including those answering the engagement question, identifying the same proportions in each group. If the variance under the full design of the survey is available, it should replace $V$. The printed limits could be kept only with a documented inferential justification that accounts for the survey design and response selection.", "evidence": "I recalculated the nine estimates from the public survey files (2014 Cooperative Congressional Election Study, Yale team module, Harvard Dataverse, \\url{https://doi.org/10.7910/DVN/FBXY8V}). Exact rational comparisons confirmed the rounded percentages and endpoints of both intervals in the table.\n\nThe following notation is introduced for the recalculation; the letter prints no formula. Fix one of the nine estimates. For respondent $i$, let $d_i\\in\\{0,1\\}$ indicate that $i$ belongs to the group and gave a valid answer, let $y_i\\in\\{0,1\\}$ be the answer, and let $w_i>0$ be the weight. The sums below run over the $N=1384$ respondents. Set $n=\\sum_i d_i$, $W=\\sum_i d_iw_i$ and $\\hat p=\\sum_i d_iw_iy_i/W$, and define\n\\[\nV_0=\\frac{\\hat p(1-\\hat p)}{n},\\qquad V=\\frac{N}{N-1}\\cdot\\frac{\\sum_i d_iw_i^2(y_i-\\hat p)^2}{W^2}.\n\\]\nThe variance $V$ comes from linearising the ratio $\\hat p=\\bar a/\\bar b$, where $\\bar a=N^{-1}\\sum_i d_iw_iy_i$ and $\\bar b=W/N$. To first order, a change in $(\\bar a,\\bar b)$ changes the ratio by $(\\mathrm{d}\\bar a-\\hat p\\,\\mathrm{d}\\bar b)/\\bar b$, so respondent $i$ enters through the residual $z_i=d_iw_i(y_i-\\hat p)$. These residuals sum to zero. Their empirical variance is $\\sum_i z_i^2/(N-1)$, and the estimated variance of their mean is $\\sum_i z_i^2/(N(N-1))$. Dividing by $\\bar b^2=W^2/N^2$ gives $V$. If all weights are equal, then $\\sum_i d_iw_i^2(y_i-\\hat p)^2/W^2=\\hat p(1-\\hat p)/n$, and $V$ equals $V_0$ up to the factor $N/(N-1)$. The standard errors $\\sqrt V$ agree with those of the survey package of R for a design with independent respondents and these weights.\n\nFor an intercept-only weighted least-squares fit with classical standard errors, binary responses give\n\\[\n\\sum_i d_iw_i(y_i-\\hat p)^2=W\\hat p(1-\\hat p),\\qquad\nV_{\\mathrm{classical}}=\\frac{\\hat p(1-\\hat p)}{n-1}.\n\\]\nThus entering the team weights in R's \\texttt{lm} leaves a classical intercept variance differing from $V_0$ only by the factor $n/(n-1)$. Numerical agreement with the print establishes compatibility with the method I report using, not its historical use or the inverse-variance assumptions that would justify its classical SEs. Both matching formulas remain distinct from the survey-ratio variance $V$.\n\nThe estimates are coded as follows. Favouring universal checks is the answer ``strongly favor'' or ``favor'', and strong support is ``strongly favor''. Preferring stricter laws is the answer ``more strict'' to the question with three answers. The last two estimates are the share answering ``No'' to the question whether one could still vote for someone standing for office with whom one disagrees on gun control. Respondents without a valid answer to it are left out.\n\n\\begin{audittable}{lrrlllr}{The nine estimates in the order of the letter. Printed: percentage and 95\\% interval as printed. $n$: respondents in the group. $\\hat p$: weighted percentage. Fourth and fifth columns: the two intervals, in per cent. Rounded: the fifth column at the precision of the letter. Last column: ratio of the two standard errors.}\nPrinted & $n$ & $\\hat p$ (\\%) & $\\hat p\\pm1.96\\sqrt{V_0}$ & $\\hat p\\pm1.96\\sqrt{V}$ & Rounded & $\\sqrt{V/V_0}$ \\\\\n77 (75--79) & 1384 & 76.82 & 74.60--79.05 & 73.68--79.97 & 74--80 & 1.41 \\\\\n53 (51--56) & 1384 & 53.24 & 50.61--55.87 & 49.51--56.96 & 50--57 & 1.42 \\\\\n41 (38--43) & 1384 & 40.82 & 38.23--43.41 & 37.08--44.56 & 37--45 & 1.44 \\\\\n47 (44--50) & 1384 & 47.05 & 44.42--49.68 & 43.34--50.76 & 43--51 & 1.41 \\\\\n12 (10--14) & 1384 & 12.13 & 10.41--13.85 & 9.67--14.58 & 10--15 & 1.43 \\\\\n74 (69--79) & 306 & 73.90 & 68.98--78.82 & 66.86--80.95 & 67--81 & 1.43 \\\\\n89 (86--92) & 334 & 89.15 & 85.81--92.48 & 84.89--93.40 & 85--93 & 1.28 \\\\\n71 (65--78) & 191 & 71.16 & 64.74--77.59 & 62.52--79.81 & 63--80 & 1.35 \\\\\n34 (30--37) & 746 & 33.82 & 30.43--37.22 & 28.85--38.80 & 29--39 & 1.47 \\\\\n\\end{audittable}\n\nIn every row the fourth column rounds to the printed interval and the fifth does not. The ratio $V/V_0$ ranges from approximately $1.626$ to $2.147$. It differs from one estimate to the next, since it depends on the answers as well as on the weights. No common rescaling factor was used. Without weights, or with the weights of the August 2015 release, the nine percentages are not all reproduced: the rounded values are 77, 54, 39, 49, 12, 76, 88, 71, 33 and 75, 53, 40, 48, 12, 73, 88, 73, 33. The five tested interval methods for a proportion in the survey package of R, applied to the same design, reproduce none of the nine printed intervals. If each group is treated as a separate sample, the factor $N/(N-1)$ becomes $n/(n-1)$; eight rounded intervals are unchanged, and the eighth row becomes 62--80.", "problem": "The letter states neither its weights, nor the rule that gives $n=1384$, nor how its intervals were computed. The public survey files reproduce all nine printed percentages and intervals under the reconstruction below. With the team weight of the February 2015 release, on the 1384 respondents with valid answers to the first three questions about guns, each of the nine percentages agrees with the print after rounding. Let $\\hat p$ be one of these weighted proportions and let $n$ be the unweighted number of respondents in its group. Then the interval $\\hat p\\pm1.96\\sqrt{\\hat p(1-\\hat p)/n}$ rounds to the printed interval in all nine cases, at both endpoints.\n\nThis interval uses the weights in $\\hat p$ but does not account for their unequal influence in the variance. It treats $\\hat p$ as the proportion in a simple random sample of $n$ respondents with equal weights. The weights of the 1384 respondents are unequal: they range from $0.05$ to $7.00$. The usual linearised variance of a weighted proportion, $V$ in the display under Evidence, allows for them. With $V$, every standard error is larger, by a factor ranging from approximately $1.28$ to $1.47$, and every interval is wider. For example, where the letter prints 74\\% (69--79), the interval becomes 74\\% (67--81). Under this approximation, the printed intervals overstate the precision of all nine estimates. No percentage changes.\n\nTwo limits apply. First, the rounded figures do not identify the original calculation uniquely. An actual intercept-only weighted least-squares fit in R, with classical standard errors and either normal or default $t$ critical values, also gives the same nine intervals after rounding, and six other rules for selecting respondents reproduce every printed percentage and interval. I report using classical intercept-only weighted least squares in R for the original analysis. That method is compatible with every printed number, while its historical use is not independently established by the letter, the available deposit or rounded agreement. Each matching calculation omits the squared-weight residual term from the survey-ratio variance, and under each of the seven rules none of the nine intervals based on the same linearisation rounds to the printed one. Second, $V$ is the ordinary approximation for independent respondents. Each release of the public files holds at most one weight and no strata, clusters or replicate weights, and the files do not document how the weight was built. The variance under the full design of the survey cannot be computed from them, and whether any of these intervals covers the national proportion 95\\% of the time is not established.", "what_survives": "All nine percentages are reproduced and stand, and the three knowledge percentages still sum to 100. The three comparisons that the letter draws keep 95\\% intervals that exclude zero when the same linearisation is applied to each difference. The difference between 77\\% and 53\\% is $23.59$ percentage points ($19.83$ to $27.34$). The difference between 89\\% and 74\\% among strong supporters is $15.24$ points ($7.01$ to $23.47$), and the difference between 71\\% and 34\\% is $37.34$ points ($27.36$ to $47.32$). The printed precision of each estimate is lost under this approximation. One confidence inference also changes: the matching stricter-law interval is $50.60848$--$55.86594\\%$, while the interval using $V$ is $49.51388$--$56.96055\\%$. The latter contains $50\\%$, so a simple majority preferring stricter laws is no longer established by the reported two-sided 95\\% interval criterion. The paper does not explicitly report a majority hypothesis test, and this change does not reverse the $77\\%$-versus-$53\\%$ comparison. Neither calculation establishes a causal education effect or statistical equivalence to zero among non-strong supporters. The group to which the last two estimates refer is corrected in \\fid{P054-N005}." }, "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-N001", "work": { "arxiv_id": null, "doi": "10.1016/S0140-6736(16)00042-8", "title": "Policy Misperceptions and Support for Gun Control Legislation", "url": "https://pmaronow.github.io/llm-audit/works/P054/" }, "work_id": "P054" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "recalculation_from_source", "exhibits": [ { "alt": "Crop of twelve lines of the second column of page 223. The first two lines end the preceding paragraph: 'those who did not strongly favour universal background checks.' The rest reads: 'We also identified evidence of an engagement gap: among voters who thought gun laws should be less strict than at present, 71 per cent (95 per cent CI 65-78) said they would never vote for a political candidate who did not share their position on gun control, compared with just 34 per cent (30-37) of those who support stricter gun laws. However, the role of policy'. A red outline surrounds the word voters.", "caption": "Column 2, second paragraph, p. 223 (PDF p. 1), with the last two lines of the paragraph before it. The outline marks the word “voters”.", "url": "https://pmaronow.github.io/llm-audit/works/P054/exhibits/P054-N005.png" } ], "formal_release": null, "formal_statements": [], "headline": "In the engagement comparison, “among voters” should read “among respondents”: the 71% and 34% are reproduced without a voter restriction, in groups that include people who did not vote.", "id": "P054-N005", "location": "Column 2, second paragraph, sentence beginning “We also identified evidence of an engagement gap”, p. 223 (PDF p. 1)", "pdf_page": 1, "printed_page": "223", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "We also identified evidence of an engagement gap: among \\hl{voters} who thought gun laws should be less strict than at present, 71\\% (95\\% CI \\hl{65--78}) said they would never vote for a political candidate who did not share their position on gun control, compared with just 34\\% (\\hl{30--37}) of those who support stricter gun laws.", "corrected": "We also identified evidence of an engagement gap: among \\hl{respondents} who thought gun laws should be less strict than at present, 71\\% (95\\% CI \\hl{63--80}) said they would never vote for a political candidate who did not share their position on gun control, compared with just 34\\% (\\hl{29--39}) of those who support stricter gun laws.\n\nIf a comparison among voters is wanted instead, the restriction must be defined and both figures recomputed. Among respondents who report that they definitely voted in November 2014, the two shares are 72\\% and 31\\%.", "evidence": "The evidence is a recalculation from the public files of the survey (2014 Cooperative Congressional Election Study, Yale team module, Harvard Dataverse, \\url{https://doi.org/10.7910/DVN/FBXY8V}). The rounding of the percentages and endpoints quoted here was confirmed in exact rational arithmetic. Each share is the weighted proportion $p$ answering ``No'' to the question whether one could still vote for someone standing for office with whom one disagrees on gun control. I use the team weight of the February 2015 release and reproduce the printed intervals with $p\\pm1.96\\sqrt{p(1-p)/n}$ for a group of $n$ respondents. I repeated the calculation in the three releases of the data, with the team weight where a release has one and without weights. I also restricted the groups by reported turnout, by registration and, in the August 2015 release, by the match to the voter file. Among the calculations I tried, both printed percentages are reproduced only with the team weight of the February 2015 release and no restriction. The rounded figures do not fix the sample or interval method exactly. Without the requirement of valid answers to the first three questions about guns, the groups have 192 and 750 respondents, and the rounded figures are the same. The analysis code of the letter is not available.", "problem": "The letter does not define ``voters''. From the public files of the survey, I reproduce the printed 71\\% (65--78) and 34\\% (30--37) with no restriction to voters. Using the 1384 respondents with valid answers to the first three questions about guns and excluding missing engagement answers, the two groups have 191 and 746 respondents. They include 18 and 149 respondents who report that they did not vote in November 2014, and 14 and 59 who report that they are not registered to vote. When I restrict the groups to respondents who report that they definitely voted in that election, I obtain 72\\% (65--79) and 31\\% (27--35) by the same calculation. These are not the printed figures. As a description of people who voted, the word ``voters'' therefore misdescribes these groups.", "what_survives": "The comparison stands as one among respondents, 71\\% against 34\\%. Among respondents who report that they definitely voted, the gap is of similar size, 72\\% against 31\\%. Changing this population label does not alter any other estimate in the letter. The corrected intervals below allow for the survey weights under the independent-respondent approximation described in \\fid{P054-N001}." }, "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-N005", "work": { "arxiv_id": null, "doi": "10.1016/S0140-6736(16)00042-8", "title": "Policy Misperceptions and Support for Gun Control Legislation", "url": "https://pmaronow.github.io/llm-audit/works/P054/" }, "work_id": "P054" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of six lines of the reference list on page 223: the last two lines of reference 1, which end with the access date March 30, 2015, and the four lines of reference 2. Reference 2 reads: Fleegler EW, Lee LK, Monteaux MC, Hemenway D, Mannix R. Firearm legislation and firearm-related fatalities in the United States. JAMA Intern Med 2013; 173: 732-40. A red outline surrounds the word Monteaux.", "caption": "Reference 2, p. 223 (PDF p. 1), foot of the second column. The outline marks the surname “Monteaux”.", "url": "https://pmaronow.github.io/llm-audit/works/P054/exhibits/P054-N006.png" } ], "formal_release": null, "formal_statements": [], "headline": "In reference 2 the surname of the third author should read “Monuteaux”, not “Monteaux”.", "id": "P054-N006", "location": "Reference 2, p. 223 (PDF p. 1), foot of the second column, third author", "pdf_page": 1, "printed_page": "223", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "Fleegler EW, Lee LK, \\hl{Monteaux} MC, Hemenway D, Mannix R. Firearm legislation and firearm-related fatalities in the United States. \\emph{JAMA Intern Med} 2013; \\textbf{173:} 732--40.", "corrected": "Fleegler EW, Lee LK, \\hl{Monuteaux} MC, Hemenway D, Mannix R. Firearm legislation and firearm-related fatalities in the United States. \\emph{JAMA Intern Med} 2013; \\textbf{173:} 732--40.", "evidence": "I checked the page against a public source. On the publisher's page for the article (\\url{https://doi.org/10.1001/jamainternmed.2013.1286}), the byline names the third author as ``Michael C. Monuteaux, ScD'', and the citation of the article prints ``Monuteaux MC''. The title, journal, year, volume and pages of reference 2 agree with that page.", "problem": "The third author of the cited article is Michael C. Monuteaux. The reference omits the ``u'' that follows ``Mon'' in his surname.", "what_survives": "The article remains identifiable from its title, journal, volume and pages. No estimate or argument in the letter depends on the spelling." }, "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-N006", "work": { "arxiv_id": null, "doi": "10.1016/S0140-6736(16)00042-8", "title": "Policy Misperceptions and Support for Gun Control Legislation", "url": "https://pmaronow.github.io/llm-audit/works/P054/" }, "work_id": "P054" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of six lines of the reference list on page 223: the last line of reference 2, the four lines of reference 3, and the first line of reference 4, Quinnipiac University Polling Institute. Reference 3 reads: Webster D, Crifasi KC, Vernick JS. Effects of the repeal of Missouri's handgun purchaser licensing law on homicides. J Urban Health 2014; 91: 293-302. A red outline surrounds the initials KC and the comma after them.", "caption": "Reference 3, p. 223 (PDF p. 1), foot of the second column. The outline marks the initials “KC”.", "url": "https://pmaronow.github.io/llm-audit/works/P054/exhibits/P054-N007.png" } ], "formal_release": null, "formal_statements": [], "headline": "In reference 3 the initials of the second author should read “CK”, not “KC”.", "id": "P054-N007", "location": "Reference 3, p. 223 (PDF p. 1), foot of the second column, second author", "pdf_page": 1, "printed_page": "223", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "Webster D, Crifasi \\hl{KC}, Vernick JS. Effects of the repeal of Missouri’s handgun purchaser licensing law on homicides. \\emph{J Urban Health} 2014; \\textbf{91:} 293--302.", "corrected": "Webster D, Crifasi \\hl{CK}, Vernick JS. Effects of the repeal of Missouri’s handgun purchaser licensing law on homicides. \\emph{J Urban Health} 2014; \\textbf{91:} 293--302.", "evidence": "I checked the page against a public source. On the publisher's page for the article (\\url{https://doi.org/10.1007/s11524-014-9865-8}), the byline names the second author as ``Cassandra Kercher Crifasi'', and the citation of the article prints ``Crifasi, C.K.'' The publisher's page for the erratum to the article (\\url{https://doi.org/10.1007/s11524-014-9882-7}) has the same byline and the same form of citation. The title, journal, year, volume and pages of reference 3 agree with the publisher's page for the article.", "problem": "The second author of the cited article is Cassandra Kercher Crifasi, whose initials are CK. The reference prints them in the reverse order.", "what_survives": "The article remains identifiable from its title, journal, volume and pages. No estimate or argument in the letter depends on the order of the initials." }, "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-N007", "work": { "arxiv_id": null, "doi": "10.1016/S0140-6736(16)00042-8", "title": "Policy Misperceptions and Support for Gun Control Legislation", "url": "https://pmaronow.github.io/llm-audit/works/P054/" }, "work_id": "P054" } ], "lean": { "applicable": true, "guide": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/docs/works/P054.md", "repository": "https://github.com/pmaronow/llm-audit-lean", "targets": [ { "declarations": [ { "claim_revision_id": "catalog:2a6fa35d0209d8f8c1a70fad9566c58a637917e63b4e316283441d09a30c2f68", "conditional_on": [], "credit_reason": "This assesses the corrected reconstruction target, not an unprinted formula attributed to the original letter. The three exported identities jointly give the stated equal-weight V=N/(N-1) times V0 consequence. All denominators are nonzero from the explicit assumptions. No claim-stage verdict changed after reading the record.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P054.ratio_reconstruction", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P054/P054_Reconstruction.lean#L117", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:2a6fa35d0209d8f8c1a70fad9566c58a637917e63b4e316283441d09a30c2f68", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P054-T1", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P054-T1" }, "html": { "body": "

Letter p.223; audit P054-N001, final Evidence formulas. For fixed binary data and positive weights, ratio residuals sum to zero, the normalized linearisation variance has the squared-weight formula, and equal weights give both the reference variance inside that formula and its N/(N-1) correction. These are conditional reconstruction identities; they establish no historical procedure, survey-design validity, or national confidence-interval coverage.

", "headline": "Audit reconstruction: weighted-ratio residual and linearisation identities", "location": "Letter p.223; audit P054-N001, final Evidence formulas" }, "id": "P054-T1", "text": { "body": "Letter p.223; audit P054-N001, final Evidence formulas. For fixed binary data and positive weights, ratio residuals sum to zero, the normalized linearisation variance has the squared-weight formula, and equal weights give both the reference variance inside that formula and its N/(N-1) correction. These are conditional reconstruction identities; they establish no historical procedure, survey-design validity, or national confidence-interval coverage.", "headline": "Audit reconstruction: weighted-ratio residual and linearisation identities", "location": "Letter p.223; audit P054-N001, final Evidence formulas" }, "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-T1" }, { "declarations": [ { "claim_revision_id": "catalog:c1f909dd6a172dfeb32252dbc8ba7dada79907544d60c47d2dde24aa857c56b4", "conditional_on": [], "credit_reason": "This assesses the reconstruction formula with classical variance defined as RSS_w/((n-1)W). The arbitrary finite index type is equivalent to finite observation labels, and n>=2 implies at least two selected observations. No minimization, sampling validity, or historical-use conclusion is required by this algebraic target. All denominators are nonzero. No claim-stage verdict changed after reading the record.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P054.classical_intercept_wls", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P054/P054_Reconstruction.lean#L140", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:c1f909dd6a172dfeb32252dbc8ba7dada79907544d60c47d2dde24aa857c56b4", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P054-T2", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P054-T2" }, "html": { "body": "

Letter p.223; audit P054-N001, final Evidence classical-WLS formula. For at least two selected respondents, the weighted residual sum of squares is W*p*(1-p), and the classical intercept-only variance is both p*(1-p)/(n-1) and n/(n-1) times V0. This algebra gives compatibility with the reported calculation, without establishing historical use or the inverse-variance assumptions underlying classical standard errors.

", "headline": "Audit reconstruction: classical intercept-only weighted least-squares variance", "location": "Letter p.223; audit P054-N001, final Evidence classical-WLS formula" }, "id": "P054-T2", "text": { "body": "Letter p.223; audit P054-N001, final Evidence classical-WLS formula. For at least two selected respondents, the weighted residual sum of squares is W*p*(1-p), and the classical intercept-only variance is both p*(1-p)/(n-1) and n/(n-1) times V0. This algebra gives compatibility with the reported calculation, without establishing historical use or the inverse-variance assumptions underlying classical standard errors.", "headline": "Audit reconstruction: classical intercept-only weighted least-squares variance", "location": "Letter p.223; audit P054-N001, final Evidence classical-WLS formula" }, "url": "https://pmaronow.github.io/llm-audit/works/P054/#P054-T2" } ] }, "missing_sources": [], "reproduction": { "agreement": "agrees", "status": "selected_recalculation", "targets": [ { "agreement": "exact", "location": "p. 223 (PDF p. 1), left column, paragraph beginning 'To answer this question'", "method": "own recalculation from the public survey data (R code): valid YAL401–403 responses in the February 2015 matched release reproduce the sample count; historical sample rule not uniquely identified", "status": "recalculated", "target": "In-text survey sample" }, { "agreement": "to_rounding", "location": "p. 223 (PDF p. 1), left column, survey-results paragraph", "method": "own recalculation from the public survey data (R code): February 2015 team-weighted percentages and simple binomial normal intervals reproduce the printed figures to rounding; historical choices not uniquely identified", "status": "recalculated", "target": "In-text policy support gap" }, { "agreement": "to_rounding", "location": "p. 223 (PDF p. 1), left column, survey-results paragraph", "method": "own recalculation from the public survey data (R code): February 2015 team-weighted percentages and simple binomial normal intervals reproduce the printed figures to rounding; historical choices not uniquely identified", "status": "recalculated", "target": "In-text knowledge of federal law" }, { "agreement": "to_rounding", "location": "p. 223 (PDF p. 1), last paragraph of left column continuing at top of middle column", "method": "own recalculation from the public survey data (R code): February 2015 team-weighted percentages and simple binomial normal intervals reproduce the printed figures to rounding; historical choices not uniquely identified", "status": "recalculated", "target": "In-text misperceptions and stricter-law support" }, { "agreement": "to_rounding", "location": "p. 223 (PDF p. 1), middle column, paragraph beginning 'We also identified evidence'", "method": "own recalculation from the public survey data (R code): February 2015 team-weighted percentages and simple binomial normal intervals reproduce the printed figures to rounding; historical choices not uniquely identified; respondents without a voter restriction (P054-N005)", "status": "recalculated", "target": "In-text engagement gap" } ] }, "summary": "This one-page letter asks whether mistaken beliefs about existing gun laws contribute to the lack of public mobilisation for new legislation. In a survey from the 2014 Cooperative Congressional Election Study ($n = 1384$), 77\\% favoured universal background checks, 53\\% preferred stricter gun laws, and 41\\% believed that universal checks are already federal law. 4 entries are classified as errors (1 Moderate, 3 Minor); 4 other concerns were not confirmed. Under an independent-respondent survey approximation with the released weights, all nine 95\\% intervals widen and the stricter-law interval includes 50\\%; the nine estimates and three main positive comparisons remain (\\fid{P054-N001}). The letter reports no majority hypothesis test, and coverage under the full survey design is not established. I report using intercept-only weighted least squares in R with classical standard errors; independent fits reproduce the rounded numbers but do not identify the original procedure.", "what_was_checked": "The letter is one page, with five references and no table, figure, formula or appendix, and I read all of it. I checked the following.\n\\begin{itemize}\n\\item Every number quoted here from the letter was confirmed on the page.\n\\item The letter's descriptions of the survey questions agree with the questionnaire and the codebooks of the survey. The questions are these: support for background checks for all gun buyers; the knowledge question with its three answers (all purchases, some, none); whether one could still vote for someone standing for office with whom one agrees on other issues but disagrees on gun control; and whether gun laws should be more strict, kept as they are, or less strict.\n\\item The three knowledge percentages, 41, 47 and 12, sum to 100. The three comparisons imply differences of 24, 15 and 37 percentage points, and each printed interval contains its estimate.\n\\item The codebooks record 1393 post-election respondents, and $n = 1384$ is attainable as the number with valid answers to the first three questions about guns ($1393 - 1 - 4 - 4$).\n\\item The printed 77\\% and 12\\% agree with the unweighted codebook counts, $1066/1389 = 76.7\\%$ and $172/1389 = 12.4\\%$. The printed 53\\%, 41\\% and 47\\% cannot be unweighted proportions on 1384 of these respondents, and they are compatible with weighted ones.\n\\item Each of the nine printed intervals agrees at the printed precision with $p \\pm 1.96\\sqrt{p(1-p)/n}$ for a suitable proportion and group size. The apparent asymmetry of 53 (51--56) and 41 (38--43) can arise from rounding. This is a check of internal consistency and not of the method behind the intervals.\n\\item A recalculation from the respondent-level file of the February 2015 release, with the team weight, on the 1384 respondents with valid answers to the first three questions about guns, reproduced to rounding all nine printed percentages. With $p \\pm 1.96\\sqrt{p(1-p)/n}$, where $n$ is the unweighted size of the group, it also reproduced all nine printed intervals. These choices give a compatible reconstruction; agreement does not uniquely identify the original release, weight, sample rule or interval method, which the letter does not document.\n\\item Under an independent-respondent survey linearisation with the released weights, all nine intervals widen; retaining covariance, the three reported differences retain 95\\% intervals that exclude zero (\\fid{P054-N001}). The letter itself does not specify the interval method.\n\\item Among respondents who did not strongly favour universal background checks, the same weighted calculation gives 30\\% support for stricter laws among those who believed checks cover all purchases and 16\\% among those who knew they cover only some. This opposite direction supports the letter's ``no such relation'' in its natural meaning of no such positive relation.\n\\item The engagement comparison's 71\\% and 34\\% are reproduced without restricting the groups to voters. The reconstructed groups include 18 and 149 respondents who explicitly reported not voting, so ``voters'' should be replaced by ``respondents'' (\\fid{P054-N005}).\n\\item For references 2 and 3, the titles, journals, years, volumes and pages agree with the publishers' pages. The author names are corrected in \\fid{P054-N006} and \\fid{P054-N007}.\n\\end{itemize}\nThe following were not checked. References 1, 4 and 5 were not compared with their sources, and neither were the statements that rest on them: more than 30~000 firearm deaths a year, and 92\\% of 1446 people polled. Which sales of guns are exempt from background checks under federal law was not researched. The suggestion that public education could mobilise support is framed as a possibility. The comparisons in the letter contain no information treatment and cannot identify an effect of education.\n\nNo analysis code for the letter was identified, and the letter does not say which release of the survey data or which weights it used. No supplement or published correction was identified. The absence of further corrections is not a finding that the letter is correct.\n\nI report that the original analysis used intercept-only weighted least squares in R with classical standard errors. Actual fits reproduce all nine survey percentages and their interval endpoints after rounding, establishing compatibility without independently identifying the original procedure. Under this survey linearisation the three principal difference intervals remain positive. For stricter-law support, the interval reconstructed with $p \\pm 1.96\\sqrt{p(1-p)/n}$ is $50.60848$--$55.86594\\%$, while the survey-linearised interval is $49.51388$--$56.96055\\%$, which includes $50\\%$. The paper does not explicitly report a majority hypothesis test, and these calculations do not establish coverage under the full national survey design or justify the weighting and response assumptions needed for population inference.", "work": { "arxiv_id": null, "audit_complete": true, "authors": [ "P. M. Aronow", "Benjamin T. Miller" ], "category": "Essays, Commentary, Reports", "citation": "P. M. Aronow and Benjamin T. Miller (2016). Policy Misperceptions and Support for Gun Control Legislation. The Lancet 387(10015):223.", "doi": "10.1016/S0140-6736(16)00042-8", "edition": { "pages_in_audited_file": 1, "sha256": "cf8325e4f625073599232e6abbbfd26b98a0d4b961458facead1f12c0ac0a08e", "source_url": "https://doi.org/10.1016/S0140-6736(16)00042-8", "venue": "The Lancet", "year": "2016" }, "id": "P054", "later_materials": null, "title": "Policy Misperceptions and Support for Gun Control Legislation", "type": "Letter", "url": "https://pmaronow.github.io/llm-audit/works/P054/", "venue": "The Lancet", "year": "2016" } }