{
 "claims": [
  {
   "claim": "Field experiments overcome the main weaknesses of survey-based research on turnout: random assignment makes treatment and control groups comparable in expectation, the treatment is known, and turnout is measured from administrative records. When some subjects are not reached, the experiment identifies the average effect among Compliers, estimated as the intent-to-treat difference divided by the contact rate.",
   "claim_id": "P042-H1",
   "related_corrections": [],
   "tied_to": "Section 'Field Experimentation as a Departure from Survey-Based Approaches', pp. 29-30; Notes 2-3; CACE paragraph, p. 32"
  },
  {
   "claim": "Conventional direct mail raises turnout only slightly: 0.162 percentage points per mailing (95% CI $(0.078, 0.247)$; 110 treatments). Nonpartisan encouragements (0.194; CI $(0.106, 0.282)$; 79 treatments) differ significantly from advocacy mail ($-0.213$; CI $(-0.515, 0.088)$; 31 treatments), whose effect is indistinguishable from zero.",
   "claim_id": "P042-H2",
   "related_corrections": [
    "P042-E001",
    "P042-N003"
   ],
   "tied_to": "In-text fixed-effect meta-analysis of direct mail, p. 31 (no table); do-file lines 'nonsocial', 'nonsocial, nonadvocacy', 'nonsocial, advocacy'"
  },
  {
   "claim": "Mailings that invoke social norms work far better than conventional mail: 29 social-pressure treatments average 2.850 percentage points (95% CI $(2.686, 3.014)$), and eight gratitude treatments average 1.332 points (95% CI $(0.916, 1.747)$).",
   "claim_id": "P042-H3",
   "related_corrections": [
    "P042-E001",
    "P042-N005"
   ],
   "tied_to": "In-text meta-analysis, p. 32; p. 36; summary pp. 36-37; do-file lines 'social' and 'thank you'"
  },
  {
   "claim": "Door-to-door canvassing has a large effect among those reached: a weighted average CACE of 2.536 percentage points (95% CI $(1.817, 3.255)$) over 71 treatments, an estimate that if anything understates canvassing because many studies target low-propensity voters.",
   "claim_id": "P042-H4",
   "related_corrections": [
    "P042-N006",
    "P042-E001"
   ],
   "tied_to": "In-text meta-analysis, p. 32; p. 33; do-file line 'all canvassing'"
  },
  {
   "claim": "Phone calls: pre-recorded calls do almost nothing (0.156 percentage points, 95% CI $(-0.118, 0.430)$, 11 treatments, effect of assignment); commercial live calls help a little (CACE 0.980, CI $(0.504, 1.456)$, 25 treatments); volunteer live calls help more (CACE 1.936, CI $(1.298, 2.575)$, 37 treatments); two social-pressure pre-recorded calls do better (1.720, CI $(0.306, 3.135)$).",
   "claim_id": "P042-H5",
   "related_corrections": [
    "P042-E002",
    "P042-N010",
    "P042-N011",
    "P042-A002",
    "P042-E001"
   ],
   "tied_to": "In-text meta-analysis, pp. 32-33; Note 5, p. 40; do-file lines 'all robo', 'all professional', 'all volunteer'"
  },
  {
   "claim": "The overall pattern is that impersonal tactics (conventional mail, pre-recorded calls, mass e-mail) produce weak effects while personal ones (canvassing, volunteer calls) produce stronger ones, and that how authentically a live contact is delivered matters.",
   "claim_id": "P042-H6",
   "related_corrections": [
    "P042-N006",
    "P042-N010"
   ],
   "tied_to": "The pooled estimates of pp. 31-33 taken together; synthesis on p. 33; narrative on e-mail and text messages"
  },
  {
   "claim": "Experiments that combine tactics give no consistent support for either synergy or diminishing returns; interactions between treatments tend to be weak, with the possible exception of volunteer follow-up calls to people who said they would vote.",
   "claim_id": "P042-H7",
   "related_corrections": [
    "P042-N012",
    "P042-N011"
   ],
   "tied_to": "Section 'Synergy, Diminishing Returns, or Neither?', pp. 34-35 (narrative review; no pooled estimate)"
  },
  {
   "claim": "What a message argues (civic duty, closeness, group solidarity, positive or negative framing) matters little, whereas the norm it invokes (social pressure, gratitude) matters a great deal, to the point that forcefully asserted norms make even pre-recorded calls effective; implementation-intention prompts fall in between.",
   "claim_id": "P042-H8",
   "related_corrections": [
    "P042-E002",
    "P042-A002"
   ],
   "tied_to": "Abstract (second strand); section on message effects, pp. 35-37; the social-pressure and gratitude estimates of p. 32; Note 5"
  },
  {
   "claim": "Mobilisation spills over within households (about 60% of the canvassing effect reaches a housemate in Nickerson 2008, while the multilevel mail experiment finds no cross-household evidence) and persists into later elections, which is consistent with, but does not uniquely establish, habit formation.",
   "claim_id": "P042-H9",
   "related_corrections": [],
   "tied_to": "Section on spatial and temporal spillovers, pp. 37-38 (narrative review)"
  },
  {
   "claim": "Experimental opportunities extend beyond campaign tactics to institutional rules and voter attributes, including information interventions, naturally randomized quotas and term lengths, school lotteries and interventions that affect education or income.",
   "claim_id": "P042-H10",
   "related_corrections": [
    "P042-E005",
    "P042-N014"
   ],
   "tied_to": "Section 'Experimental Frontiers', pp. 38-39"
  }
 ],
 "date_published": "2026-10-25",
 "findings": [
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "exact_counterexample",
   "exhibits": [
    {
     "alt": "Crop of seven lines of text on page 32. A red box marks the words 95\\% CI = (2.686, 3.014), which follow the statement that the 29 social pressure treatments generated a weighted average treatment effect of 2.850 percentage points. Two lines below, the gratitude estimate reads 1.332 percentage points, 95\\% CI = (0.916, 1.747).",
     "caption": "Page 32 (PDF p. 8), first paragraph: the interval for social-pressure mail, marked. The other eight pooled intervals on pp. 31–33 are computed in the same way.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E001.png"
    },
    {
     "alt": "Crop of six lines of text on page 31. Red boxes mark the words: weighting each study by the precision with which it estimates the treatment effect. This fixed effects weighting scheme.",
     "caption": "Page 31 (PDF p. 7), second paragraph: the weighting scheme, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E001-b.png"
    },
    {
     "alt": "Crop of six lines of text on page 31. Red boxes mark the clause that says the two types of mailings have significantly different average effects. The next sentence gives the nonadvocacy estimate 0.194.",
     "caption": "Page 31 (PDF p. 7), third paragraph: the statement of significance that rests on the same calculation, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E001-c.png"
    },
    {
     "alt": "Crop of nine lines of text on page 36. A red box marks the words: The control condition received no mail. The surrounding text describes the Civic Duty, Hawthorne and Self treatments of the direct mail experiment of Gerber et al. (2008).",
     "caption": "Page 36 (PDF p. 12), first paragraph: one control condition, marked, for the four mailings of Gerber et al. (2008). Three of the four contrasts are among the 29 social-pressure treatments.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E001-d.png"
    },
    {
     "alt": "Chart titled: Shared control groups raise the pooled standard errors: social-pressure mail by 21\\% under the arm sizes in the dataset, and by up to 83\\% at the broader bound. The horizontal axis is the pooled standard error divided by the printed standard error, from 1.0 to 1.9, with a vertical line at 1 labelled as printed. Nine rows give, for the model, the narrower bound and the broader bound: conventional mail 1.01, 1.06, 1.09; nonadvocacy mail 1.00, 1.03, 1.06; advocacy mail 1.07, 1.28, 1.37; social-pressure mail 1.21, 1.51, 1.83, labelled plus 21\\%, plus 51\\% and plus 83\\%; gratitude mail 1.00, 1.34, 1.34; canvassing 1.00, 1.08, 1.10; recorded calls 1.00, 1.08, 1.08; commercial calls 1.00, 1.23, 1.27; volunteer calls 1.00, 1.00, 1.01. The model is drawn as light blue circles, the narrower bound as blue squares and the broader bound as dark blue diamonds.",
     "caption": "The pooled standard error of each of the nine estimates, divided by the standard error behind the printed interval, under the shared-control model with the arm sizes in the dataset (circles) and at the narrower and broader upper bounds (squares and diamonds). For gratitude mail and recorded calls the two bounds are equal. The values are conditional recalculations and bounds, not the covariances of the actual experiments.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E001-se-ratio.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [
    {
     "id": "P042-T1",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T1"
    },
    {
     "id": "P042-T2",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T2"
    },
    {
     "id": "P042-T5",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T5"
    }
   ],
   "headline": "The pooled 95% intervals on pp. 31–33 treat all pooled contrasts as independent, although several are arms of one experiment that share a control group; the variance of each pooled estimate should include the covariances between such contrasts, and under the standard shared-control model with the arm sizes in the replication dataset the standard error of the social-pressure estimate is 21% larger than the one behind the printed interval.",
   "id": "P042-E001",
   "location": "pp. 31–33 (PDF pp. 7–9): the weighting scheme described on p. 31, second paragraph, and every pooled 95% interval that follows, shown here for the social-pressure interval on p. 32 (PDF p. 8), first paragraph. The shared control group is described on p. 36 (PDF p. 12).",
   "pdf_page": 7,
   "printed_page": "31–33",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Moderate",
   "subject": "paper",
   "target_ids": [
    "P042-T1",
    "P042-T2",
    "P042-T5"
   ],
   "text": {
    "as_printed": "p.~31: ``Meta-analysis is a statistical method that pools the experiments in a given domain, weighting each study by the precision with which it estimates the treatment effect. This fixed effects weighting scheme means that smaller experiments tend to be accorded less weight than larger studies.''\n\np.~32: ``The 29 social pressure treatments generated a weighted average treatment effect of 2.850 percentage points, \\hl{95\\% CI} $\\hl{= (2.686, 3.014)}$.''\n\np.~31: ``we find the two types of mailings have \\hl{significantly different} average effects.''",
    "corrected": "Let a pool consist of contrasts $d_1,\\dots,d_K$ with standard errors $s_1,\\dots,s_K$ and covariance matrix $\\Sigma$, so that $\\Sigma_{jj}=s_j^2$, and let the weights $a_j=s_j^{-2}/\\sum_i s_i^{-2}$ be treated as fixed. The pooled estimate $\\sum_j a_jd_j$ has variance\n\\[\na^{\\mathsf T}\\Sigma a=\\frac{1}{\\sum_j s_j^{-2}}+\\hl{2\\sum_{j<k}a_ja_k\\Sigma_{jk}}.\n\\]\nThe printed intervals use the first term alone. It is the full variance only if the second term vanishes, for instance if no two contrasts of the pool are correlated. Approximate 95\\% coverage also depends on the validity of the normal approximation.\n\nSuppose that $d_j=\\bar Y_j-\\bar Y_0$ and $d_k=\\bar Y_k-\\bar Y_0$ compare treatment groups of $n_j$ and $n_k$ subjects with a common control group of $n_0$ subjects, and that subjects' outcomes are independent with the same variance in all three groups. Then\n\\[\n\\Sigma_{jk}=\\rho_{jk}\\,s_js_k,\\qquad \\rho_{jk}=\\sqrt{\\frac{n_jn_k}{(n_j+n_0)(n_k+n_0)}}.\n\\]\nI refer to these assumptions as the shared-control model. The correlation $\\rho_{jk}$ is the same if each contrast is a fixed positive multiple of the difference in means, as for an effect per mailing.\nFor the difference between the nonadvocacy and advocacy pooled estimates the variance is $b^{\\mathsf T}\\Sigma b$, with $b_j=a_j$ in the first pool and $b_j=-a_j$ in the second.\n\nThe sentence on p.~32 should read: ``The 29 social pressure treatments generated a weighted average treatment effect of 2.850 percentage points. \\hl{If the 29 contrasts are treated as independent,} the \\hl{nominal} 95\\% CI is (2.686, 3.014)\\hl{; using the shared-control model's correlations for the arm sizes in the replication dataset with the stored standard errors, it is (2.651, 3.049).}'' The other eight intervals need the same qualification, with the values in the table above. On p.~31, the two types of mailings have significantly different average effects \\hl{under the independence-based normal approximation ($z=2.54$)}. An interval for the actual experiments requires the covariance of the contrasts of each experiment, which the replication dataset does not contain.",
    "evidence": "The evidence is a derivation, two exact counterexamples to the program's variance formula, and a recalculation from the replication files.\n\n\\emph{Reproduction.} The pooling commands of the replication program receive an effect and its standard error for each treatment and no covariance. With the variance $1/\\sum_j s_j^{-2}$, all nine estimates and intervals on pp.~31--33 reproduce to the printed precision. For the social-pressure pool the standard error is 0.0838.\n\n\\emph{Derivation.} Let $\\delta_j=d_j-\\mathbb{E}[d_j]$ and $\\Sigma_{jk}=\\mathbb{E}[\\delta_j\\delta_k]$, so that $\\Sigma_{jj}=s_j^2$. For fixed weights, I have\n\\[\n\\operatorname{Var}\\Bigl(\\sum_j a_jd_j\\Bigr)=\\mathbb{E}\\Bigl[\\Bigl(\\sum_j a_j\\delta_j\\Bigr)^2\\Bigr]=\\sum_j a_j^2s_j^2+2\\sum_{j<k}a_ja_k\\Sigma_{jk}.\n\\]\nWith $a_j=s_j^{-2}/\\sum_i s_i^{-2}$, the first sum equals $1/\\sum_j s_j^{-2}$. Let $\\bar Y_j$, $\\bar Y_k$ and $\\bar Y_0$ be the mean outcomes of two treatment groups and of their common control group. If the three means are uncorrelated, then\n\\[\n\\Sigma_{jk}=\\operatorname{Cov}(\\bar Y_j-\\bar Y_0,\\ \\bar Y_k-\\bar Y_0)=\\operatorname{Var}(\\bar Y_0).\n\\]\n\n\\emph{Counterexamples.} Consider six voters whose turnout is $(1,1,1,0,0,0)$ whatever they are sent, and assign two of them at random to each of a control group and two mail groups. Both true treatment effects are zero. Enumerating the 90 equally likely assignments shows that each contrast has variance $3/10$ and that the two contrasts have covariance $3/20$. Their average then has variance $9/40$, and the program's formula gives $3/20$. In the second example, each voter in three independent groups of 100 votes independently with probability $1/2$. Each of the two contrasts has variance $1/200$, and their covariance is $1/400$. Their average has variance $3/800$, where the program's formula gives $1/400$. By exact enumeration, the nominal 95\\% normal interval built from $1/400$ covers the true value zero with probability 0.8887. The examples show that the program's formula can fail for contrasts with a common control group. They do not measure the coverage of the article's intervals.\n\n\\emph{Recalculation.} The dataset records the arm sizes for some groups of contrasts with a common control group. The groups with at least two contrasts in the social-pressure pool hold 50.3\\% of its weight, and the four Kentucky treatments hold 32.4\\%. I use the shared-control model's correlations with the stored standard errors for these groups and take all other contrasts to be independent. For the social-pressure estimate, I obtain a standard error of 0.1014, which is 21\\% above 0.0838. With the Kentucky group alone, the standard error is 0.0997, which is 19\\% above. The three mailings of Gerber et al. (2008) hold 23.4\\% of the weight, each with $a\\approx0.0780$. If their three covariances equal $c$, then they add $6a^2c\\approx0.0365\\,c$ to the variance. The dataset does not determine $c$, since its sample-size fields for these rows cannot be used (\\fid{P042-E003}), and the model treats the three contrasts as independent.\n\nI also computed two upper bounds. At the narrower bound, each group of contrasts that appear to come from one experiment, but lack sizes, has the largest covariance that their standard errors allow, $s_js_k$. The broader bound also allows overlaps that the dataset leaves unresolved, among them a control group common to all seven Kentucky rows. At both bounds, different groups are independent. Neither bound is the covariance of the actual experiments.\n\\begin{audittable}{lrrrr}{The 95\\% intervals of the nine pooled estimates, in percentage points: as printed, under the shared-control model with the arm sizes in the dataset, and at the two upper bounds. The number of treatments is in parentheses.}\nPool & As printed & Shared-control model & Narrower bound & Broader bound \\\\\nConventional mail (110) & $(0.078,\\ 0.247)$ & $(0.077,\\ 0.247)$ & $(0.073,\\ 0.251)$ & $(0.071,\\ 0.254)$ \\\\\nNonadvocacy mail (79) & $(0.106,\\ 0.282)$ & $(0.106,\\ 0.282)$ & $(0.103,\\ 0.285)$ & $(0.101,\\ 0.287)$ \\\\\nAdvocacy mail (31) & $(-0.515,\\ 0.088)$ & $(-0.536,\\ 0.109)$ & $(-0.599,\\ 0.173)$ & $(-0.627,\\ 0.201)$ \\\\\nSocial-pressure mail (29) & $(2.686,\\ 3.014)$ & $(2.651,\\ 3.049)$ & $(2.602,\\ 3.098)$ & $(2.549,\\ 3.150)$ \\\\\nGratitude mail (8) & $(0.916,\\ 1.747)$ & $(0.916,\\ 1.747)$ & $(0.777,\\ 1.887)$ & $(0.777,\\ 1.887)$ \\\\\nCanvassing (71) & $(1.817,\\ 3.255)$ & $(1.817,\\ 3.255)$ & $(1.763,\\ 3.309)$ & $(1.743,\\ 3.329)$ \\\\\nRecorded calls (11) & $(-0.118,\\ 0.430)$ & $(-0.118,\\ 0.430)$ & $(-0.139,\\ 0.451)$ & $(-0.139,\\ 0.451)$ \\\\\nCommercial calls (25) & $(0.504,\\ 1.456)$ & $(0.504,\\ 1.456)$ & $(0.394,\\ 1.567)$ & $(0.378,\\ 1.583)$ \\\\\nVolunteer calls (37) & $(1.298,\\ 2.575)$ & $(1.298,\\ 2.575)$ & $(1.297,\\ 2.576)$ & $(1.290,\\ 2.583)$ \\\\\n\\end{audittable}\nThe difference between the nonadvocacy and advocacy estimates is 0.407. Its variance is $b^{\\mathsf T}\\Sigma b$, where $b_j=a_j$ in the first pool and $b_j=-a_j$ in the second. No group of contrasts in the model or the bounds joins the two pools.\n\\begin{audittable}{lrrrr}{Nonadvocacy minus advocacy mail (difference 0.407): standard error, $z$, two-sided $p$ and 95\\% interval.}\nCovariance & Standard error & $z$ & $p$ & 95\\% interval \\\\\nIndependence, as in the article & 0.160 & 2.54 & 0.011 & $(0.093,\\ 0.722)$ \\\\\nShared-control model & 0.170 & 2.39 & 0.017 & $(0.073,\\ 0.741)$ \\\\\nNarrower bound & 0.202 & 2.01 & 0.044 & $(0.011,\\ 0.804)$ \\\\\nBroader bound & 0.217 & 1.88 & 0.060 & $(-0.017,\\ 0.832)$ \\\\\n\\end{audittable}\nBoth tables use the estimates as stored. I did not repeat the calculation with the estimates corrected in \\fid{P042-N003}, \\fid{P042-N005} and \\fid{P042-N006}.",
    "problem": "The replication program computes each pooled estimate as a weighted average $\\sum_j a_jd_j$ of the contrasts $d_j$, with weights $a_j$ proportional to $1/s_j^2$, where $s_j$ is the standard error of $d_j$. It takes the variance of this average to be $1/\\sum_j s_j^{-2}$. That is the variance only if the weighted covariance terms sum to zero, as they do for uncorrelated contrasts. Several pooled contrasts are arms of one experiment that share a control group. The Hawthorne, Self and Neighbors mailings of Gerber et al. (2008), which p.~36 describes with one control condition, are three of the 29 social-pressure treatments. Four Kentucky treatments of Mann (2010) share a control group of 19,561 people and hold 32\\% of the weight of that pool. Two contrasts with a common control group both contain the sampling error of the control mean. For unscaled differences in means under independent sampling, their covariance is the variance of that mean, which is positive. The printed intervals omit these terms. How much the intervals' widths should change for the actual experiments cannot be recovered from the dataset, which gives marginal standard errors but no covariances; the size of the omission can be computed under stated models, as below.",
    "what_survives": "The nine pooled estimates, their counts and their ordering do not depend on the variance and are unchanged. Under the model and at both bounds, each of the nine intervals keeps its position relative to zero. The social-pressure and gratitude normal intervals stay above zero for every covariance matrix that is compatible with the stored standard errors, with minimum lower endpoints of 2.087 and 0.257 to three decimal places. The difference between nonadvocacy and advocacy mail stays significant at the 5\\% level under the model and at the narrower bound. At the broader bound it does not ($z=1.88$), which shows that the printed statement could fail and not that it does. What changes is the stated precision: the printed intervals use the independence-based variance and a normal approximation."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-E001",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "exact_counterexample",
   "exhibits": [
    {
     "alt": "Crop of Note 5 on page 40. Red boxes mark '1.720' and '(0.306, 3.135)' in the last sentence of the note: 'These studies found stronger effects among Compliers: a weighted average effect of 1.720 percentage points, with a 95 percent CI = (0.306, 3.135).' The sentence before it refers to the two pre-recorded call experiments that confronted recipients with their vote histories.",
     "caption": "Note 5, p. 40 (PDF p. 16). The marked estimate and interval match a random-effects calculation.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E002.png"
    },
    {
     "alt": "Crop of a paragraph on page 31. A red box marks the words 'fixed effects' in the sentence 'This fixed effects weighting scheme means that smaller experiments tend to be accorded less weight than larger studies.'",
     "caption": "p. 31 (PDF p. 7), second paragraph. The marked words declare the pooling method of the article.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E002-b.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [
    {
     "id": "P042-T3",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T3"
    },
    {
     "id": "P042-T4",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T4"
    }
   ],
   "headline": "The figure in Note 5 for the two experiments on pre-recorded calls with a social-pressure message is not the fixed effects average that the article declares on p. 31; the fixed effects average of the two estimates in the replication dataset is 1.866, with 95% interval (0.963, 2.768), and the printed 1.720, with interval (0.306, 3.135), matches a random-effects estimate.",
   "id": "P042-E002",
   "location": "Note 5, p. 40 (PDF p. 16), last sentence; the method is declared on p. 31 (PDF p. 7)",
   "pdf_page": 16,
   "printed_page": "40",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [
    "P042-T3",
    "P042-T4"
   ],
   "text": {
    "as_printed": "``These studies found stronger effects among Compliers: a weighted average effect of \\hl{1.720} percentage points, with a 95\\% CI = \\hl{(0.306, 3.135)}.''\n\nThe method paragraph on p.~31 says: ``Meta-analysis is a statistical method that pools the experiments in a given domain, weighting each study by the precision with which it estimates the treatment effect. This fixed effects weighting scheme means that smaller experiments tend to be accorded less weight than larger studies.''",
    "corrected": "``These studies found stronger effects among Compliers: a weighted average effect of \\hl{1.866} percentage points, with a 95\\% CI = \\hl{(0.963, 2.768)}.''\n\nThis is the fixed effects average of the two estimates in the replication dataset, with weights equal to the inverse squared standard errors and a normal interval, as declared on p.~31; the replication program should include the command that produces it. If the printed figure is the intended one, the note should instead say that it is a DerSimonian--Laird random-effects estimate, with $\\widehat\\tau^2=0.5819$, and that it is an exception to the fixed effects weighting of p.~31, and the replication program should include that command. Both alternatives keep a positive interval.",
    "evidence": "The evidence is an exact calculation from the two estimates in the replication dataset. Let $x_1=2.32988$ and $x_2=0.86596$ denote the two estimates and $v_1=0.55712^2$ and $v_2=0.81782^2$ their variances, and set $w_i=1/v_i$. The fixed effects average and its variance are\n\\[\n\\widehat\\mu_F=\\frac{w_1x_1+w_2x_2}{w_1+w_2}=\\frac{114191974724271}{61200765425000}=1.865859,\\qquad V_F=\\frac{1}{w_1+w_2}=0.212000 .\n\\]\nSince $\\widehat\\mu_F>1.7205$, it cannot round to the printed 1.720. For the random-effects estimate, the heterogeneity statistic for two studies is $Q=(x_1-x_2)^2/(v_1+v_2)=2.188557$, and the DerSimonian--Laird estimate of the variance between studies is\n\\[\n\\widehat\\tau^2=\\frac{Q-1}{2/(v_1+v_2)}=\\frac{2909623799}{5000000000}=0.5819247598 .\n\\]\nWith $u_i=1/(v_i+\\widehat\\tau^2)$, I obtain\n\\[\n\\widehat\\mu_R=\\frac{u_1x_1+u_2x_2}{u_1+u_2}=\\frac{12592252377}{7319600000}=1.720347,\\qquad V_R=\\frac{1}{u_1+u_2}=0.520777 .\n\\]\n\\begin{audittable}{lrrr}{The figure of Note~5 and the two calculations from the estimates in the replication dataset, in percentage points.}\nCalculation & Estimate & SE & 95\\% interval \\\\\nPrinted in Note~5 & 1.720 & & (0.306, 3.135) \\\\\nDerSimonian--Laird random effects & 1.720347 & 0.721649 & (0.305941, 3.134753) \\\\\nFixed effects, as declared on p.~31 & 1.865859 & 0.460435 & (0.963423, 2.768294) \\\\\n\\end{audittable}\nThe rounding of both intervals is the same for every multiplier between 1.9599 and 1.9600.",
    "problem": "The article declares one pooling method, a fixed effects average in which each study is weighted by its precision, and its nine other pooled figures on pp.~31--33 follow it. The figure in Note~5 does not. The two estimates for these experiments in the replication dataset are 2.32988 with standard error 0.55712 (``Robo SP1'') and 0.86596 with standard error 0.81782 (``Robo SP2''), and their fixed effects average is 1.866, with interval (0.963, 2.768). The printed estimate and both limits agree to three decimals with the DerSimonian--Laird random-effects estimate from the same two estimates. The note does not say that the method differs. The replication program has no command that pools the two estimates: its ten pooling commands all specify fixed effects, and its only mention of the two entries excludes them from the ordinary pre-recorded calls. Read alone, the words ``a weighted average'' admit the random-effects calculation. Read with the method declared on p.~31, the figure is wrong. Note that the numerical agreement does not identify the command that produced the printed figure, and it does not show whether the exception was intended.",
    "what_survives": "Both calculations give a positive estimate with a 95\\% interval above zero, so the statement of the note that these calls had stronger effects stands, as does the statement on p.~36 that such calls increase turnout. The nine other pooled figures of the article agree with the fixed effects calculation and do not change. Whether a third estimate for pre-recorded calls belongs with these two is a separate matter (\\fid{P042-N010}), and I do not change the count."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-E002",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "recalculation_from_source",
   "exhibits": [
    {
     "alt": "Listing of cells of the replication dataset with their row numbers. First part: rows 7 and 9, author Gillespie, year 2005. Red boxes mark the Notes cells, St. Louis Experiment and Detroit Experiment 2001, and the two Election/Intervention date cells, both reading 2002 Newark NJ mayoral (May) and runoff (June). Second part: rows 256 to 259, Gerber, Green, Larimer, 2008, with notes civic duty, hawthorne, self and neighbors. One red box encloses their N Treatment cells, all 353341, and their N Control cells, 3238, 6307, 6325 and 6325; the N cells are 356579, 359648, 359666 and 359666. Below, unmarked, rows 86 to 88, Gerber, Green, Larimer, 2010: N Treatment 3238, 6307 and 6325, N Control 353341 in each, N 356579, 359648 and 359666.",
     "caption": "The replication dataset, not a page of the article: rows 7 and 9, and rows 256–259 above rows 86–88. In rows 7 and 9 the marked fields name different places. In rows 256–259 the marked sample sizes are those of rows 86–88 with treatment and control exchanged.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E003.png"
    },
    {
     "alt": "Crop of the first paragraph of page 36. It says that Gerber et al. (2008) conducted a large-scale direct mail experiment that varied the forcefulness with which the norm of voting was asserted. A red box marks the words: The control condition received no mail. The text goes on to describe the Civic Duty, Hawthorne and Self treatments.",
     "caption": "Page 36 (PDF p. 12), first paragraph: the article describes the experiment of rows 256–259 as having one control condition, marked, and four mailings.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E003-b.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "In the replication dataset, two rows for Gillespie (2005) give conflicting experiment and election labels, and the sample-size fields of the four rows for Gerber, Green and Larimer (2008) repeat those of the 2010 study with treatment and control exchanged; these fields should be checked against the sources and corrected.",
   "id": "P042-E003",
   "location": "Replication dataset (Yale Dataverse, release v2.0; the CSV file of the release), rows 7, 9 and 256–259, where the header is row 1. The dataset is not printed in the article; the experiment of rows 256–259 is described on p. 36 (PDF p. 12).",
   "pdf_page": null,
   "printed_page": null,
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "\\begin{audittable}{rll}{Rows 7 and 9 (Gillespie, 2005).}\nRow & Notes & Election/Intervention date \\\\\n7 & \\hl{St. Louis Experiment} & \\hl{2002 Newark NJ mayoral (May) \\& runoff (June)} \\\\\n9 & \\hl{Detroit Experiment 2001} & \\hl{2002 Newark NJ mayoral (May) \\& runoff (June)} \\\\\n\\end{audittable}\n\\begin{audittable}{rlrrr}{Rows 256--259 (Gerber, Green and Larimer, 2008; election field ``2006 primary Michigan'').}\nRow & Notes & N Treatment & N Control & N \\\\\n256 & civic duty & \\hl{353341} & \\hl{3238} & 356579 \\\\\n257 & hawthorne & \\hl{353341} & \\hl{6307} & 359648 \\\\\n258 & self & \\hl{353341} & \\hl{6325} & 359666 \\\\\n259 & neighbors & \\hl{353341} & \\hl{6325} & 359666 \\\\\n\\end{audittable}",
    "corrected": "Rows 7 and 9: the fields Notes and Election/Intervention date should name the same experiment, as established from Gillespie (2005). Until then, the election field of these two rows should be left blank or marked as unchecked.\n\nRows 256--259: the sizes should be taken from Table~2 of Gerber, Green and Larimer (2008). The sizes proposed for these rows are 38,218, 38,204, 38,218 and 38,201 individuals in the Civic Duty, Hawthorne, Self and Neighbors groups and 191,243 in the common control group, 344,084 in all. If the table confirms them, the rows should read as follows.\n\\begin{audittable}{rlrrr}{Rows 256--259, conditional on Table 2 of Gerber, Green and Larimer (2008).}\nRow & Notes & N Treatment & N Control & N \\\\\n256 & civic duty & \\hl{38218} & \\hl{191243} & \\hl{229461} \\\\\n257 & hawthorne & \\hl{38204} & \\hl{191243} & \\hl{229447} \\\\\n258 & self & \\hl{38218} & \\hl{191243} & \\hl{229461} \\\\\n259 & neighbors & \\hl{38201} & \\hl{191243} & \\hl{229444} \\\\\n\\end{audittable}",
    "evidence": "The evidence is a check of the replication dataset and a recalculation. I transcribed the fields above from the file. The ten pooling commands of the replication program use only the effect and standard-error columns, the classification flags and two title exclusions. In exact arithmetic, the ten pooled estimates and their variances are the same before and after the election and sample-size fields are changed. Neither Table~2 of Gerber, Green and Larimer (2008) nor Gillespie's dissertation was available for the check, and the correct sizes and dates are not established here. The name of the file carries the date 19 March 2014, and I did not establish whether the file used for the article had the same entries.",
    "problem": "Each row of the dataset should describe the experiment behind its estimate. Row 7 names a St.~Louis experiment and row 9 a Detroit experiment of 2001, and both carry the Newark elections of 2002 in the election field. At least one of the two fields is wrong in each row, and the dataset does not show which. In rows 256--259 the treatment group has 353,341 members in every row and the control group has 3,238, 6,307, 6,325 and 6,325 members, although the article describes one control group for the four mailings. These are the sizes stored for the 2010 study in rows 86--88 with treatment and control exchanged, and row 259 repeats row 258. The pattern is consistent with a copying slip, but it does not show what the correct sizes are.",
    "what_survives": "The pooled estimates, intervals and counts on pp.~31--33 do not use these fields and are unchanged. The correction matters to anyone who uses the election or sample-size fields. The replication program also summarizes the column \\texttt{N}, and that summary may change."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-E003",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "recalculation_from_source",
   "exhibits": [
    {
     "alt": "Crop of the head of the reference list on page 40. Red boxes mark the words 'but are not otherwise referenced in the text.' in the legend 'References marked with an asterisk are included in the meta-analysis, but are not otherwise referenced in the text.'",
     "caption": "References, p. 40 (PDF p. 16). The marked clause of the legend says that a starred work is not cited in the text.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E004.png"
    },
    {
     "alt": "Crop of the reference list on page 40. A red box marks the asterisk, authors and year of the entry 'Almond, Gabriel A. and Verba, Sidney (1963) The Civic Culture (Princeton: Princeton University Press)', which begins with an asterisk. The entries above and below it, Alford and Lee (1968) and Alvarez, Hopkins and Sinclair (2010), have none.",
     "caption": "References, p. 40 (PDF p. 16). The marked entry carries an asterisk.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E004-b.png"
    },
    {
     "alt": "Crop of the top of page 28, with the page number 28 and the running head. A red box marks the citation '(Almond \\& Verba, 1963;' in the line 'socialization and culture (Almond \\& Verba, 1963; Litt, 1963), patronage (Blaydes,'.",
     "caption": "p. 28 (PDF p. 4), first line. The starred work of Almond and Verba is cited in the text.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E004-c.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The asterisks in the reference list do not follow the rule stated above the list: five starred works are cited in the text, one of them matches no entry in the replication dataset, and six uncited works that supply estimates there carry no asterisk.",
   "id": "P042-E004",
   "location": "References, p. 40 (PDF p. 16): the legend above the list, and the starred entries on pp. 40–48 (PDF pp. 16–24)",
   "pdf_page": 16,
   "printed_page": "40",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``References marked with an * are included in the meta-analysis, \\hl{but are not otherwise referenced in the text}.''\n\n``\\hl{*}Almond, Gabriel A. \\& Verba, Sidney (1963) \\emph{The Civic Culture} (Princeton: Princeton University Press).''",
    "corrected": "``\\del{*}Almond, Gabriel A. \\& Verba, Sidney (1963) \\emph{The Civic Culture} (Princeton: Princeton University Press).''\n\nThe asterisk on Almond and Verba (1963) is removed. The remaining asterisks and the legend are to be regenerated from the list of inputs used for the article; I have not established a complete replacement set of marks.",
    "evidence": "The evidence is a check of the reference list on pp.~40--48 and of the citations in the text, together with a comparison of the list with the 295 entries of the replication dataset and the selections of the replication program. The replication dataset is the release of March 2014, after publication, and I cannot confirm that it is the list of inputs used for the article. For this reason I give no complete set of corrected marks. The five citations in the text contradict the legend on the printed page and do not depend on the dataset.",
    "problem": "By the legend, a starred work is an input of the meta-analysis and is cited nowhere in the text. The marks depart from this rule in three ways.\n\\begin{itemize}\n\\item Five starred works are cited in the text: Almond and Verba (1963) on p.~28; Arceneaux and Nickerson (2009) on p.~38; Arceneaux and Nickerson (2010) on p.~35; Gerber and Green (2000a) on pp.~28, 31, 32 and 35; and Sinclair, McConnell and Green (2012) on pp.~36 and 37. Almond and Verba (1963) also matches no entry of the replication dataset.\n\\item Six works that supply estimates to the pooled figures and are not cited in the text carry no asterisk: Arceneaux (2005); Arceneaux, Gerber and Green (2010); Barton, Castillo and Petrie (2012); Mann (2005); Mann and Mayhew (2012); and Nickerson (2007b).\n\\item Shaw, Green, Gimpel and Gerber (2012) supplies two of the estimates for pre-recorded calls and has no entry in the reference list.\n\\end{itemize}\nIn addition, eight starred works supply no usable estimate to the pooled figures in the replication dataset. Seven match no entry: Aker, Collier and Vicente (2010); Bond et al.\\ (2012); Gerber and Green (2000b); Gerber, Karlan and Bergan (2009); Mann and Klofstad (2011); Panagopoulos (2009b); and Panagopoulos and Green (2011). The eighth, Cann et al.\\ (2012), matches two entries dated 2010, without standard errors.",
    "what_survives": "No formula, estimate or argument of the article depends on the asterisks. The review and the pooled figures are unaffected; the defect is in the bookkeeping of the reference list."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-E004",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of text on page 29. A red box marks the words Redefining Citizenship in the italic book title Mobilizing Inclusion: Redefining Citizenship through Get-Out-the-Vote Campaigns, cited as García Bedolla and Michelson, 2012.",
     "caption": "Page 29 (PDF p. 5), first paragraph: the subtitle given in the text, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005.png"
    },
    {
     "alt": "Crop of the reference entry García Bedolla, Lisa and Michelson, Melissa R. (2012). A red box marks the words Transforming the Electorate in the title Mobilizing Inclusion: Transforming the Electorate through Get-Out-the-Vote Campaigns.",
     "caption": "Reference list, p. 43 (PDF p. 19): the subtitle of the same book in its entry, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-b.png"
    },
    {
     "alt": "Crop of two consecutive reference entries by Gerber, Alan S. and Green, Donald P., both titled Do phone calls increase voter turnout, A field experiment, Public Opinion Quarterly, 65, pages 75 to 85. Red boxes mark the year (2001) in the first entry and (2001b) in the second. The second title lacks the question mark.",
     "caption": "Reference list, p. 43 (PDF p. 19): one article entered twice, under the marked years.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-c.png"
    },
    {
     "alt": "Crop of the reference entry Kelley, Stanley, Jr., Ayres, Richard and Bowen, William C. (1967) Registration and voting: putting first things first. American Political Science Review, 10, pages 233 to 246. A red box marks the volume number 10.",
     "caption": "Reference list, p. 45 (PDF p. 21): the volume number of the entry for Kelley, Ayres and Bowen (1967), marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-d.png"
    },
    {
     "alt": "Crop of the reference entry Chong, Alberto, De La O, Ana L., Karlan, Dean and Wantchekron, Leonard (2011). A red box marks the surname Wantchekron.",
     "caption": "Reference list, p. 42 (PDF p. 18): the misspelled surname, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-e.png"
    },
    {
     "alt": "Crop of two reference entries. The first is Huckfeldt and Sprague (1992), American Political Science Review, 86. In the second, Ichino, Nahomi and Schündeln, Matthias (2012), a red box marks the word orregularities in the title Deterring or displacing electoral orregularities?",
     "caption": "Reference list, p. 45 (PDF p. 21): the misspelled word, marked. The entry above gives volume 86 of the American Political Science Review for 1992.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-f.png"
    },
    {
     "alt": "Crop of five lines of text on page 38. A red box marks the citation Cutts et al. (2010) in the sentence beginning Although studies like Cutts et al. (2010) are designed to test whether results from American experiments apply to the United Kingdom.",
     "caption": "Page 38 (PDF p. 14): the citation that has no entry in the reference list, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-g.png"
    },
    {
     "alt": "Crop of the reference entry Michelson, Melissa R., Bedolla, Lisa García and Green, Donald P. (2007) New experiments in minority voter mobilization. A red box marks the words Bedolla, Lisa García.",
     "caption": "Reference list, p. 46 (PDF p. 22): the inverted surname in the entry for Michelson et al. (2007), marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-h.png"
    },
    {
     "alt": "Crop of the reference entry Michelson, Melissa R., Bedolla, Lisa García and McConnell, Margaret A. (2009) Heeding the call. A red box marks the words Bedolla, Lisa García.",
     "caption": "Reference list, p. 47 (PDF p. 23): the inverted surname in the entry for Michelson et al. (2009), marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-E005-i.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "One citation and the reference list contain seven slips: two subtitles for one book, a duplicated entry, a wrong volume number, two misspellings, a citation with no matching entry, and a surname inverted in two entries.",
   "id": "P042-E005",
   "location": "p. 29 (PDF p. 5), first paragraph; p. 38 (PDF p. 14); reference list, pp. 42, 43, 45, 46 and 47 (PDF pp. 18, 19, 21, 22 and 23).",
   "pdf_page": 5,
   "printed_page": "29",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "\\begin{enumerate}\n\\item p.~29: ``The recent book \\emph{Mobilizing Inclusion: \\hl{Redefining Citizenship} through Get-Out-the-Vote Campaigns} (García Bedolla \\& Michelson, 2012) alone reports more than 100 experiments.'' Reference list, p.~43: ``García Bedolla, Lisa \\& Michelson, Melissa R. (2012) \\emph{Mobilizing Inclusion: \\hl{Transforming the Electorate} through Get-Out-the-Vote Campaigns} (New Haven: Yale University Press).''\n\\item p.~43: ``*Gerber, Alan S. \\& Green, Donald P. \\hl{(2001)} Do phone calls increase voter turnout? A field experiment. \\emph{Public Opinion Quarterly}, 65, pp.~75--85.'' and ``*Gerber, Alan S. \\& Green, Donald P. \\hl{(2001b)} Do phone calls increase voter turnout A field experiment. \\emph{Public Opinion Quarterly}, 65, pp.~75--85.''\n\\item p.~45: ``Kelley, Stanley, Jr., Ayres, Richard \\& Bowen, William C. (1967) Registration and voting: putting first things first. \\emph{American Political Science Review}, \\hl{10}, pp.~233--246.''\n\\item p.~42: ``Chong, Alberto, De La O, Ana L., Karlan, Dean \\& \\hl{Wantchekron}, Leonard (2011) Looking beyond the incumbent: the effects of exposing corruption on electoral outcomes. NBER Working Paper 17679.''\n\\item p.~45: ``Ichino, Nahomi \\& Schündeln, Matthias (2012) Deterring or displacing electoral \\hl{orregularities}? Spillover effects of observers in a randomized field experiment in Ghana.''\n\\item p.~38: ``Although studies like \\hl{Cutts et al. (2010)} are designed to test whether results from American experiments apply to the United Kingdom, the literature has yet to produce a study that fields the same treatments in different countries.''\n\\item p.~46: ``Michelson, Melissa R., \\hl{Bedolla, Lisa García} \\& Green, Donald P. (2007)''; p.~47: ``Michelson, Melissa R., \\hl{Bedolla, Lisa García} \\& McConnell, Margaret A. (2009)''.\n\\end{enumerate}",
    "corrected": "\\begin{enumerate}\n\\item One subtitle in both places, as on the title page of the book; the reference list has ``Transforming the Electorate through Get-Out-the-Vote Campaigns''.\n\\item One entry, ``*Gerber, Alan S. \\& Green, Donald P. (2001) Do phone calls increase voter turnout? A field experiment. \\emph{Public Opinion Quarterly}, 65, pp.~75--85.'', with the suffixes of the citations reconciled.\n\\item ``\\emph{American Political Science Review}, \\hl{61}'', with the page range and the authors' initials checked against the article.\n\\item ``\\hl{Wantchekon}, Leonard''.\n\\item ``electoral \\hl{irregularities}?''\n\\item The citation on p.~38 should name the source meant, and the list should contain it; no replacement is established here.\n\\item ``Michelson, Melissa R., \\hl{García Bedolla, Lisa} \\& \\ldots'' in both entries.\n\\end{enumerate}",
    "evidence": "The evidence is a check of the pages: I read each item on the page and compared it with the rest of the reference list. The book, the article of Kelley et al. (1967) and the source meant by the Cutts citation were not available. The correct subtitle, the page range and the authors' initials of the Kelley entry, and the intended Cutts reference are therefore not settled here. The entry for Bhavnani (2009) on p.~41 has a single page, 23, and I could not check whether a page range is missing.",
    "problem": "The seven items are these. (1) The text and the reference list give different subtitles for one book. (2) The entries for 2001 and 2001b are the same article. (3) Volume 10 cannot be right for 1967, since the list has volume 50 of the same journal for 1956, 62 for 1968, 72 for 1978 and 80 for 1986, which implies volume 61 for 1967. (4) The surname is Wantchekon on pp.~38 and 48. (5) The word is ``irregularities''. (6) No entry matches ``Cutts et al. (2010)''; the only entry led by Cutts is dated 2009, and the article does not show which source is meant. (7) The list itself has the surname García Bedolla on p.~43.",
    "what_survives": "The slips affect reference lookup and attribution. No estimate or argument depends on them, and the duplicated entry does not show that a study entered the meta-analysis twice."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-E005",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "clarity_note",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of a paragraph on page 36. A red box marks the citation '(Gerber et al., 2010)' at the end of the sentence 'When social norms are asserted forcefully, the effects tend to be quite large, and even pre-recorded phone calls conveying social pressure messages significantly increase turnout (Gerber et al., 2010).'",
     "caption": "p. 36 (PDF p. 12), end of the first paragraph. The marked citation fits two entries of the reference list.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-A002.png"
    },
    {
     "alt": "Crop of the reference list on page 44. Red boxes mark the year '(2010)' in two entries: the starred entry of Gerber, Green and Larimer, 'An experiment testing the relative effectiveness of encouraging voter participation by inducing feelings of pride or shame', and the unstarred entry of Gerber, Green, Kaplan and Kern, 'Baseline, placebo, and treatment: efficient estimation for three-group experiments'.",
     "caption": "Reference list, p. 44 (PDF p. 20). The two marked entries of 2010 both abbreviate to “Gerber et al., 2010”; the second is the study of pre-recorded calls.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-A002-b.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The citation “(Gerber et al., 2010)” for pre-recorded calls with a social-pressure message refers to Gerber, Green, Kaplan and Kern (2010); the abbreviation also fits a second entry of the reference list.",
   "id": "P042-A002",
   "location": "p. 36 (PDF p. 12), first paragraph, last sentence; reference list, p. 44 (PDF p. 20)",
   "pdf_page": 12,
   "printed_page": "36",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": null,
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``When social norms are asserted forcefully, the effects tend to be quite large, and even pre-recorded phone calls conveying social pressure messages significantly increase turnout \\hl{(Gerber et al., 2010)}.''",
    "corrected": "``\\ldots\\ and even pre-recorded phone calls conveying social pressure messages significantly increase turnout \\hl{(Gerber, Green, Kaplan \\& Kern, 2010)}.''",
    "intended_meaning": "The reference list has two entries that abbreviate to ``Gerber et al., 2010'': Gerber, Green and Larimer (2010), on feelings of pride or shame, and Gerber, Green, Kaplan and Kern (2010), ``Baseline, placebo, and treatment: efficient estimation for three-group experiments''. The sentence concerns pre-recorded calls, and the second entry is the study of such calls. The replication dataset codes its estimate, 1.85 with standard error 0.89, as a pre-recorded call with a social-pressure message, and the abstract stored there describes pre-recorded phone calls. The statement is correct, and only the abbreviation is ambiguous. The asterisk on the first entry points the same way, since by the legend a starred work is not cited in the text, but the asterisks are not reliable (\\fid{P042-E004})."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-A002",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "recalculation_from_source",
   "exhibits": [
    {
     "alt": "Crop of six lines of text on page 31. Red boxes mark the sentence that begins Using linear regression and ends with the words percentage-point increase in turnout per mailing. The lines above say that 147 distinct experimental treatments assessed the effects of direct mail, sometimes sending several mailings to a given recipient.",
     "caption": "Page 31 (PDF p. 7), third paragraph: the marked sentence states that the mail estimates are increases in turnout per mailing.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N003.png"
    },
    {
     "alt": "Crop of ten lines of text on page 31. A red box marks the words 0.162, with a 95\\% confidence interval of (0.078, 0.247). A second red box marks the words minus 0.213 with a 95\\% interval of (minus 0.515, 0.088). Between them the text says that the two types of mailings have significantly different average effects and that nonadvocacy mailings increase turnout by 0.194, with a 95\\% confidence interval of (0.106, 0.282).",
     "caption": "Page 31 (PDF p. 7), same paragraph: the two pooled estimates that change, marked. The estimate for nonadvocacy mail, 0.194, does not change.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N003-b.png"
    },
    {
     "alt": "Chart titled: Putting five estimates on the per-mailing scale lowers conventional mail from 0.162 to 0.132 and moves advocacy mail toward zero; nonadvocacy mail is unchanged. The horizontal axis is the pooled effect on turnout in percentage points per mailing, with a vertical line at zero. For conventional mail, 110 treatments, the printed estimate is 0.162 with interval 0.078 to 0.247 and the corrected estimate is 0.132 with interval 0.055 to 0.208. For nonadvocacy mail, 79 treatments, both are 0.194 with interval 0.106 to 0.282. For advocacy mail, 31 treatments, the printed estimate is minus 0.213 with interval minus 0.515 to 0.088 and the corrected estimate is minus 0.066 with interval minus 0.222 to 0.091. Printed values are drawn as orange open circles and corrected values as blue squares.",
     "caption": "The pooled mail estimates of p. 31 with their 95% intervals, as printed (open circles) and with the five estimates of rows 226–230 on the per-mailing scale (squares). The intervals are computed as in the article.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N003-mail-pools.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "Five of the mail estimates, taken from Gerber, Green and Green (2003), are differences in turnout for whole programmes of four, six or nine mailings and not effects per mailing; on the per-mailing scale the estimate for conventional mail is 0.132 with 95% interval (0.055, 0.208), not 0.162 (0.078, 0.247), and the estimate for advocacy mail is $-0.066$ $(-0.222, 0.091)$, not $-0.213$ $(-0.515, 0.088)$.",
   "id": "P042-N003",
   "location": "p. 31 (PDF p. 7), third paragraph: the sentence that begins “Using linear regression” and the pooled estimates for conventional and advocacy mail that follow; replication dataset, rows 226–230, where the header is row 1.",
   "pdf_page": 7,
   "printed_page": "31",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Moderate",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``In all, 147 distinct experimental treatments assessed the effects of direct mail, sometimes sending several mailings to a given recipient.$^{4}$ Using linear regression, we estimated the percentage-point increase in turnout \\hl{per mailing}. [\\ldots] In this restricted set of 110 studies, we find a weighted average treatment effect of $\\hl{0.162}$, with a 95\\% confidence interval of $\\hl{(0.078, 0.247)}$. [\\ldots] whereas the 31 advocacy mailings have negligible effects on turnout: an estimated effect of $\\hl{-0.213}$ with a 95\\% interval of $\\hl{(-0.515, 0.088)}$.''\n\\begin{audittable}{rlrr}{Replication dataset, rows 226--230 (Gerber, Green and Green, 2003), as stored.}\nRow & Notes & Mail ITT & Mail SE \\\\\n226 & CT Mayoral Race & $\\hl{-1.1}$ & \\hl{1.13} \\\\\n227 & 6 mailers, other Dems \\& Independents & \\hl{0.02} & \\hl{1.2} \\\\\n228 & 4 mailers, other registered voters & \\hl{0.45} & \\hl{1.67} \\\\\n229 & 6 mailers, other registered voters & \\hl{1.04} & \\hl{1.49} \\\\\n230 & 4 mailers, prime democrats only & \\hl{1.92} & \\hl{1.47} \\\\\n\\end{audittable}",
    "corrected": "``In this restricted set of 110 studies, the weighted average treatment effect is $\\hl{0.132}$, with a 95\\% confidence interval of $\\hl{(0.055, 0.208)}$. [\\ldots] whereas the 31 advocacy mailings have negligible effects on turnout: an estimated effect of $\\hl{-0.066}$ with a 95\\% interval of $\\hl{(-0.222, 0.091)}$.''\n\\begin{audittable}{rrrr}{Rows 226--230 on the per-mailing scale: the stored effect and standard error divided by the number of mailings $m$.}\nRow & $m$ & Effect per mailing & Standard error \\\\\n226 & 9 & $\\hl{-0.1222}$ & \\hl{0.1256} \\\\\n227 & 6 & \\hl{0.0033} & \\hl{0.2000} \\\\\n228 & 4 & \\hl{0.1125} & \\hl{0.4175} \\\\\n229 & 6 & \\hl{0.1733} & \\hl{0.2483} \\\\\n230 & 4 & \\hl{0.4800} & \\hl{0.3675} \\\\\n\\end{audittable}",
    "evidence": "The evidence is a check of the source, a derivation and a recalculation from the replication files.\n\n\\emph{Source.} Gerber, Green and Green (2003) write on p.~572 that ``the four mailings increase turnout from 63.7 to 65.6\\%, or approximately 0.5 percentage-points per mailing.'' On p.~573 the Connecticut treatment group ``received nine pieces of direct mail''. Their Table~3 reports differences in turnout of about 1.9, 0.0, 0.4 and 1.0 points for the groups in New Jersey that were sent 4, 6, 4 and 6 mailings, and of $-1.1$ points for Connecticut. The stored values 1.92, 0.02, 0.45, 1.04 and $-1.1$ are these programme differences. Table~3 reports rounded differences and not every standard error; the stored standard errors are close to binomial standard errors on the programme scale. For the New Jersey sample in which the number of mailings varies, the same article reports a slope of 0.16 per mailing with standard error 0.24 (p.~573), which is not among the stored values.\n\n\\emph{Derivation.} Let a treatment group be assigned $m$ mailings and the control group none, let $D$ indicate assignment to treatment, and let $M=mD$. The regression of turnout on $D$ has slope $\\hat b_D$, the difference in turnout between the two groups. Since $a+b_MM=a+(mb_M)D$, the regression on $M$ has the same fitted values. This implies that\n\\[\n\\hat b_M=\\frac{\\hat b_D}{m},\\qquad \\operatorname{SE}(\\hat b_M)=\\frac{\\operatorname{SE}(\\hat b_D)}{m}.\n\\]\nA stored pair $(e_i,s_i)$ that is a programme difference therefore corresponds to the per-mailing pair $(e_i/m_i,\\ s_i/m_i)$, and its weight $1/s_i^2$ becomes $m_i^2/s_i^2$. Let $W=\\sum_i 1/s_i^2$ and $A=\\sum_i e_i/s_i^2$ over a pool that contains the set $T$ of the five rows, so that the pooled estimate is $A/W$ with the article's standard error $1/\\sqrt{W}$. On the per-mailing scale, I obtain\n\\[\nW'=W+\\sum_{i\\in T}\\frac{m_i^2-1}{s_i^2},\\qquad A'=A+\\sum_{i\\in T}\\frac{(m_i-1)\\,e_i}{s_i^2},\n\\]\nand the pooled estimate $A'/W'$ with standard error $1/\\sqrt{W'}$ by the same calculation.\n\n\\emph{Recalculation.} The pooled estimates as printed reproduce from the dataset. Dividing the five rows by 9, 6, 4, 6 and 4 and keeping every other estimate, I find 0.1317 with standard error 0.0391 for conventional mail and $-0.0657$ with standard error 0.0797 for advocacy mail. The division is exact only for an estimate that compares a fixed number of mailings with none. The other scale checks were Niven's (2006) single-mailing estimate and Ramirez's (2005) two Colorado estimates, all per mailing; I did not examine the remaining estimates' units.",
    "problem": "The article states that every mail estimate is an increase in turnout per mailing. Five of the 31 advocacy estimates are not on that scale. Rows 226--230 of the dataset are the differences in turnout between a group that was sent a whole programme of mailings and a control group that was sent none: nine mailings in the Connecticut mayoral race, and four or six in the New Jersey assembly race. For a fixed programme of $m$ mailings versus none, the per-mailing regression coefficient and its standard error are the programme contrast and standard error divided by $m$. The five rows therefore entered the pooling with effect magnitudes that are too large and with weights that are too small by the factors 81, 36, 16, 36 and 16. As stored, they hold 0.5\\% of the weight in the estimate for conventional mail, and on the per-mailing scale they hold 18.0\\%. The corrected estimate for conventional mail is 18.8\\% below the printed one.",
    "what_survives": "The counts and the classification of the treatments are unchanged, as are the estimates for nonadvocacy mail, 0.194 (0.106, 0.282), for social-pressure and gratitude mail, and for canvassing and phone calls. The interval for conventional mail stays above zero, and the interval for advocacy mail still contains zero. With the article's variance calculation, the difference between the two types of mailings stays significant at the 5\\% level: it falls from 0.407 ($z=2.54$) to 0.260 ($z=2.84$), with 95\\% interval (0.080, 0.439). The statements on pp.~31--33 that conventional mail has weak effects and that advocacy mail has little effect on turnout stand. The pooled estimate for all 147 mail treatments in the replication program, which the article does not print, changes from 0.743 (0.669, 0.816) to 0.637 (0.568, 0.705). These intervals omit covariance between contrasts sharing controls; see \\fid{P042-E001}."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-N003",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "recalculation_from_source",
   "exhibits": [
    {
     "alt": "Crop of six lines of text on page 32. Red boxes mark the words The 29 social pressure treatments and the number 2.850 in the sentence: The 29 social pressure treatments generated a weighted average treatment effect of 2.850 percentage points, 95\\% CI = (2.686, 3.014).",
     "caption": "Page 32 (PDF p. 8), first paragraph: the count and the pooled estimate for social-pressure mail, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N005.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "Among the 29 estimates for social-pressure mail, three treatments of the 2007 Kentucky experiment of Mann (2010) are counted twice, once for people who did not vote in 2006 and once for a broader sample that contains those people; each treatment should enter once, which leaves 26 estimates with a weighted average of 2.874 or 3.069 according to the sample kept, against the printed 2.850.",
   "id": "P042-N005",
   "location": "p. 32 (PDF p. 8), first paragraph: “The 29 social pressure treatments …”; replication dataset, rows 171–176 (Mann, 2010), where the header is row 1.",
   "pdf_page": 8,
   "printed_page": "32",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``The \\hl{29} social pressure treatments generated a weighted average treatment effect of $\\hl{2.850}$ percentage points, 95\\% CI $= (2.686, 3.014)$.''\n\\begin{audittable}{rlrr}{Replication dataset, rows 171--176 (Mann, 2010; election field ``2007 KY gubernatorial''), as stored.}\nRow & Notes & Mail ITT & Mail SE \\\\\n171 & Mann\\_2007\\_KY hawthorne (didn't vote 06) & 2.1 & 0.3 \\\\\n172 & Mann\\_2007\\_KY self (didn't vote 06) & 2.1 & 0.3 \\\\\n173 & Mann\\_2007\\_KY hawthorne+survey (didn't vote 06) & 2.2 & 0.3 \\\\\n174 & Mann\\_2007\\_KY hawthorne+survey & 2.5 & 0.35 \\\\\n175 & Mann\\_2007\\_KY hawthorne & 2.6 & 0.35 \\\\\n176 & Mann\\_2007\\_KY self & 3.1 & 0.35 \\\\\n\\end{audittable}",
    "corrected": "Each Kentucky treatment should enter once, for one stated sample. With the estimates for the 2006 nonvoters, rows 171--173: ``The \\hl{26} social pressure treatments generated a weighted average treatment effect of $\\hl{2.874}$ percentage points.'' With the estimates for the broader sample, rows 174--176, the average is $\\hl{3.069}$. I could not determine which sample was intended. The interval should then account for all dependent contrasts, including the Kentucky contrasts, which share a control group (\\fid{P042-E001}).",
    "evidence": "The evidence is a check of the source and a recalculation from the replication files. I read Mann (2010) in the author's posted manuscript of 20 April 2010, not in the journal's typesetting. The first sample of the study has 78,441 people who did not vote in 2006. The broader sample has no further restriction (p.~12 of the manuscript), and Table~6 reports turnout of 13.2\\%, 16.3\\%, 15.8\\% and 15.7\\% in it. The differences from the first figure, 3.1, 2.6 and 2.5 points, are the values stored in rows 176, 175 and 174. The analysis of the broader sample leaves out only the 17,609 recipients of a fourth mailing from 95,788 people, so that 60,832 of the 2006 nonvoters are in both analyses. The 19,561 nonvoters in the control group are among the 25,037 controls of the broader sample. The printed 2.850 reproduces from the 29 rows. Without rows 174--176, the 26 remaining rows have a weighted average of 2.874, and without rows 171--173, of 3.069.",
    "problem": "For the Hawthorne, Self and Hawthorne-plus-survey mailings the dataset holds two estimates each. Rows 171--173 are labelled as estimates for people who did not vote in 2006. Rows 174--176 carry no such label. They are not estimates for the people who did vote in 2006. They are the estimates for the broader sample of the study, which contains the 2006 nonvoters. The same people therefore stand behind two estimates of each mailing, and three treatments are counted twice.",
    "what_survives": "The effect of social-pressure mail stays large under either choice, and the pooled estimate moves by 1\\% to 8\\%. The pooled estimates for conventional and gratitude mail are unaffected, and no other pooled estimate of pp.~31--33 uses these rows. The overlap adds to the dependence between contrasts that the printed interval omits (\\fid{P042-E001}), and I did not establish a corrected interval."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-N005",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "recalculation_from_source",
   "exhibits": [
    {
     "alt": "Crop of six lines of text on page 32. Red boxes mark the words 71 canvassing treatments and the words 2.536, with a 95\\% confidence interval of (1.817, 3.255) in the sentence: Analyzing the effects of 71 canvassing treatments reveals a weighted average CACE of 2.536, with a 95\\% confidence interval of (1.817, 3.255).",
     "caption": "Page 32 (PDF p. 8), third paragraph: the count, the pooled estimate and the interval for canvassing, marked.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N006.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The 71 canvassing estimates include the 1998 New Haven experiment twice, as first reported in 1999 and as corrected in 2005; with the superseded estimate removed, the 70 remaining estimates have a weighted average CACE of 2.434 with nominal 95% interval (1.706, 3.163), not 2.536 (1.817, 3.255).",
   "id": "P042-N006",
   "location": "p. 32 (PDF p. 8), third paragraph, first sentence; replication dataset, rows 10 and 11, where the header is row 1.",
   "pdf_page": 8,
   "printed_page": "32",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``Analyzing the effects of \\hl{71} canvassing treatments reveals a weighted average CACE of $\\hl{2.536}$, with a 95\\% confidence interval of $\\hl{(1.817, 3.255)}$.''\n\\begin{audittable}{rlrrrrl}{Replication dataset, rows 10 and 11, as stored.}\nRow & Author & Year & Canvass ATT & Canvass SE & N & Election/Intervention date \\\\\n10 & Gerber, Green & 2005 & 8.4 & 2.6 & 29380 & 1998 general New Haven \\\\\n11 & Gerber, Green & 1999 & \\hl{6.33} & \\hl{2.27} & 28430 & 1998 general New Haven \\\\\n\\end{audittable}",
    "corrected": "``Analyzing the \\hl{70 remaining} canvassing \\hl{estimates} reveals a weighted average CACE of $\\hl{2.434}$, with a \\hl{nominal} 95\\% confidence interval of $\\hl{(1.706, 3.163)}$.'' Row 11 of the dataset is removed from the pool and row 10 is kept.",
    "evidence": "The evidence is a recalculation from the replication files in exact arithmetic. The 71 rows reproduce the printed 2.536 (1.817, 3.255). Without row 11, I obtain 2.4344 with standard error 0.3717 and nominal interval (1.7059, 3.1629) from the 70 remaining rows. That the two rows are one experiment rests on their election fields, on the abstracts stored with them and on p.~31 of the article, which identifies the 1998 New Haven study. I did not read the two source articles. A second pair, for a Detroit experiment of 2001 in rows 9 and 16, is not established as a duplicate, because the fields of row 9 conflict (\\fid{P042-E003}). Removing row 9 as well would give 2.410 (1.681, 3.139) over 69 estimates.",
    "problem": "Rows 10 and 11 of the dataset are two analyses of one experiment, the canvassing arm of the 1998 New Haven study. Row 11 is the report of 1999, ``Does canvassing increase voter turnout? A field experiment'', and row 10 is the correction of 2005, ``Correction to Gerber and Green (2000), Replication of Disputed Findings, and Reply to Imai (2005)''. The canvassing command of the replication program pools every row that has a canvassing estimate, and both rows enter. The experiment receives 2.31 times the weight of the corrected estimate alone, and a superseded estimate is pooled as if it were a separate treatment.",
    "what_survives": "The estimate for canvassing stays positive, with an interval above zero, and the qualitative comparison of tactics on p.~33 is unchanged. No other pooled estimate uses these rows. The interval keeps the article's calculation, which is the subject of \\fid{P042-E001}."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-N006",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "recalculation_from_source",
   "exhibits": [
    {
     "alt": "Crop of the paragraph on GOTV phone calls on page 32. Red boxes mark '0.156', 'eleven' and '-0.118 to 0.430' in the sentences 'Excluding two experiments that tested the effectiveness of a pre-recorded social pressure message, we find a weighted average effect of 0.156 percentage points among the remaining eleven treatments, which tested the effects of messages recorded by politicians, celebrities or local clergy. The confidence interval, which ranges from -0.118 to 0.430 percentage points, indicates that the treatment effects of this tactic are very close to zero.'",
     "caption": "p. 32 (PDF p. 8). The marked estimate, count and interval are those of the eleven estimates for pre-recorded calls; one of the eleven is coded as a social-pressure call in the replication dataset.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N010.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The eleven estimates pooled as ordinary pre-recorded calls include one that the replication dataset codes as a social-pressure call; without it the weighted average is 0.113 percentage points, with interval $(-0.164, 0.391)$, over ten treatments, in place of 0.156, with interval $(-0.118, 0.430)$, over eleven.",
   "id": "P042-N010",
   "location": "p. 32 (PDF p. 8), paragraph beginning “Studies that assess the effects of GOTV phone calls”, third and fourth sentences",
   "pdf_page": 8,
   "printed_page": "32",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``Excluding two experiments that tested the effectiveness of a pre-recorded social pressure message, we find a weighted average effect of \\hl{0.156} percentage points among the remaining \\hl{eleven} treatments, which tested the effects of messages recorded by politicians, celebrities or local clergy. The confidence interval, which ranges from $\\hl{-0.118}$ to \\hl{0.430} percentage points, indicates that the treatment effects of this tactic are very close to zero.''",
    "corrected": "``\\hl{After excluding the social-pressure entries, the remaining ten pre-recorded-call treatment estimates have} a weighted average effect of \\hl{0.113} percentage points, with a 95\\% confidence interval from $\\hl{-0.164}$ to \\hl{0.391} percentage points. \\hl{This summary continues to indicate a small average effect of conventional recorded calls.}''\n\nHere the weighted average is the fixed effects average with weights $s_i^{-2}$, as on p.~31, and the ten estimates are the eleven of the replication program without the estimate of Gerber, Green, Kaplan and Kern (2010). In the replication program, the selection of pre-recorded-call estimates takes the further condition \\texttt{socialpressure != 1}.",
    "evidence": "The evidence is a recalculation from the replication files in exact rational arithmetic. Let $b_i$ denote the estimates and $s_i$ their standard errors, in percentage points, and set $w_i=s_i^{-2}$. The fixed effects average and its standard error are\n\\[\n\\widehat b=\\frac{\\sum_i w_i b_i}{\\sum_i w_i},\\qquad \\operatorname{SE}(\\widehat b)=\\Bigl(\\sum_i w_i\\Bigr)^{-1/2},\n\\]\nand the 95\\% interval is $\\widehat b\\pm z\\operatorname{SE}(\\widehat b)$ with $z\\approx1.95996398454$. For the eleven estimates, I have $\\sum_i w_i=51.25488$ and $\\sum_i w_i b_i=8.00301$. Then $\\widehat b=0.15614$, with standard error 0.13968 and interval $(-0.11763, 0.42991)$, which is the printed figure. The estimate of Gerber, Green, Kaplan and Kern has $w=0.89^{-2}=1.26247$ and $wb=2.33556$. Removing it, I obtain $\\sum_i w_i=49.99241$ and $\\sum_i w_i b_i=5.66744$, and therefore\n\\[\n\\widehat b=0.11337,\\qquad \\operatorname{SE}(\\widehat b)=0.14143,\\qquad \\widehat b\\pm z\\operatorname{SE}(\\widehat b)=(-0.16384,\\ 0.39057).\n\\]\nThe removed estimate holds 2.46\\% of the weight of the eleven, and the average falls by 0.043 points.",
    "problem": "The paragraph sets the social-pressure calls aside and reports the average effect of the remaining pre-recorded calls, which it describes as messages recorded by politicians, celebrities or local clergy. The replication program forms this set by dropping two entries, titled ``Robo SP1'' and ``Robo SP2'', from the thirteen estimates for pre-recorded calls. Among the eleven that remain is the estimate of Gerber, Green, Kaplan and Kern (2010), which is 1.85 with standard error 0.89 and which the replication dataset codes as a social-pressure call. This is the study cited on p.~36 for the statement that pre-recorded calls with a social-pressure message increase turnout (see \\fid{P042-A002}). The printed average is not the average for ordinary pre-recorded calls that the paragraph describes. The arithmetic is not at fault. From the eleven estimates, I find 0.156 and $(-0.118, 0.430)$, as printed, and it is true that two entries were excluded.\n\nThree points are not established. The replication files do not show whether the three entries coded or titled as social pressure are three separate experiments, or whether the estimate of Gerber, Green, Kaplan and Kern repeats one of the two excluded entries. They do not show who recorded its message. They also do not show whether the stored 1.85 is an effect of assignment, as the paragraph says of the eleven, or an effect among those reached.",
    "what_survives": "The conclusion of the paragraph stands: the average effect of ordinary pre-recorded calls is small, and its interval contains zero. The count, the estimate and the interval on p.~32 change, and the rough conversion to a CACE in Note~5, which divides the effect of assignment by 0.8, starts from 0.113. The pooled figures for mail, canvassing, commercial calls and volunteer calls use other estimates and do not change. I do not change the count of two experiments in Note~5, since the replication files do not show that there were three. The figure that Note~5 gives for those two is the subject of \\fid{P042-E002}."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-N010",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "recalculation_from_source",
   "exhibits": [
    {
     "alt": "Crop of the top of page 33. Red boxes mark '1.936', '(1.298, 2.575)' and '37' in the sentence 'Excluding experiments that conducted follow-up calls with respondents who had previously indicated an intention to vote (see below), volunteer phone banks generate an average CACE of 1.936 percentage points, with a 95 percent CI = (1.298, 2.575), over 37 experimental treatments.' The sentence before it says that live calls from volunteer phone banks seem to be more effective than commercial calls, whose average CACE is 0.980.",
     "caption": "p. 33 (PDF p. 9), first paragraph. The marked estimate, interval and count are those of the 37 estimates for volunteer calls, one of which is the overall effect of a two-round campaign with follow-up calls.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N011.png"
    },
    {
     "alt": "Chart with five horizontal intervals under the fixed effects independence model on an axis in percentage points of turnout, from -0.5 to 3. Commercial calls, 25 treatments, grey diamond: 0.980 (0.504, 1.456). Volunteer calls as printed, 37 treatments, blue circle: 1.936 (1.298, 2.575). Volunteer calls corrected, 36 treatments, orange square: 1.710 (1.064, 2.357). Volunteer minus commercial with the 37 treatments, blue circle: 0.956 (0.160, 1.752). Volunteer minus commercial with the 36 treatments, orange square: 0.730 (-0.072, 1.533), an interval that crosses the vertical line at zero.",
     "caption": "The pooled CACE for volunteer calls as printed and without the OCAPICA June 2008 estimate, beside the pooled CACE for commercial calls, and the difference between the volunteer and commercial figures in each case, with 95% intervals under the fixed effects independence model. Without the estimate, the interval for the difference includes zero.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N011-pooled.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The 37 estimates for volunteer calls include the overall effect of a two-round campaign with follow-up calls, which the sentence says is excluded; without it the pooled CACE is 1.710, with 95% interval (1.064, 2.357), over 36 treatments, and under independence the pooled figures no longer separate volunteer from commercial calls at the two-sided 5% level.",
   "id": "P042-N011",
   "location": "p. 33 (PDF p. 9), first paragraph, sentence beginning “Excluding experiments that conducted follow-up calls”",
   "pdf_page": 9,
   "printed_page": "33",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Moderate",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``Live calls from volunteer phone banks seem to be more effective. Excluding experiments that conducted follow-up calls with respondents who had previously indicated an intention to vote (see below), volunteer phone banks generate an average CACE of \\hl{1.936} percentage points, with a 95\\% CI = \\hl{(1.298, 2.575)}, over \\hl{37} experimental treatments.''",
    "corrected": "``Excluding \\hl{the OCAPICA June 2008 experiment, which} conducted follow-up calls with respondents who had previously indicated an intention to vote (see below), volunteer phone banks generate an average CACE of \\hl{1.710} percentage points, with a 95\\% CI = \\hl{(1.064, 2.357)}, over \\hl{36} experimental treatments.''\n\nThe average is the fixed effects average with weights $s_i^{-2}$, as on p.~31, over the 37 estimates of the replication program without the OCAPICA June 2008 estimate, and the interval assumes that the 36 estimates are independent. The comparison with commercial calls then reads: ``The volunteer pooled CACE is higher than the commercial pooled CACE by \\hl{0.730} percentage points, but under the fixed effects independence model its 95\\% difference interval is $\\hl{(-0.072, 1.533)}$; delivery-mode and authenticity explanations remain tentative.'' Any comparison with the effect of a volunteer phone call, or with the typical CACE of a single call if this pooled figure is meant, uses 1.710. Whether the other nine estimates from the same book meet the exclusion is not established, and I remove none of them.",
    "evidence": "The evidence has two parts. The first is the text of Michelson, García Bedolla and McConnell (2009), pp.~1556--1557, quoted above, available in indexed excerpts of a repository copy; the typeset Table~7 was not inspected. The second is a recalculation from the replication files using exact rational weights and sums. Let $y_i$ denote the estimates and $s_i$ their standard errors, in percentage points. I set $w_i=s_i^{-2}$, $W=\\sum_i w_i$ and $S=\\sum_i w_i y_i$. The fixed effects average is $m=S/W$, its variance under independence is $1/W$, and the 95\\% interval is $m\\pm zW^{-1/2}$ with $z\\approx1.95996$. For the 37 estimates, I have $W\\approx9.42573$ and $S\\approx18.25079$, giving an average that rounds to the printed 1.936. The OCAPICA estimate has $y_j=11.1$ and $s_j=2.1$, so that $w_j=100/441$ and $w_jy_j=370/147$. Removing it, I obtain\n\\[\nW'=W-\\frac{100}{441}\\approx9.19897,\\qquad S'=S-\\frac{370}{147}\\approx15.73379,\\qquad m'=\\frac{S'}{W'}\\approx1.71039,\n\\]\nwith standard error $(W')^{-1/2}\\approx0.32971$. The change in the average is\n\\[\nm-m'=\\frac{w_j}{W}\\,(y_j-m')\\approx0.22589 .\n\\]\nFor the difference $d$ between two pooled estimates with weight sums $W_A$ and $W_B$, independence implies that $\\operatorname{SE}(d)=(1/W_A+1/W_B)^{1/2}$, and I compare $z=d/\\operatorname{SE}(d)$ with the normal distribution. The commercial calls have weight sum 16.95194.\n\\begin{audittable}{lrrrr}{Pooled CACE in percentage points, and the difference between volunteer and commercial calls, from the replication files.}\nQuantity & Treatments & Estimate & SE & 95\\% interval \\\\\nCommercial calls & 25 & 0.980 & 0.243 & (0.504, 1.456) \\\\\nVolunteer calls, as printed & 37 & 1.936 & 0.326 & (1.298, 2.575) \\\\\nVolunteer calls, corrected & 36 & 1.710 & 0.330 & (1.064, 2.357) \\\\\nVolunteer minus commercial, with the 37 & & 0.956 & 0.406 & (0.160, 1.752) \\\\\nVolunteer minus commercial, with the 36 & & 0.730 & 0.410 & $(-0.072, 1.533)$ \\\\\n\\end{audittable}\nThe two differences have $z=2.353$ and $z=1.783$, with two-sided $p$-values 0.0186 and 0.0746.",
    "problem": "The sentence says that the pooled estimate for volunteer phone banks leaves out the experiments in which callers made follow-up calls to respondents who had said that they intended to vote. The 37 estimates pooled by the replication program include one from this class, the estimate for the OCAPICA phone bank in the election of June 2008. It is 11.1 with standard error 2.1, and the replication dataset lists its source as the book of García Bedolla and Michelson (2012). The campaign is described by Michelson, García Bedolla and McConnell (2009), the study of two-round phone banks that the article discusses on pp.~34--35. First-round respondents who intended to vote ``were targeted for reminder calls just before Election Day'' (p.~1556), and Table~7 of that study (p.~1557) reports 11.1 (2.1) as ``the effect of one or more contacts'', that is, the effect of the two-round campaign as a whole. The estimate lies 9.4 points above the average of the other 36, and with it the pooled CACE is too high by 0.226 points.\n\nThe comparison drawn in the paragraph is affected as well. The article says that volunteer calls ``seem to be more effective'' than commercial calls, whose pooled CACE is 0.980. Under the independence model that the pooling uses, the difference between the two pooled estimates is 0.956 with $z=2.35$ when the estimate is included, and 0.730 with $z=1.78$ when it is removed. The pooled figures then no longer separate volunteer from commercial calls at the two-sided 5\\% level. The article prints no such test, and it words the comparison tentatively. The correction weakens the numerical support for the comparison, and for the reading on p.~33 that the effect of a live call depends on whether the message is ``delivered in a routinized way by a commercial phone bank or in a more authentic manner by a volunteer phone bank''.\n\nOne point is not established. Nine other estimates among the 37 come from the same book. The replication dataset contains neither the number of rounds of calls nor the contrast behind each estimate, and therefore does not show whether any of the nine is also the overall effect of a two-round campaign. The corrected figure removes the one estimate known to belong to the excluded class.",
    "what_survives": "The pooled estimate for volunteer calls remains positive, with a 95\\% interval above zero under independence, and its point estimate remains above that of commercial calls. The figures for commercial calls, canvassing, mail and pre-recorded calls do not use the removed estimate and are not changed by this correction. The experiments with follow-up calls reported on pp.~34--35 are also unchanged. If the statement on p.~35 that the CACE in those experiments is ``more than twice as large as the typical CACE of a single phone call'' was valid before correction, it remains valid when the typical CACE is this pooled figure, since the bound to exceed falls from 3.873 to 3.421. The text-message CACE of 4.1 on p.~33, which ``rivals the effect of a volunteer phone call'', is now compared with 1.710. The count of 37 and the estimate 1.936 change, and under independence the pooled figures no longer separate volunteer from commercial calls at the two-sided 5\\% level."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-N011",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "error",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of a paragraph on page 34. Red boxes mark 'in 2006 and 2008' and '(Michelson et al., 2007)' in the sentence 'A series of experiments directed at minority voters in California in 2006 and 2008 turned up no evidence of interactions between mailings and phone calls (Michelson et al., 2007).'",
     "caption": "p. 34 (PDF p. 10), second paragraph of the section on synergy. The marked years and the marked citation do not agree.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N012.png"
    },
    {
     "alt": "Crop of the reference list on page 46. A red box marks the year '(2007)' in the entry 'Michelson, Melissa R., Bedolla, Lisa García and Green, Donald P. (2007) New experiments in minority voter mobilization, Report for The James Irvine Foundation.'",
     "caption": "Reference list, p. 46 (PDF p. 22). The cited report is dated 2007.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N012-b.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The sentence dates the California experiments to 2006 and 2008 but cites only a report dated 2007, which cannot cover experiments of 2008.",
   "id": "P042-N012",
   "location": "p. 34 (PDF p. 10), section “Synergy, Diminishing Returns, or Neither?”, second paragraph; reference entry on p. 46 (PDF p. 22)",
   "pdf_page": 10,
   "printed_page": "34",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": "Minor",
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``A series of experiments directed at minority voters in California \\hl{in 2006 and 2008} turned up no evidence of interactions between mailings and phone calls \\hl{(Michelson et al., 2007)}.''\n\nThe reference list has one matching entry: ``Michelson, Melissa R., Bedolla, Lisa García \\& Green, Donald P. (2007) New experiments in minority voter mobilization, Report for The James Irvine Foundation.''",
    "corrected": "``A series of experiments directed at minority voters in California \\del{in 2006 and 2008} turned up no evidence of interactions between mailings and phone calls (Michelson et al., 2007).''\n\nAlternatively, the sentence keeps the years and adds a later source for the experiments of 2008, once that source has been checked.",
    "evidence": "The evidence is a check of the two printed pages: the sentence on p.~34 and the single matching entry of the reference list on p.~46.",
    "problem": "A report issued in 2007 cannot report experiments conducted in 2008, so the years in the sentence and the citation do not agree. Which of the two should change is not established, since I have read neither the report nor a later source for the experiments of 2008. The statement that the experiments showed no interactions is not in question.",
    "what_survives": "The conclusion of the section, that interactions between treatments tend to be weak, rests on several other studies that the paragraph discusses separately, and it is unaffected."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-N012",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  },
  {
   "before_review": null,
   "class": "clarity_note",
   "date_modified": "2026-10-25",
   "date_published": "2026-10-25",
   "evidence_type": "page_check",
   "exhibits": [
    {
     "alt": "Crop of a paragraph on page 39. A red box marks the word 'schooling' in the sentence 'For example, a growing literature attempts to gauge the causal effect of schooling on voter turnout by tracing the downstream consequences of random interventions that increase educational attainment (Sondheimer \\& Green, 2010) or family income (De La O, 2012; Doherty et al., 2006).' The sentence before it reads 'Other attributes, such as education or affluence, are also susceptible to experimental manipulation.'",
     "caption": "p. 39 (PDF p. 15), last paragraph before the Notes. The marked word names one of the two attributes that the sentence goes on to discuss; the preceding sentence names both.",
     "url": "https://pmaronow.github.io/llm-audit/works/P042/exhibits/P042-N014.png"
    }
   ],
   "formal_release": null,
   "formal_statements": [],
   "headline": "The sentence names only schooling as the cause under study, although it cites experiments that raise schooling and experiments that raise family income; “schooling or family income” is meant.",
   "id": "P042-N014",
   "location": "p. 39 (PDF p. 15), last paragraph before the Notes, sentence beginning “For example, a growing literature”",
   "pdf_page": 15,
   "printed_page": "39",
   "reaches": [],
   "related_entries": [],
   "revisions": [],
   "same_defect_as": [],
   "severity": null,
   "subject": "paper",
   "target_ids": [],
   "text": {
    "as_printed": "``For example, a growing literature attempts to gauge the causal effect of \\hl{schooling} on voter turnout by tracing the downstream consequences of random interventions that increase educational attainment (Sondheimer \\& Green, 2010) or family income (De La O, 2012; Doherty et al., 2006).''",
    "corrected": "``For example, a growing literature attempts to gauge the causal effect of schooling \\hl{or family income} on voter turnout by tracing the downstream consequences of random interventions that increase educational attainment (Sondheimer \\& Green, 2010) or family income (De La O, 2012; Doherty et al., 2006).''",
    "intended_meaning": "The preceding sentence reads: ``Other attributes, such as education or affluence, are also susceptible to experimental manipulation.'' The example covers both attributes. Interventions that raise educational attainment are used to study the effect of schooling, and interventions that raise family income are used to study the effect of income. The sentence does not propose income interventions as a way to measure the effect of schooling. One point is left open. Doherty et al.\\ (2006) is, by its title in the reference list, a study of personal income and attitudes toward redistribution, and its abstract describes effects on political attitudes. I could read only the abstract and cannot say whether the study reports a turnout outcome. This decides whether the citation fits a sentence about voter turnout."
   },
   "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-N014",
   "work": {
    "arxiv_id": null,
    "doi": "10.1080/17457289.2012.728223",
    "title": "Field Experiments and the Study of Voter Turnout",
    "url": "https://pmaronow.github.io/llm-audit/works/P042/"
   },
   "work_id": "P042"
  }
 ],
 "lean": {
  "applicable": true,
  "guide": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/docs/works/P042.md",
  "repository": "https://github.com/pmaronow/llm-audit-lean",
  "targets": [
   {
    "declarations": [
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The row intentionally verifies the sharp-null property built into target A's chosen potential-outcome table; its definitional character is appropriate to this auxiliary witness certificate. It is not credited as the full counterexample or as an empirical null for the paper's studies.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.six_sharp_null",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L101",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The complement definition also supplies the control's disjointness and exhaustiveness. Uniformity is represented by the 1/|S| averaging convention in rationalMean, rather than a missing random-assignment premise. This is the design certificate for witness A.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.six_design",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L105",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The existing six_moments theorem already transports the integer computation to the actual contrasts.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.six_integer_moments",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L134",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The existing six_moments theorem already computes moments of the actual contrasts.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.six_contrast_numerators",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L145",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "This is the moment component of witness A, exactly A1–A3 and A5. Population variances use the allocation count, not a sample-variance divisor; all finite sums are rational and every denominator is positive.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.six_moments",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L154",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The other A rows in this chunk certify the sharp null, design and actual marginal variances, so the real vector 3/10 used for the weights is not an unrelated abstract variance vector. This row exports A4 and the strict variance discrepancy in A5, rather than asserting a coverage error for the article's historical intervals. The record revealed no misread fact and did not change the claim-stage verdict.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.six_counterexample",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L227",
      "status": "counterexample_proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The computational descending-factorial definition is independent of Nat.choose, not an assumed coefficient equality. Natural division is exact; truncated subtraction gives the standard zero coefficient when k>n. This is the coefficient-representation auxiliary used by B's masses and coverage sums.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.exactChoose_eq",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L995",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The derived count variance is 2525-50^2=25. This is an auxiliary fact for the specified witness, not an assertion that all binomial laws have these moments.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_arm_moments",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1033",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_expectation_add",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1045",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_expectation_sub",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1051",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_expectation_mul",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1057",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_expectation_div",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1062",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_expectation_tensor",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1072",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_marginal_zero",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1082",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_marginal_one",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1088",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Auxiliary expectation identity already used inside the existing shared-control moment theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_marginal_two",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1094",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Covers B1 and B2–B3's moment and pooled-formula components. Equal arm success probabilities supply B's zero mean effects; the common control induces covariance 1/400. The equal precision weights and Vdiag=1/W identity are assigned to binomial_diagonal in the other split part, and B4–B6 are not attributed to this row. The record revealed no misread fact and did not change the claim-stage verdict.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_shared_control_moments",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1101",
      "status": "counterexample_proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The true variance 3/800 is not established by this numerical lemma; that component is assigned to binomial_shared_control_moments in the other split chunk. Verdict unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_diagonal",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1235",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The existing numerical threshold equivalences already supply the required interval-containment result.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.covers_zero_iff_abs",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1249",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The target fixes the conventional 1.96 multiplier as 49/25, exactly as the statement does.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_diagonal_radius",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1256",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "This establishes interval-event equivalences and the squared-radius bounds, not the separate variance or coverage-probability evaluations. Verdict unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_interval_thresholds",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1265",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "The existing coverage-linkage proof already supplies the convolution step; no separate helper clause is stated.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_convolution",
      "manifest_use": "not_used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1317",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": false
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "Exact for its numerical component; R243-01 separately links the evaluated sum to the three-arm event. Verdict unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_coverage_exact",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1332",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "This is the coprimality component of B5; R243-04 supplies the fraction equality.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.binomial_coverage_fraction_reduced",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1342",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
      "conditional_on": [],
      "credit_reason": "A harmless generalization of the B5/B6 probability identity; it is not a standalone full counterexample or a claim about historical intervals. Verdict unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.Wave1.binomial_coverage_from_arms_proof",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P041/Wave1/Gap/GP041P042P045B2.lean#L110",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     }
    ],
    "formalization": {
     "claim_revision_id": "catalog:742ff6a518d3f1f62613b069d1b895a767fc1e4c5af67beeacbc7d5048acead8",
     "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P042-T1",
     "decision": "whole",
     "release_id": "sap-2026-11-08-assured-release",
     "target_id": "P042-T1"
    },
    "html": {
     "body": "<p>pooling on p. 31 (PDF p. 7), intervals on pp. 31–33, shared control on p. 36. The two constructed sharp-null shared-control witnesses have true pooled variance three halves of the precision calculation&#x27;s diagonal variance. The six-voter witness has the stated 90 allocations and exact moments. The independent binomial-arm witness has all stated moments, weights, exact coverage sums, reduced fraction, and strict coverage bounds. Its coverage thresholds are 19 and 24 both for 1.96 and for the actual standard normal 0.975 quantile, indeed for every multiplier in [1.9599, 1.96]. The usual measure-theoretic conditions are assumed; these finite laws satisfy them. These are constructed witnesses, with no assertion about historical interval coverage.</p>",
     "headline": "The pooled 95% confidence intervals of pp. 31–33 under the ‘fixed effects weighting scheme’ of p. 31, read as resting on the variance <span class=\"katex\"><span class=\"katex-mathml\"><math xmlns=\"http://www.w3.org/1998/Math/MathML\"><semantics><mrow><mn>1</mn><mi mathvariant=\"normal\">/</mi><msub><mo>∑</mo><mi>j</mi></msub><msubsup><mi>s</mi><mi>j</mi><mrow><mo>−</mo><mn>2</mn></mrow></msubsup></mrow><annotation encoding=\"application/x-tex\">1/\\sum_j s_j^{-2}</annotation></semantics></math></span><span class=\"katex-html\" aria-hidden=\"true\"><span class=\"katex-base\"><span class=\"katex-strut\" style=\"height:1.2901em;vertical-align:-0.4358em;\"></span><span class=\"mord\">1/</span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mop\"><span class=\"mop op-symbol small-op\" style=\"position:relative;top:0em;\">∑</span><span class=\"msupsub\"><span class=\"vlist-t vlist-t2\"><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.162em;\"><span style=\"top:-2.4003em;margin-left:0em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"></span><span class=\"katex-sizing reset-size6 size3 mtight\"><span class=\"mord mathnormal mtight\" style=\"margin-right:0.0572em;\">j</span></span></span></span><span class=\"vlist-s\">​</span></span><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.4358em;\"><span></span></span></span></span></span></span><span class=\"mspace\" style=\"margin-right:0.1667em;\"></span><span class=\"mord\"><span class=\"mord mathnormal\">s</span><span class=\"msupsub\"><span class=\"vlist-t vlist-t2\"><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.8542em;\"><span style=\"top:-2.4231em;margin-left:0em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"></span><span class=\"katex-sizing reset-size6 size3 mtight\"><span class=\"mord mathnormal mtight\" style=\"margin-right:0.0572em;\">j</span></span></span><span style=\"top:-3.1031em;margin-right:0.05em;\"><span class=\"pstrut\" style=\"height:2.7em;\"></span><span class=\"katex-sizing reset-size6 size3 mtight\"><span class=\"mord mtight\"><span class=\"mord mtight\">−</span><span class=\"mord mtight\">2</span></span></span></span></span><span class=\"vlist-s\">​</span></span><span class=\"vlist-r\"><span class=\"vlist\" style=\"height:0.413em;\"><span></span></span></span></span></span></span></span></span></span> of the precision-weighted average with no condition on the dependence between the pooled estimates; the two witnesses are the validators&#x27; constructions and are not in the paper",
     "location": "p. 31 (PDF p. 7), paragraph beginning ‘Meta-analysis is a statistical method’; the nine pooled intervals on pp. 31–33 (PDF pp. 7–9); the single control group of Gerber et al. (2008) on p. 36 (PDF p. 12)"
    },
    "id": "P042-T1",
    "text": {
     "body": "pooling on p. 31 (PDF p. 7), intervals on pp. 31–33, shared control on p. 36. The two constructed sharp-null shared-control witnesses have true pooled variance three halves of the precision calculation's diagonal variance. The six-voter witness has the stated 90 allocations and exact moments. The independent binomial-arm witness has all stated moments, weights, exact coverage sums, reduced fraction, and strict coverage bounds. Its coverage thresholds are 19 and 24 both for 1.96 and for the actual standard normal 0.975 quantile, indeed for every multiplier in [1.9599, 1.96]. The usual measure-theoretic conditions are assumed; these finite laws satisfy them. These are constructed witnesses, with no assertion about historical interval coverage.",
     "headline": "The pooled 95% confidence intervals of pp. 31–33 under the ‘fixed effects weighting scheme’ of p. 31, read as resting on the variance $1/\\sum_j s_j^{-2}$ of the precision-weighted average with no condition on the dependence between the pooled estimates; the two witnesses are the validators' constructions and are not in the paper",
     "location": "p. 31 (PDF p. 7), paragraph beginning ‘Meta-analysis is a statistical method’; the nine pooled intervals on pp. 31–33 (PDF pp. 7–9); the single control group of Gerber et al. (2008) on p. 36 (PDF p. 12)"
    },
    "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T1"
   },
   {
    "declarations": [
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The mean component of (i) belongs to a different credited row; this row is accurately limited to weights.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.precision_weights",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L241",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The row establishes only the diagonal contribution and does not claim that the true pooled variance omits covariances.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.precision_diagonal",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L255",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "This covers only the expectation clause of (i); positivity and normalization of precision weights are separate credited declarations. The probability and L² assumptions are already H1, so being stronger than necessary for bare linearity does not narrow the credited claim. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.pooled_mean",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L275",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "H1 explicitly supplies finite second moments, so MemLp 2 is not an added moment restriction. The additional empty and degenerate families do not change the claimed specialization.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.pooled_variance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L287",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Finite variances for real random variables on a probability space entail the L2 hypotheses used here. No independence or nondegenerate-control assumption is added.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.shared_control_covariance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L301",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The finite sums defining both pooled means preserve L² under H1, so their substitution into the general identity is legitimate. No extra independence or positive contrast variance is required. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.difference_variance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L312",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Symmetry is automatic for the target covariance matrix. Extending the algebraic identity beyond positive semidefinite matrices does not alter its covariance specialization. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.covariance_form_decomposition",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L496",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The iff concerns the whole weighted off-diagonal sum: individual covariances may be nonzero and cancel. H1 supplies exactly the L² and positive-variance hypotheses used here. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.corrected_precision_variance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L540",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The covariance-zero premise is part of the corrected sufficiency claim (iii); its absence from the original article does not make this corrected row a narrowing.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.uncorrelated_precision_variance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L569",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The positive-pair hypothesis forces J>=2. Pairwise covariance symmetry makes existence of a positive unequal pair equivalent to existence of a positive ordered pair j<k.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.positive_covariance_inflation",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L588",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The target defines s and t as marginal standard errors, so their positivity follows from its positive diagonal variances; this does not exclude target cases. No independence is assumed. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.covariance_abs_bound",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L620",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Specializing a to the precision weights establishes the target bound, since those weights are positive. Attainment is supplied separately by R246-01.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.pooled_variance_upper_bound",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L643",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "This is the sufficient alignment equality case of (v), not an existence assertion or an equality-only-if characterization. L2 for Z is implied by the ordinary random-variable mean and unit-variance assumptions of the target. The added signed and empty cases do not exclude any target case.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.pooled_rank_one_equality",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L667",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Nonempty pools are the within-pool H1–H2 setting and are also assured by the historical 79/31 pools. Positivity and L² for the ambient family are H1; taking the ambient type to be the union avoids requiring any extra observations. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.pooled_difference_covariance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L708",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Auxiliary arm-mean fact already used inside the existing independent-arm shared-control theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.arm_sample_mean_memLp",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L743",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Auxiliary arm-mean fact already used inside the existing independent-arm shared-control theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.arm_sample_mean_variance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L759",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Auxiliary arm-mean fact already used inside the existing independent-arm shared-control theorem.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.arm_sample_means_uncorrelated",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L786",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The independent-arm theorem already includes the correlation formula.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.shared_control_correlation_from_moments",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L802",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "Every asserted variance, covariance and correlation formula is exported, with no equality-of-means assumption. Positive common variance and positive sizes make both contrast standard deviations nonzero. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.independent_arm_shared_control",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L825",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The covariance parameters must be realized by the actual L2 family D. The row claims a conditional block identity, not historical validation of the three-arm block or its numerical covariances.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.three_contrast_block",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L888",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "The finite-population shared-control theorem already includes the individual-effect expansion.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.population_effect_covariance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L967",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
      "conditional_on": [],
      "credit_reason": "This row proves the corrected target (viii), not an assertion of universal covariance positivity or numerical covariances of the historical experiments. Three positive arm sizes imply N≥3, so the explicit N≥2 bound and natural subtraction N−1 introduce no edge case. The claim-stage verdict is unchanged after reading the record.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.Wave1.finite_population_shared_control_proof",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/Wave1/Gap/GLP041P042P045P042B1.lean#L409",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     }
    ],
    "formalization": {
     "claim_revision_id": "catalog:be16486fc768f133541c928da8104c6c0852b5088d0950cc4a6e0750269c77d1",
     "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P042-T2",
     "decision": "restricted",
     "release_id": "sap-2026-11-08-assured-release",
     "target_id": "P042-T2"
    },
    "html": {
     "body": "<p>pooling method and mail comparison on p. 31, shared control on p. 36. Fixed-coefficient pooling has the stated mean, full covariance variance, precision-weight identities, shared-control formulas, bounds, signed-pool comparison and randomization covariance.</p>",
     "headline": "The pooled 95% confidence intervals of pp. 31–33 and the statement on p. 31 that the two types of mailings ‘have significantly different average effects’, as repaired by Sol: the variance of the precision-weighted average with the covariance between the pooled estimates included",
     "location": "p. 31 (PDF p. 7), paragraphs beginning ‘Meta-analysis is a statistical method’ and ‘We begin our discussion with direct mail’; the nine pooled intervals on pp. 31–33 (PDF pp. 7–9); p. 36 (PDF p. 12) for the shared control"
    },
    "id": "P042-T2",
    "text": {
     "body": "pooling method and mail comparison on p. 31, shared control on p. 36. Fixed-coefficient pooling has the stated mean, full covariance variance, precision-weight identities, shared-control formulas, bounds, signed-pool comparison and randomization covariance.",
     "headline": "The pooled 95% confidence intervals of pp. 31–33 and the statement on p. 31 that the two types of mailings ‘have significantly different average effects’, as repaired by Sol: the variance of the precision-weighted average with the covariance between the pooled estimates included",
     "location": "p. 31 (PDF p. 7), paragraphs beginning ‘Meta-analysis is a statistical method’ and ‘We begin our discussion with direct mail’; the nine pooled intervals on pp. 31–33 (PDF pp. 7–9); p. 36 (PDF p. 12) for the shared control"
    },
    "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T2"
   },
   {
    "declarations": [
     {
      "claim_revision_id": "catalog:c848ac9750779333022c09d1386b329a661ba67cca08a288801ea9af91bce9d4",
      "conditional_on": [],
      "credit_reason": "roboFixedVariance is the nominal scalar 1/W by definition; this row computes it and does not state a stochastic variance or coverage theorem. That is the numerical component specified by T3(i).",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_fixed_exact",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L365",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:c848ac9750779333022c09d1386b329a661ba67cca08a288801ea9af91bce9d4",
      "conditional_on": [],
      "credit_reason": "This is the numerical countercalculation for the declared fixed-effects reading, not a claim that no weighted-average method could produce 1.720.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_fixed_counterexample",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L376",
      "status": "counterexample_proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:c848ac9750779333022c09d1386b329a661ba67cca08a288801ea9af91bce9d4",
      "conditional_on": [],
      "credit_reason": "The standard k=2 DerSimonian–Laird formulas are expanded definitions, not assumptions of the reported fractions. Their numerical agreement with the printed figures establishes reproduction, not historical intent.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_random_exact",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L384",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:c848ac9750779333022c09d1386b329a661ba67cca08a288801ea9af91bce9d4",
      "conditional_on": [],
      "credit_reason": "This reproduces the target’s alternative-method point estimate, without attributing that method to the authors.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_random_rounding",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L403",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:c848ac9750779333022c09d1386b329a661ba67cca08a288801ea9af91bce9d4",
      "conditional_on": [],
      "credit_reason": "The row covers exactly the target’s every-z endpoint inequalities. Identifying the standard normal 0.975 quantile as one of these z values is a separate fact, not exported here.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_fixed_endpoint_counterexample",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L441",
      "status": "counterexample_proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:c848ac9750779333022c09d1386b329a661ba67cca08a288801ea9af91bce9d4",
      "conditional_on": [],
      "credit_reason": "The row is the endpoint calculation in T3(ii), for every z in the target’s stated range; neither historical method choice nor normal-quantile membership is concluded.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_random_interval",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L472",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     }
    ],
    "formalization": {
     "claim_revision_id": "catalog:c848ac9750779333022c09d1386b329a661ba67cca08a288801ea9af91bce9d4",
     "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P042-T3",
     "decision": "whole",
     "release_id": "sap-2026-11-08-assured-release",
     "target_id": "P042-T3"
    },
    "html": {
     "body": "<p>Note 5 on p. 40 (PDF p. 16), with the declared pooling method on p. 31. On the two stipulated deposited recorded-call inputs, all displayed exact fixed-effects and DerSimonian–Laird calculations hold. The fixed-effects estimate and interval do not round to Note 5&#x27;s figures, whereas the DerSimonian–Laird ones do. The endpoint statements hold throughout [1.9599, 1.96] and at the actual standard normal 0.975 quantile. The stated square-root-free comparisons and endpoint-check monotonicity hold as well. The usual measure-theoretic conditions are assumed. The inputs are stipulated exact rationals; agreement with a calculation makes no assertion about historical intent.</p>",
     "headline": "Note 5, last sentence (‘a weighted average effect of 1.720 percentage points, with a 95% CI = (0.306, 3.135)’), read with the ‘fixed effects weighting scheme’ declared on p. 31; the calculation is the validator&#x27;s, on the two deposited estimates, and is not in the paper",
     "location": "p. 40 (PDF p. 16), Note 5; the method paragraph is on p. 31 (PDF p. 7) and the note is attached to p. 32 (PDF p. 8)"
    },
    "id": "P042-T3",
    "text": {
     "body": "Note 5 on p. 40 (PDF p. 16), with the declared pooling method on p. 31. On the two stipulated deposited recorded-call inputs, all displayed exact fixed-effects and DerSimonian–Laird calculations hold. The fixed-effects estimate and interval do not round to Note 5's figures, whereas the DerSimonian–Laird ones do. The endpoint statements hold throughout [1.9599, 1.96] and at the actual standard normal 0.975 quantile. The stated square-root-free comparisons and endpoint-check monotonicity hold as well. The usual measure-theoretic conditions are assumed. The inputs are stipulated exact rationals; agreement with a calculation makes no assertion about historical intent.",
     "headline": "Note 5, last sentence (‘a weighted average effect of 1.720 percentage points, with a 95% CI = (0.306, 3.135)’), read with the ‘fixed effects weighting scheme’ declared on p. 31; the calculation is the validator's, on the two deposited estimates, and is not in the paper",
     "location": "p. 40 (PDF p. 16), Note 5; the method paragraph is on p. 31 (PDF p. 7) and the note is attached to p. 32 (PDF p. 8)"
    },
    "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T3"
   },
   {
    "declarations": [
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "The half-open rounding bin is the convention explicitly specified in T4(i); the actual value is strictly inside it.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_fixed_rounding",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L397",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "The existing fixed- and random-effects endpoint theorems already include the square-root bounds.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.sqrt_enclosure",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L410",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "The target explicitly asks for these endpoint bins for every z in this same interval. The statement does not identify a normal quantile or assert exact frequentist coverage.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_fixed_interval",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L419",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "This proves the positivity of the corrected numerical lower endpoint throughout the range in T4(iii). It does not require or establish historical independence.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_fixed_positive",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L459",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "The covered component is the exact scalar calculation in T4; the random estimators, their stipulated zero covariance, and a stochastic variance identity are not stated in this declaration.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_fixed_exact",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L365",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "This is the numerical part of the alternative repair expressly included in T4(iv), rather than evidence of the method historically used.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_random_exact",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L384",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "The row_says label mentions T3(ii), but the same alternative calculation is explicitly incorporated by T4(iv), so this second credit has a mathematical component to cover.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_random_rounding",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L403",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
      "conditional_on": [],
      "credit_reason": "This is the alternative repair’s every-z endpoint calculation. It proves all the bins requested, with no normal-quantile membership or historical attribution.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.robo_random_interval",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L472",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     }
    ],
    "formalization": {
     "claim_revision_id": "catalog:5fa416c97ef1b71c15c25fab42bd811e3690519c466379c9ef6b70ffd385665a",
     "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P042-T4",
     "decision": "restricted",
     "release_id": "sap-2026-11-08-assured-release",
     "target_id": "P042-T4"
    },
    "html": {
     "body": "<p>Note 5 on p. 40, corrected using the pooling method on p. 31. The fixed-effects calculation rounds to 1.866 and (0.963, 2.768), stays above zero, and has variance 1/W under the stipulated zero covariance; the alternative method reproduces Note 5.</p>",
     "headline": "Note 5, last sentence, corrected: the fixed-effects weighted average of the two social-pressure recorded-call estimates and its 95% interval",
     "location": "p. 40 (PDF p. 16), Note 5"
    },
    "id": "P042-T4",
    "text": {
     "body": "Note 5 on p. 40, corrected using the pooling method on p. 31. The fixed-effects calculation rounds to 1.866 and (0.963, 2.768), stays above zero, and has variance 1/W under the stipulated zero covariance; the alternative method reproduces Note 5.",
     "headline": "Note 5, last sentence, corrected: the fixed-effects weighted average of the two social-pressure recorded-call estimates and its 95% interval",
     "location": "p. 40 (PDF p. 16), Note 5"
    },
    "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T4"
   },
   {
    "declarations": [
     {
      "claim_revision_id": "catalog:b0aa8a573a8085a7532340a9295a88003f46c431c6b9c341594e69013751c4f0",
      "conditional_on": [],
      "credit_reason": "This exactly negates the unrestricted pooling identity explicitly adopted in the target’s formal reading. It does not individually disprove any of the nine printed numerical intervals; the paper itself prints no such identity. A finite probability space suffices to refute the universal claim.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.Wave1.printed_pooling_variance_refuted",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/Wave1/PrintedPoolingVariance.lean#L17",
      "status": "counterexample_proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:b0aa8a573a8085a7532340a9295a88003f46c431c6b9c341594e69013751c4f0",
      "conditional_on": [],
      "credit_reason": "This exactly negates the unrestricted comparison identity explicitly adopted in the target’s formal reading. It is not a finding about the empirical significance of the 79/31 treatment comparison; the paper itself prints no such identity. Restricting type universes to Type 0 loses no refutation because a finite counterexample suffices.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.Wave1.printed_comparison_variance_refuted",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/Wave1/PrintedComparisonVariance.lean#L17",
      "status": "counterexample_proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     }
    ],
    "formalization": {
     "claim_revision_id": "catalog:b0aa8a573a8085a7532340a9295a88003f46c431c6b9c341594e69013751c4f0",
     "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P042-T5",
     "decision": "restricted",
     "release_id": "sap-2026-11-08-assured-release",
     "target_id": "P042-T5"
    },
    "html": {
     "body": "<p>pooling and mail comparison on p. 31, nine pooled intervals on pp. 31–33. The audit&#x27;s unconditional diagonal-variance readings for a pooled mean and a difference of pools fail.</p>",
     "headline": "The fixed-effects pooling described on p. 31 and the nine pooled estimates with 95% confidence intervals of pp. 31–33, as printed",
     "location": "p. 31 (PDF p. 7); p. 32 (PDF p. 8); p. 33 (PDF p. 9)"
    },
    "id": "P042-T5",
    "text": {
     "body": "pooling and mail comparison on p. 31, nine pooled intervals on pp. 31–33. The audit's unconditional diagonal-variance readings for a pooled mean and a difference of pools fail.",
     "headline": "The fixed-effects pooling described on p. 31 and the nine pooled estimates with 95% confidence intervals of pp. 31–33, as printed",
     "location": "p. 31 (PDF p. 7); p. 32 (PDF p. 8); p. 33 (PDF p. 9)"
    },
    "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T5"
   },
   {
    "declarations": [
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "Auxiliary finite-expectation identity already used inside the existing balance and CACE theorems.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.finite_expectation_sub",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1372",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "Auxiliary finite-expectation identity already used inside the existing balance and CACE theorems.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.finite_expectation_sum",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1377",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "Auxiliary finite-expectation identity already used inside the existing balance and CACE theorems.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.finite_expectation_mul",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1385",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "Auxiliary finite-expectation identity already used inside the existing balance and CACE theorems.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.finite_expectation_div",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1391",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "Auxiliary finite-expectation identity already used inside the existing balance and CACE theorems.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.finite_expectation_const",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1397",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "The existing balance theorem already uses the uniform-design inclusion calculation.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.complete_inclusion_probability",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1404",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "The existing balance theorem already includes both group-mean identities.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.complete_group_mean",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1431",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "{'source': 'dispute', 'reason': \"T6(a)'s formal reading requires potential-outcome mean balance and unbiasedness of the observed treatment-control difference under uniform complete randomization with nonempty arms. P042/P042_Turnout.lean:1527 states both conclusions for every fixed real potential-outcome array indexed by subjects and Bool, with exactly N>=2 and 1<=m<=N-1. The assignment law is the uniform average of all m-subsets, as defined at P042/P042_Turnout.lean:1404 and P042/P042_Turnout.lean:959, and assignedOutcome at P042/P042_Turnout.lean:1418 selects the potential outcome corresponding to assignment. Thus the substantive design restriction is already in the supplied target, rather than an additional Lean hypothesis, and the row accurately discloses it. Although the printed prose discusses random assignment more broadly, this labelled formal special case makes no assertion of exhaustive coverage, so neither statement repair nor claim-text correction is required for this row's credit.\", 'defensible_statement': \"For N>=2 subjects with fixed real potential outcomes, assign exactly m subjects uniformly at random to treatment, with 1<=m<=N-1. Each treatment or control mean of either potential-outcome vector has expectation equal to that vector's population mean, and the expected observed treatment-control difference equals the population average treatment effect.\"}",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.complete_randomization_balance",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1477",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "Covers T6(b1), which is a finite algebraic identification identity independent of an assignment law. The one-sided own-contact outcome model and positive complier count are explicit in the target's own formal reading; N>0 follows already from positive complierCount. Arbitrary real outcomes include all binary-turnout instances.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.itt_complier_identity",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1546",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "{'source': 'dispute', 'reason': \"The credited components T6(b2)-(b3) expressly use 'the complete randomization of (a)', which specifies equally likely m-subsets and nonempty treatment and control arms. P042/P042_Turnout.lean:1621 uses exactly that law through completeAssignments at P042/P042_Turnout.lean:1404 and uniform finiteExpectation at P042/P042_Turnout.lean:959, and concludes both required expectation identities and the ratio equal to CACE. The Boolean contact indicator, positive complier count, contactedOutcome at P042/P042_Turnout.lean:1585 and assignedContactRate at P042/P042_Turnout.lean:1590 implement the target's one-sided contact, exclusion and no-interference model, while complierEffect at P042/P042_Turnout.lean:1573 is the stated average contact effect among compliers. Other random assignment laws lie outside this expressly restricted formal target; their omission does not make its complete-randomization assertion incorrect or misquote the separately identified paper prose. The target and row both distinguish the ratio of expectations from unbiasedness of the random ratio estimator, so there is no missing unbiasedness conclusion or description overpromise.\", 'defensible_statement': 'For a fixed population of N>=2 subjects, assign exactly m subjects uniformly at random to treatment, with 1<=m<=N-1, fixed real contact potential outcomes, one-sided contact, exclusion and no interference, and at least one complier. Then the expected observed turnout difference equals ITT, the expected assigned-treatment contact rate equals the population complier share, and CACE equals the ratio of these two expectations.'}",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.cace_randomization_identity",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1562",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     },
     {
      "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
      "conditional_on": [],
      "credit_reason": "Covers only T6(b4), the printed numerical illustration; it does not certify the underlying empirical figures.",
      "credit_status": "ok",
      "fidelity_status_baseline": "ok",
      "lean_name": "AronowAudit.P042.michelson_cace_example",
      "manifest_use": "used",
      "named_axioms": [],
      "says": null,
      "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P042/P042_Turnout.lean#L1594",
      "status": "proved",
      "support_scope": "historical declaration",
      "used_in_solution": true
     }
    ],
    "formalization": {
     "claim_revision_id": "catalog:06145fd1788e3323a6da343a33e49a7a673665997ce1b86cba30c726a89438f2",
     "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P042-T6",
     "decision": "whole",
     "release_id": "sap-2026-11-08-assured-release",
     "target_id": "P042-T6"
    },
    "html": {
     "body": "<p>p. 30 (PDF p. 6), Note 2 on pp. 39–40 (PDF pp. 15–16), and contact/CACE and the Michelson illustration on p. 32 (PDF p. 8). Random assignment independent of subjects&#x27; potential outcomes balances the two arms in expectation. Uniform complete randomization gives the finite potential-outcome mean identities and an unbiased observed assignment contrast; a uniformly sampled subject&#x27;s treatment is independent of their fixed attributes. Under one-sided contact, exclusion and no interference, ITT equals Complier share times CACE, the expected treated contact fraction is that positive share, and the ratio of the expected turnout contrast to the expected contact fraction equals CACE. The same population identities hold on arbitrary probability spaces under independent assignment. The realized estimator is the turnout contrast divided by its positive realized contact fraction. Never-taker effects are not identified, even with binary turnout. The illustration is (9.4 - 7.0)/0.50 = 4.8 percentage points. The usual measure-theoretic conditions are assumed. No unbiasedness or consistency of the realized ratio is asserted.</p>",
     "headline": "Unnumbered statements: under random assignment the potential outcomes of the treatment and control groups are the same in expectation (p. 30 with Note 2), and the contact-rate formula for the Complier average causal effect (p. 32)",
     "location": "p. 30 (PDF p. 6) with Note 2 on pp. 39–40 (PDF pp. 15–16); p. 32 (PDF p. 8)"
    },
    "id": "P042-T6",
    "text": {
     "body": "p. 30 (PDF p. 6), Note 2 on pp. 39–40 (PDF pp. 15–16), and contact/CACE and the Michelson illustration on p. 32 (PDF p. 8). Random assignment independent of subjects' potential outcomes balances the two arms in expectation. Uniform complete randomization gives the finite potential-outcome mean identities and an unbiased observed assignment contrast; a uniformly sampled subject's treatment is independent of their fixed attributes. Under one-sided contact, exclusion and no interference, ITT equals Complier share times CACE, the expected treated contact fraction is that positive share, and the ratio of the expected turnout contrast to the expected contact fraction equals CACE. The same population identities hold on arbitrary probability spaces under independent assignment. The realized estimator is the turnout contrast divided by its positive realized contact fraction. Never-taker effects are not identified, even with binary turnout. The illustration is (9.4 - 7.0)/0.50 = 4.8 percentage points. The usual measure-theoretic conditions are assumed. No unbiasedness or consistency of the realized ratio is asserted.",
     "headline": "Unnumbered statements: under random assignment the potential outcomes of the treatment and control groups are the same in expectation (p. 30 with Note 2), and the contact-rate formula for the Complier average causal effect (p. 32)",
     "location": "p. 30 (PDF p. 6) with Note 2 on pp. 39–40 (PDF pp. 15–16); p. 32 (PDF p. 8)"
    },
    "url": "https://pmaronow.github.io/llm-audit/works/P042/#P042-T6"
   }
  ]
 },
 "missing_sources": [],
 "reproduction": {
  "agreement": "agrees_in_part",
  "status": "partial",
  "targets": [
   {
    "agreement": "to_rounding",
    "location": "p. 31 (PDF p. 7), final paragraph",
    "method": "I translated the Stata pooling into Python and checked it independently in R. The treatment counts, pooled estimates and confidence limits agree with the printed values to rounding.",
    "status": "reproduced",
    "target": "In-text results: conventional direct mail"
   },
   {
    "agreement": "to_rounding",
    "location": "p. 32 (PDF p. 8), first paragraph; interpretation pp. 36-37 (PDF pp. 12-13)",
    "method": "I translated the Stata pooling into Python and checked it independently in R. The treatment counts, pooled estimates and confidence limits agree with the printed values to rounding.",
    "status": "reproduced",
    "target": "In-text results: social-pressure and gratitude mail"
   },
   {
    "agreement": "to_rounding",
    "location": "p. 32 (PDF p. 8), paragraph beginning 'Analyzing the effects'",
    "method": "I translated the Stata pooling into Python and checked it independently in R. The treatment counts, pooled estimates and confidence limits agree with the printed values to rounding.",
    "status": "reproduced",
    "target": "In-text results: canvassing"
   },
   {
    "agreement": "agrees_in_part",
    "location": "p. 32 (PDF p. 8), penultimate paragraph; note 5, p. 40 (PDF p. 16)",
    "method": "I translated the Stata pooling into Python and checked it independently in R. The eleven-treatment estimate, interval and count agree to rounding. Note 5 differs under the stated fixed-effects method; its numbers agree to rounding under a separate DerSimonian–Laird random-effects calculation.",
    "status": "reproduced",
    "target": "In-text results: prerecorded phone calls"
   },
   {
    "agreement": "to_rounding",
    "location": "pp. 32-33 (PDF pp. 8-9), paragraph spanning the page break",
    "method": "I translated the Stata pooling into Python and checked it independently in R. The treatment counts, pooled estimates and confidence limits agree with the printed values to rounding.",
    "status": "reproduced",
    "target": "In-text results: commercial live phone calls"
   },
   {
    "agreement": "to_rounding",
    "location": "p. 33 (PDF p. 9), first paragraph",
    "method": "I translated the Stata pooling into Python and checked it independently in R. The treatment counts, pooled estimates and confidence limits agree with the printed values to rounding.",
    "status": "reproduced",
    "target": "In-text results: volunteer live phone calls"
   },
   {
    "agreement": "not_compared",
    "location": "p. 33 (PDF p. 9), final paragraph",
    "method": "The replication dataset contains no entry for the Dale and Strauss (2009) text-message experiment or the inputs needed to reconstruct it. I did not compare the cited CACE of 4.1.",
    "status": "blocked",
    "target": "Cited result: text-message reminders"
   },
   {
    "agreement": "not_compared",
    "location": "p. 37 (PDF p. 13), second paragraph under 'Spatial and Temporal Spillovers'",
    "method": "I compared the 60% statement with the abstract stored in the dataset, where it agrees. The underlying direct and housemate effects were not compared: the supplied data do not contain the inputs needed to recalculate their ratio.",
    "status": "blocked",
    "target": "Cited result: household spillover"
   }
  ]
 },
 "summary": "The article reviews field experiments on voter turnout and pools them by tactic in fixed-effects meta-analyses. It reports effects of 0.162 percentage points per mailing for conventional mail, 2.850 for social-pressure mail, and 2.536 among Compliers for canvassing. 11 entries are classified as errors (3 Moderate, 8 Minor), and 2 are clarity notes; 5 possible errors were not confirmed and 3 are undetermined. The main finding is \\fid{P042-E001}: the pooled 95\\% intervals treat contrasts that share a control group as independent. Under an equal-variance shared-control model, the standard error for social-pressure mail is 21\\% larger.",
 "what_was_checked": "I checked the following by recalculation from the replication files or against the pages; they held up subject to the corrections cited.\n\\begin{itemize}\n\\item The nine pooled estimates of pp.~31--33, their intervals and their counts reproduce to three decimals from the replication dataset by fixed-effects pooling with inverse-variance weights. Estimates that enter six of the pools are corrected in \\fid{P042-N003}, \\fid{P042-N005}, \\fid{P042-N006}, \\fid{P042-N010} and \\fid{P042-N011}, and the intervals are qualified in \\fid{P042-E001}.\n\\item The printed counts add up: 147 mail treatments are 110 conventional, 29 social-pressure and 8 gratitude treatments; 110 are 79 nonadvocacy and 31 advocacy treatments; 13 recorded-call rows are 11 and 2.\n\\item The difference between nonadvocacy and advocacy mail on p.~31 is 0.407 with $z=2.54$ when the contrasts are treated as independent (\\fid{P042-E001}, \\fid{P042-N003}). The difference between volunteer and commercial calls on p.~33 is 0.956 with $z=2.35$ for the estimates as stored (\\fid{P042-N011}).\n\\item The arithmetic of the Michelson (2006) example on p.~32: $(9.4-7.0)/0.50=4.8$.\n\\item The paragraph on Compliers and Never-takers on p.~32 gives the CACE formula under random assignment, one-sided receipt, exclusion, stable outcomes for the units analysed and a positive contact rate. Note~2 defines random assignment.\n\\item The statements about single studies that could be compared with rows of the dataset agree with them: the 1998 New Haven estimates for commercial calls and mail, the two message arms of Michelson, the two frames of Arceneaux and Nickerson (2010), the three timings of Panagopoulos (2011b) in Note~6, and the results of Ramirez (2005) for recorded calls and mail. The Colorado B estimate of Ramirez (2005) matches Table~4 of that article.\n\\item The pooling commands follow the dataset's category codes, with inclusion errors corrected in \\fid{P042-N010} and \\fid{P042-N011}. The estimate in Note~5 is not produced by the program (\\fid{P042-E002}).\n\\item Every citation in the text has an entry in the reference list except the one in \\fid{P042-E005}; \\fid{P042-A002} and \\fid{P042-E004} concern an ambiguous citation and the asterisks of the list. The note markers 1 to 6 match the Notes.\n\\item The variance correction of \\fid{P042-E001} alters no estimate, and the ordering of the nine pooled estimates is unchanged by it.\n\\item The discussions of factorial designs, household spillovers and persistence were read for their reasoning. The statement that about 60\\% of the canvassing effect reaches a housemate agrees with the abstract stored in the dataset, and ``just eight years later'' on p.~28 is 2000 to 2008.\n\\item I checked the publisher's indexed text of the official erratum (\\emph{Journal of Elections, Public Opinion and Parties} 23(1), p.~i; \\href{https://doi.org/10.1080/17457289.2012.757871}{doi:10.1080/17457289.2012.757871}). It corrects the order of the authors only.\n\\end{itemize}\nI did not check the following.\n\\begin{itemize}\n\\item The studies that the review summarizes, about 200 references, apart from a few consulted for particular points: Gerber, Green and Green (2003), Ramirez (2005) through indexed table text, Mann (2010) in the author's posted manuscript, and Nickerson, Friedrichs and King (2006). Statements about other studies were compared only with the dataset, where it holds them.\n\\item The exclusion of compound treatments (Note~3) and of follow-up calls (p.~33) from the full set of studies, since the authors' unfiltered spreadsheet is not deposited.\n\\item The numerical statements of the sections on synergy (pp.~34--35) and on spillovers and persistence (pp.~37--38), which were read for consistency and citations only.\n\\item The Stata program itself was not run; the pooling was reconstructed independently.\n\\end{itemize}\nI could not settle four points for want of sources: whether the St.~Louis canvassing estimate of Gillespie (2005) is entered correctly, whether an unpublished canvassing estimate from Michigan belongs to the experiment of Nickerson, Friedrichs and King (2006), whether the increase in the Michelson (2006) example is statistically significant in the source, and whether Doherty et al. (2006) report a turnout outcome. The list above is not a statement that the rest of the article is correct.",
 "work": {
  "arxiv_id": null,
  "audit_complete": true,
  "authors": [
   "Donald P. Green",
   "Mary C. McGrath",
   "P. M. Aronow"
  ],
  "category": "Articles - Empirics",
  "citation": "Donald P. Green, Mary C. McGrath, and P. M. Aronow (2013). Field Experiments and the Study of Voter Turnout. Journal of Elections, Public Opinion and Parties 23(1):27–48.",
  "doi": "10.1080/17457289.2012.728223",
  "edition": {
   "pages_in_audited_file": 24,
   "sha256": "9503eb9d67c6734335c975221cfe6da4049d963e5ee5875f638668ec4cf432d6",
   "source_url": "https://doi.org/10.1080/17457289.2012.728223",
   "venue": "Journal of Elections, Public Opinion and Parties",
   "year": "2013"
  },
  "id": "P042",
  "later_materials": null,
  "title": "Field Experiments and the Study of Voter Turnout",
  "type": "Journal article",
  "url": "https://pmaronow.github.io/llm-audit/works/P042/",
  "venue": "Journal of Elections, Public Opinion and Parties",
  "year": "2013"
 }
}