{ "claims": [ { "claim": "For a randomized experiment with interference of arbitrary but known form (a known design plus a correctly specified exposure mapping), the inverse-probability-weighted (Horvitz-Thompson) estimators of exposure-specific totals, means and average unit-level causal effects are unbiased, with the stated randomization variance and covariance.", "claim_id": "P013-H1", "related_corrections": [ "P013-E003", "P013-E018", "P013-N012" ], "tied_to": "Lemma 4.1; Proposition 4.2; equations (1)-(6)" }, { "claim": "The proposed randomization-based variance estimators are conservative (non-negatively biased for the variance of the estimated effect), including when exposures are clustered and some joint exposure probabilities are zero, and the covariance estimator is unbiased when the two exposures have identical potential outcomes.", "claim_id": "P013-H2", "related_corrections": [ "P013-E009", "P013-N001" ], "tied_to": "Lemma 5.1; Propositions 5.1-5.6; expressions (7)-(11); Appendix A.5 (Propositions A.1, A.2, Corollaries A.1, A.2)" }, { "claim": "Under bounded outcomes and bounded inverse marginal exposure probabilities and a vanishing fraction of pairwise exposure dependencies, the HT estimator of the average effect is consistent.", "claim_id": "P013-H3", "related_corrections": [ "P013-E012" ], "tied_to": "Proposition 6.1; Conditions 3-4; Appendix A.6" }, { "claim": "Under local dependence and a positive limiting scaled variance, the effect estimator is asymptotically normal and Wald intervals built from the conservative variance estimator cover at least at the nominal rate for large $N$.", "claim_id": "P013-H4", "related_corrections": [ "P013-E001", "P013-E002", "P013-E006", "P013-E007", "P013-E011", "P013-E012" ], "tied_to": "Proposition 6.2; Conditions 3, 5-6; Appendix A.7" }, { "claim": "Efficiency can be improved at little cost through covariate adjustment (difference and regression estimators) and Hajek ratio estimation; the difference estimator stays unbiased, and the bias of the Hajek estimator is bounded and vanishes asymptotically.", "claim_id": "P013-H5", "related_corrections": [ "P013-E004", "P013-E005" ], "tied_to": "Section 7; equations (12)-(15); bias bound on p. 1929; Table 1" }, { "claim": "In a naturalistic simulation on Add Health school networks, the HT, Hajek and WLS estimators show no perceivable bias and have conservative standard errors with coverage above nominal, while OLS and difference-in-sample-means are biased and undercover.", "claim_id": "P013-H6", "related_corrections": [ "P013-N004", "P013-E013", "P013-E016", "P013-E017", "P013-E018", "P013-E019", "P013-N006" ], "tied_to": "Table 1; Figures 1 and 3; Section 9; Appendix B" }, { "claim": "In the anti-conflict field experiment, direct exposure raises the probability of wearing a program wristband by about 30 percentage points and indirect (peer) exposure by about 15 to 16 points, while being in a program school without direct or indirect exposure has a small effect that is statistically indistinguishable from zero.", "claim_id": "P013-H7", "related_corrections": [ "P013-E008", "P013-E014", "P013-E013", "P013-N016", "P013-N020" ], "tied_to": "Table 2; Figure 2; Section 10" }, { "claim": "If the exposure mapping is misspecified, the HT estimator is unbiased for a design-weighted average of randomization potential outcomes (the Hudgens-Halloran individual average potential outcome), so the effect estimate is a difference of such averages.", "claim_id": "P013-H8", "related_corrections": [ "P013-N003" ], "tied_to": "Proposition 8.1; Corollary 8.1; Appendix A.8" }, { "claim": "When the set of assignments is too large to enumerate, exposure probabilities can be approximated by simulation with almost-sure convergence, and the relative bias of the resulting HT estimator is at most $(1-\\pi_0)^{R+1}$, which gives a rule for choosing the number of replicates $R$.", "claim_id": "P013-H9", "related_corrections": [ "P013-E003", "P013-E010", "P013-N012" ], "tied_to": "Proposition 3.1; Proposition 4.1; Appendix A.1, A.3" }, { "claim": "The field experiment had two levels of randomization: 28 of 56 schools were selected to host the program by block randomization; in every school 40 to 64 students were nonrandomly designated eligible, and in host schools half of the eligible students were block-randomized to participate. Peers are defined by an undirected graph built from first-wave nominations of up to 10 schoolmates, and school and student assignment induce five exposure conditions combining school, direct and indirect exposure.", "claim_id": "P013-H10", "related_corrections": [ "P013-E014", "P013-N015", "P013-N016" ], "tied_to": "Section 10, design paragraph and exposure mapping; Figure 2; abstract and introduction in general terms" }, { "claim": "The program has substantial direct and indirect effects on willingness to endorse anti-conflict norms by wearing a program wristband; because of the indirect effects, comparing participants with nonparticipants inside program schools would drastically underestimate the program's effect, and ignoring indirect effects might understate the program's cost-effectiveness.", "claim_id": "P013-H11", "related_corrections": [], "tied_to": "Section 10, first paragraph of p. 1937; Table 2; motivation in the abstract and introduction" } ], "date_published": "2026-10-25", "findings": [ { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of Proposition 6.2. A red box marks the words 'Given Conditions 3, 5, and 6'. The proposition continues: Wald-type intervals constructed as tau-hat-HT(d_k, d_l) plus or minus z_(1 minus alpha/2) times the square root of the estimated variance will tend to cover tau_HT(d_k, d_l) at least 100(1 minus alpha) percent of the time for large N.", "caption": "Proposition 6.2, p. 1926 (PDF p. 15). The marked hypotheses do not suffice for the conclusion.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E001.png" }, { "alt": "Crop of the middle of the proof of Proposition 6.2. Red boxes mark the sentence 'Note that all a_ij(D_i, D_j) are bounded above by some finite constant by Condition 3'. It is followed by a display bounding the variance of N times the estimated variance by N to the power minus 2 times a fourfold sum of covariances of the terms h_ij a_ij(D_i, D_j) and h_kl a_kl(D_k, D_l).", "caption": "Appendix A.7, p. 1943 (PDF p. 32). The marked sentence takes the terms $a_{ij}(D_i,D_j)$ of the variance estimator to be bounded by Condition 3, which bounds only the marginal exposure probabilities.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E001-b.png" }, { "alt": "Chart with the number of units N on a logarithmic horizontal axis and the coverage of the nominal 95 percent interval on the vertical axis, with ticks from 0.90 to 0.95. Three blue points show midpoints of exact coverage brackets in the counterexample: 0.9350 at N = 100, 0.9232 at N = 400 and 0.9063 at N = 2,000. A solid black horizontal line at 0.95 is labelled as the nominal level that Proposition 6.2 states as a lower bound for large N. A dashed blue horizontal line at 0.9104 is labelled as the limit of the coverage in the counterexample as N grows. All three points lie below the solid line.", "caption": "The counterexample for P013-E001: coverage of the nominal 95% interval of Proposition 6.2 for $N=100$, $400$ and $2{,}000$ units ($n=50$, $200$ and $1{,}000$ pairs, critical value $1.96$), with the nominal level (solid line) and the limit $0.9104$ of the coverage (dashed line). Each point is the midpoint of an exact bracket of width below $0.0002$. The bounds use binomial probabilities and need not approach the limit monotonically.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E001-coverage.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T1", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T1" }, { "id": "P013-T2", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T2" }, { "id": "P013-T18", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T18" } ], "headline": "Proposition 6.2 needs a further condition, that the positive joint exposure probabilities that enter the variance estimator of equation (11) are bounded away from zero; Conditions 3, 5 and 6 alone do not imply the stated coverage.", "id": "P013-E001", "location": "Proposition 6.2, p. 1926 (PDF p. 15); Condition 3, p. 1925 (PDF p. 14); Appendix A.7, p. 1943 (PDF p. 32)", "pdf_page": 15, "printed_page": "1926", "reaches": [ { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "result_needs_same_repair", "effect_text": "a result or code path there needs the same repair", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Moderate", "subject": "paper", "target_ids": [ "P013-T1", "P013-T18", "P013-T2" ], "text": { "as_printed": "Condition 3 (p.~1925): ``Potential outcomes and exposure probabilities are bounded so that, for all values $i$ and $d_k$, $|y_i(d_k)|\\le c_1<\\infty$ and $|1/\\pi_i(d_k)|\\le c_2<\\infty$.''\n\nProposition 6.2 (p.~1926): ``\\hl{Given Conditions 3, 5, and 6}, Wald-type intervals constructed as\n\\[\n\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)\\pm z_{1-\\alpha/2}\\sqrt{\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}\n\\]\nwill tend to cover $\\tau_{\\mathrm{HT}}(d_k,d_l)$ at least $100(1-\\alpha)\\%$ of the time for large $N$.''\n\nAppendix A.7 (p.~1943): ``Denote $a_{ij}(D_i,D_j)$ as the sum of the elements in $\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$ that incorporate observations $i$ and $j$. Note that all $a_{ij}(D_i,D_j)$ are bounded above by some finite constant \\hl{by Condition 3}:\n\\[\n\\begin{aligned}\n\\operatorname{Var}\\bigl[N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]\\bigr]\n&\\le N^{-2}\\operatorname{Var}\\Bigl[\\sum_{i=1}^{N}\\sum_{j=1}^{N}h_{ij}a_{ij}(D_i,D_j)\\Bigr]\\\\\n&=N^{-2}\\sum_{i=1}^{N}\\sum_{j=1}^{N}\\sum_{k=1}^{N}\\sum_{l=1}^{N}\\operatorname{Cov}\\bigl[h_{ij}a_{ij}(D_i,D_j),h_{kl}a_{kl}(D_k,D_l)\\bigr].\n\\end{aligned}\n\\]\n\\ldots\\ Thus, $\\operatorname{Var}[N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]]=O(N^{-2}\\times N)=O(N^{-1})$.''", "corrected": "Proposition 6.2. Let Conditions 3 and 6 hold, and let Condition 5 hold with $h_{ij}$ the indicator of a dependency graph, as corrected in \\fid{P013-E002}. \\hl{Suppose in addition that there exists a constant} $\\hl{b>0}$\\hl{, not depending on} $\\hl{N}$\\hl{, that bounds from below every positive joint exposure probability entering equations (7) and (10)}: for all $N$ and all $i\\ne j$, if $\\pi_{ij}(d_k)>0$ then $\\pi_{ij}(d_k)\\ge b$, and likewise for $\\pi_{ij}(d_l)$; and for all $N$ and all $i,j$, if $\\pi_{ij}(d_k,d_l)>0$ then $\\pi_{ij}(d_k,d_l)\\ge b$. Joint probabilities equal to zero remain permitted, with the corrections of Section 5. Write $\\widehat{V}_N=\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$ and $\\hl{\\widehat{V}_N^+=\\max\\{0,\\widehat{V}_N\\}}$ to define the interval when the variance estimate is negative. Then the Wald-type intervals constructed as\n\\[\n\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)\\pm z_{1-\\alpha/2}\\sqrt{\\hl{\\widehat{V}_N^+}}\n\\]\nsatisfy\n\\[\n\\hl{\\liminf_{N\\to\\infty}}\\,P\\Bigl\\{\\bigl|\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)-\\tau(d_k,d_l)\\bigr|\\le z_{1-\\alpha/2}\\sqrt{\\hl{\\widehat{V}_N^+}}\\Bigr\\}\\ge1-\\alpha .\n\\]\nThe target is written $\\tau(d_k,d_l)$, as in \\fid{P013-E012}.\n\nThe added condition may be replaced by the weaker one that, for every $\\delta>0$,\n\\[\nP\\bigl\\{N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]0$, and let Condition 5 hold for a symmetric dependency graph with row sums at most $m$. I add every self-edge, writing $\\widetilde h_{ij}=1$ when $i=j$ or $h_{ij}=1$. This preserves the dependency condition and bounds every row sum by $m+1$. I write $N^2\\widehat{V}_N=\\sum_{i,j}B_{ij}(D_i,D_j)$, where $B_{ij}$ collects the terms that involve units $i$ and $j$. A term with positive joint probability is bounded in absolute value by $(1+1/b)c_1^2c_2^2$, and the corrections for zero joint probabilities divide only by marginal probabilities. Then there exists a constant $C$, which does not depend on the population size, such that $|B_{ij}|\\le C$ for every pair of units. If $\\widetilde h_{ij}=0$, then the exposures of $i$ and $j$ are independent, the joint probabilities are products of marginal probabilities, and $B_{ij}=0$. Then at most $N(m+1)$ of the $B_{ij}$ are nonzero. Two of them can be correlated only if a unit of one is adjacent under $\\widetilde h$ to a unit of the other, which leaves at most $4N(m+1)^3$ nonzero covariances, each at most $4C^2$ in absolute value. I obtain\n\\[\n\\operatorname{Var}[N\\widehat{V}_N]\\le N^{-2}\\,(4N(m+1)^3)(4C^2)=16C^2(m+1)^3/N\\to0.\n\\]\nLet $c_N=N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]\\to c>0$. Proposition 5.6 implies that $\\mathrm{E}[N\\widehat{V}_N]\\ge c_N$. By Chebyshev's inequality, I have $P\\{N\\widehat{V}_N0$. The statistic $T_N=\\sqrt{N}(\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)-\\tau(d_k,d_l))/\\sqrt{c_N}$ is asymptotically standard normal by the central limit theorem that Appendix A.7 invokes, which involves no joint probability. For $0<\\delta0$.\n\n\\hl{Convergence} of $N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$ to a non-negative constant \\hl{is generally ensured by Conditions 3 and 5}, sufficient for root-$n$ consistency of $\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)$. Condition 6 is a mild regularity condition that ensures that this constant is positive, and rules out degenerate cases (e.g., all outcomes are zero).''", "corrected": "Conditions 3 and 5 imply \\hl{that} $\\hl{N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]=O(1)}$, sufficient for root-$N$ consistency of $\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)$. Condition 6 \\hl{additionally assumes that this sequence converges to a positive constant}, and rules out degenerate cases (e.g., all outcomes are zero).", "intended_meaning": "The paragraph explains why a positive limiting variance is assumed. With Condition 5 read as a dependency graph of bounded degree, Conditions 3 and 5 leave at most $(m+1)N$ nonzero covariance terms, each bounded, so that $N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]\\le (m+1)(2c_1c_2)^2$. The scaled variance is bounded, which suffices for the root-$N$ rate, but it need not converge. For example, with $y_i(d_k)=1$, $y_i(d_l)=0$ and independent assignments with exposure probability $p_N=1/2$ for even $N$ and $1/4$ for odd $N$, I have $N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]=(1-p_N)/p_N$, which equals $1$ and $3$ along the two subsequences. Condition 6 itself assumes the limit and its positivity, and Proposition 6.2 lists Condition 6 among its hypotheses. No assumption is missing. The arrow $\\xrightarrow{p}$ in Condition 6 is applied to a deterministic sequence and denotes ordinary convergence." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E006", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of the six lines that end the proof of Proposition 6.2. The text says that under Conditions 3, 5, and 6, by Chen and Shao (2004), Theorem 2.7, the estimator minus its target, divided by the square root of its variance, is asymptotically N(0, 1), while the square root of the ratio of the variance to the estimated variance converges in probability to a quantity in (0, 1]. It continues: By Slutsky's theorem, t is asymptotically normal and Wald-type confidence intervals constructed as the estimator plus or minus z times the square root of the estimated variance will tend to cover the target at least 100(1 - alpha) percent of the time as N tends to infinity. Red outlines surround the words converges in probability to a quantity in (0, 1] and the words t is asymptotically normal.", "caption": "Appendix A.7, last paragraph, p. 1943 (PDF p. 32). The outlines mark the claim that the variance ratio converges in probability to a quantity in $(0,1]$ and the claim that $t$ is asymptotically normal.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E007.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T7", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T7" }, { "id": "P013-T8", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T8" } ], "headline": "Even when the conservative variance estimator concentrates around its expectation, it need not converge, and the variance ratio and the studentized statistic $t$ need not have limits; coverage of at least $1 - \\alpha$ in the limit inferior follows from a one-sided bound once concentration and the true-variance central limit theorem hold.", "id": "P013-E007", "location": "Appendix A.7, proof of Proposition 6.2, first and last paragraphs, p. 1943 (PDF p. 32)", "pdf_page": 32, "printed_page": "1943", "reaches": [ { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P013-T7", "P013-T8" ], "text": { "as_printed": "\\emph{First paragraph of A.7.} By Proposition 5.6, $\\mathrm{E}[N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]]\\ge N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$. Thus, by Chebyshev's inequality, $\\operatorname{Var}[N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]]\\xrightarrow{p}0$ is sufficient to establish \\hl{convergence} of $N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$ \\hl{to a value} greater than or equal to $N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$, which is itself nonzero by Condition 6.\n\n\\emph{Last paragraph of A.7.} Define\n\\[\nt=\\frac{\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)-\\tau_{\\mathrm{HT}}(d_k,d_l)}{\\sqrt{\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}}=\\frac{\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)-\\tau_{\\mathrm{HT}}(d_k,d_l)}{\\sqrt{\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}}\\left(\\frac{\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}{\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}\\right)^{1/2}.\n\\]\nUnder Conditions 3, 5, and 6, then, by Chen and Shao [(2004), Theorem 2.7], $\\frac{\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)-\\tau_{\\mathrm{HT}}(d_k,d_l)}{\\sqrt{\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}}$ is asymptotically $\\mathrm{N}(0,1)$, while $(\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]/\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)])^{1/2}$ \\hl{converges in probability to a quantity in} $(0,1]$. By Slutsky's theorem, $t$ \\hl{is asymptotically normal} and Wald-type confidence intervals constructed as $\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)\\pm z_{1-\\alpha/2}\\sqrt{\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}$ will tend to cover $\\tau_{\\mathrm{HT}}(d_k,d_l)$ at least $100(1-\\alpha)\\%$ of the time as $N\\to\\infty$.", "corrected": "\\emph{First paragraph of A.7.} By Proposition 5.6, $\\mathrm{E}[N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]]\\ge N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$. Thus, by Chebyshev's inequality, $\\operatorname{Var}[N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]]\\xrightarrow{p}0$ is sufficient to establish \\hl{concentration} of $N\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$ \\hl{around its expectation, which is} greater than or equal to $N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]$, which is itself nonzero by Condition 6. \\hl{The expectation need not converge, and no limit of the variance estimator is used below.}\n\n\\emph{Last paragraph of A.7.} Let $\\widehat\\tau_N = \\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)$ and $\\tau_N = \\tau_{\\mathrm{HT}}(d_k,d_l)$, and set $V_N = \\operatorname{Var}[\\widehat\\tau_N]$, $\\widehat V_N = \\widehat{\\operatorname{Var}}[\\widehat\\tau_N]$, $v_N = N V_N$, $W_N = N \\widehat V_N$ and $Z_N = (\\widehat\\tau_N - \\tau_N)/\\sqrt{V_N}$. Under Conditions 3, 5, and 6, then, by Chen and Shao [(2004), Theorem 2.7], $Z_N$ is asymptotically $\\mathrm{N}(0,1)$. By Condition 6, $v_N \\to c > 0$; by Proposition 5.6, $\\mathrm{E}[W_N] \\ge v_N$; \\hl{for this step, assume the concentration premise} $\\operatorname{Var}(W_N) \\to 0$. For each $\\epsilon \\in (0,1)$, Chebyshev's inequality gives\n\\[\n\\hl{\\Pr\\bigl( W_N < (1-\\epsilon) v_N \\bigr) \\le \\frac{\\operatorname{Var}(W_N)}{\\epsilon^2 v_N^2} \\longrightarrow 0 ,}\n\\]\nI \\hl{count a negative variance estimate as noncoverage}; the bound shows that its probability tends to zero. On the complementary event $\\sqrt{\\widehat V_N/V_N} \\ge \\sqrt{1-\\epsilon}$. With $z = z_{1-\\alpha/2}$, it follows that\n\\[\n\\hl{\\liminf_{N \\to \\infty} \\Pr\\bigl( |\\widehat\\tau_N - \\tau_N| \\le z \\sqrt{\\widehat V_N} \\bigr) \\ge 2 \\Phi\\bigl( z \\sqrt{1-\\epsilon} \\bigr) - 1 .}\n\\]\nLetting $\\epsilon \\downarrow 0$, Wald-type confidence intervals constructed as $\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)\\pm z_{1-\\alpha/2}\\sqrt{\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}$ cover $\\tau_{\\mathrm{HT}}(d_k,d_l)$ with probability at least $1-\\alpha$ \\hl{in the limit inferior} as $N\\to\\infty$. \\hl{No limit is asserted for the variance ratio or for the statistic} $t$.", "evidence": "The evidence is an exact counterexample to the two printed claims and a derivation of the corrected bound.\n\n\\emph{Counterexample.} Set $n_0 = 0$ and $n_r = 2^{2^r}$ for $r \\ge 1$. The units are the positive integers, the population of size $N$ is $\\{1, \\ldots, N\\}$, and block $r$ consists of the units $n_{r-1}+1, \\ldots, n_r$. There are two exposures, $d_k = 1$ and $d_l = 0$. In a block with $r$ odd, every unit has potential outcomes $(y_i(1), y_i(0)) = (1,1)$. In a block with $r$ even, the units alternate between $(2,0)$ and $(0,2)$, beginning with $(2,0)$. The assignments $Z_i$ are independent with $\\Pr(Z_i = 1) = 1/2$, and $D_i = Z_i$. Conditions 1 and 2 hold. Since the outcomes are at most $2$ and every exposure probability is $1/2$, Condition 3 holds. With $h_{ij} = \\mathbf{1}\\{i = j\\}$ and $m = 1$, Condition 5 holds, and every joint exposure probability of two distinct units equals $1/4$.\n\nLet $\\widehat\\tau_N$ and $\\tau_N$ be the estimator and the estimand in the population of size $N$, and set $R_i = 2 Z_i - 1$. Since $y_i(1) + y_i(0) = 2$ for every unit, I have\n\\[\n\\widehat\\tau_N - \\tau_N = \\frac{2}{N} \\sum_{i=1}^{N} R_i , \\qquad N V_N = 4\n\\]\nfor every $N$, so Condition 6 holds with $c = 4$, and the standardized estimator is asymptotically standard normal. Substituting the exposure probabilities into (7), (10) and (11), I obtain $\\widehat V_N = 4 N^{-2} \\sum_{i \\le N} Y_i^2$. Let $b_N$ be the number of units $i \\le N$ in blocks with $r$ even, and set $f_N = b_N/N$. A unit in an odd block has $Y_i^2 = 1$, and a unit in an even block has $Y_i^2 = 2 + 2 s_i R_i$ with $s_i = 1$ or $s_i = -1$. Taking the expectation and the variance of the sum, I find\n\\[\n\\mathbb{E}[N \\widehat V_N] = 4 + 4 f_N , \\qquad \\operatorname{Var}(N \\widehat V_N) = \\frac{64 \\, b_N}{N^2} \\le \\frac{64}{N} .\n\\]\nThe variance tends to zero, as the printed argument requires. The expectation does not converge: at the end of block $r$, I have $f_{n_r} \\le n_{r-1}/n_r$ for $r$ odd and $1 - f_{n_r} \\le n_{r-1}/n_r$ for $r$ even, and $n_{r-1}/n_r = 2^{-2^{r-1}}$ for $r \\ge 2$, which tends to zero.\n\\begin{audittable}{rrrll}{The example at the ends of the first four blocks. The scaled variance $N V_N$ equals $4$ in every row.}\n$r$ & $N = n_r$ & $b_N$ & $\\mathbb{E}[N \\widehat V_N]$ & $\\operatorname{Var}(N \\widehat V_N)$ \\\\\n$1$ & $4$ & $0$ & $4$ & $0$ \\\\\n$2$ & $16$ & $12$ & $7$ & $3$ \\\\\n$3$ & $256$ & $12$ & $67/16 \\approx 4.19$ & $3/256$ \\\\\n$4$ & $65536$ & $65292$ & $32707/4096 \\approx 7.99$ & $16323/16777216$ \\\\\n\\end{audittable}\nAlong $N = n_r$ with $r$ odd, $N \\widehat V_N$ tends to $4$ in probability, the ratio $(V_N/\\widehat V_N)^{1/2}$ tends to $1$, and $t$ converges in distribution to $\\mathcal{N}(0,1)$. Along $N = n_r$ with $r$ even, $N \\widehat V_N$ tends to $8$ in probability, the ratio tends to $1/\\sqrt{2}$, and $t$ converges in distribution to $\\mathcal{N}(0,1/2)$. Neither the ratio nor $t$ has a limit along the full sequence. Enumerating all $2^N$ assignments for each $N \\le 10$ in exact arithmetic, I obtain the same values from the printed expressions (7), (10) and (11).\n\n\\emph{Corrected bound.} Set $v_N = N V_N$, $W_N = N \\widehat V_N$ and $Z_N = (\\widehat\\tau_N - \\tau_N)/\\sqrt{V_N}$. Suppose that $v_N \\to c > 0$, that $Z_N$ converges in distribution to $\\mathcal{N}(0,1)$, that $\\mathbb{E}[W_N] \\ge v_N$, and that $\\operatorname{Var}(W_N) \\to 0$. I count a negative variance estimate as noncoverage in the probabilities below. Fix $\\epsilon \\in (0,1)$. If $W_N < (1-\\epsilon) v_N$, then $\\mathbb{E}[W_N] - W_N > \\epsilon v_N$. By Chebyshev's inequality, this implies that\n\\[\n\\Pr\\bigl( W_N < (1-\\epsilon) v_N \\bigr) \\le \\frac{\\operatorname{Var}(W_N)}{\\epsilon^2 v_N^2} \\longrightarrow 0 .\n\\]\nOn the complementary event, $\\widehat V_N$ is positive and $\\sqrt{\\widehat V_N / V_N} \\ge \\sqrt{1-\\epsilon}$. With $z = z_{1-\\alpha/2}$, I have\n\\[\n\\Pr\\bigl( |\\widehat\\tau_N - \\tau_N| \\le z \\sqrt{\\widehat V_N} \\bigr) \\ge \\Pr\\bigl( |Z_N| \\le z \\sqrt{1-\\epsilon} \\bigr) - \\Pr\\bigl( W_N < (1-\\epsilon) v_N \\bigr) .\n\\]\nTaking the limit inferior and using the convergence of $Z_N$, I obtain\n\\[\n\\liminf_{N \\to \\infty} \\Pr\\bigl( |\\widehat\\tau_N - \\tau_N| \\le z \\sqrt{\\widehat V_N} \\bigr) \\ge 2 \\Phi\\bigl( z \\sqrt{1-\\epsilon} \\bigr) - 1 .\n\\]\nSince $\\epsilon$ is arbitrary, the limit inferior of the coverage is at least $2\\Phi(z) - 1 = 1 - \\alpha$. Note that the argument uses no limit for $W_N$ and no independence between $Z_N$ and $W_N$.", "problem": "Let $V_N$ denote the variance of the HT estimator of the effect, and let $\\widehat V_N$ denote the variance estimator (11). The last paragraph of A.7 uses three premises: the estimator standardized by $\\sqrt{V_N}$ is asymptotically standard normal; $N V_N$ tends to $c > 0$ (Condition~6); and $N \\widehat V_N$ has expectation at least $N V_N$ (Proposition~5.6) and variance tending to zero. By Chebyshev's inequality, the third premise implies that $N \\widehat V_N$ is close to its own expectation with probability tending to one. It does not imply convergence ``to a value''. The expectation of $N \\widehat V_N$ exceeds $N V_N$ by the bias of the conservative estimator. No condition controls how this bias varies with $N$, and the expectation may fail to converge. The opening inference of A.7 is then not valid. The two claims of the last paragraph that rest on it can fail under Conditions 3, 5 and 6: that the ratio $(V_N/\\widehat V_N)^{1/2}$ converges in probability to a quantity in $(0,1]$, and that $t$ is asymptotically normal by Slutsky's theorem.\n\nThe conclusion of the paragraph does not need these limits. Coverage of at least $100(1-\\alpha)\\%$ for large $N$ is a lower bound, and a one-sided bound on $N \\widehat V_N$ is enough to obtain it. This finding takes the three premises as given. Whether the variance of $N \\widehat V_N$ tends to zero under Conditions 3, 5 and 6 is a separate question, treated in \\fid{P013-E001}.", "what_survives": "The HT estimator, the variance estimator (11), unbiasedness, Proposition~5.6, Proposition~6.1 and the central limit theorem for the estimator standardized by its true variance are unaffected. The coverage conclusion of Proposition~6.2 holds as a lower bound on the limit inferior of the coverage, under the three premises of the last paragraph of A.7 and with no new assumption. The proof no longer gives a single limiting law for $t$ or a single limit for the coverage. In the example, the coverage tends to $1 - \\alpha$ along one subsequence and to $2\\Phi(z_{1-\\alpha/2}\\sqrt{2}) - 1$ along the other. To keep the printed limit, one would add the assumption that $\\mathbb{E}[N \\widehat V_N]$ converges to a finite $s \\ge c$; then the ratio tends to $\\sqrt{c/s}$ and $t$ converges in distribution to $\\mathcal{N}(0, c/s)$. No number in Table~1 or Table~2 changes." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E007", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of one displayed formula and five lines of text. The display is N to the power minus 2 times a quadruple sum over i, j, k and l of the covariance of h sub ij times a sub ij (D sub i, D sub j) and h sub kl times a sub kl (D sub k, D sub l). The text reads: Note that this covariance is not equal to 0 if and only if h sub ij = 1 and h sub kl = 1, and either h sub ik = 1, h sub il = 1, h sub jk = 1, or h sub jl = 1. By Condition 5, given m much less than N, each of these four conditions is satisfied by fewer than N m cubed of the elements of the quadruple summation, and the number of elements in their union is at most 4 N m cubed. A red outline surrounds the words if and only if.", "caption": "Appendix A.7, p. 1943 (PDF p. 32): the last line of the display that expands the variance into a quadruple sum of covariances, and the sentence after it. The outline marks “if and only if”.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E011.png" } ], "formal_release": null, "formal_statements": [], "headline": "The condition on the indicators $h$ is necessary for a nonzero covariance and is not equivalent to it; the count that follows uses only the necessary direction, and “if and only if” should read “only if”.", "id": "P013-E011", "location": "Appendix A.7, sentence after the quadruple covariance sum, p. 1943 (PDF p. 32)", "pdf_page": 32, "printed_page": "1943", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [], "text": { "as_printed": "Note that $\\operatorname{Cov}[h_{ij}a_{ij}(D_i,D_j),h_{kl}a_{kl}(D_k,D_l)]\\ne0$ \\hl{if and only if} $h_{ij}=1$ and $h_{kl}=1$, and either $h_{ik}=1$, $h_{il}=1$, $h_{jk}=1$, or $h_{jl}=1$. By Condition 5, given $m\\ll N$, each of these four conditions is satisfied by fewer than $Nm^3$ of the elements of the quadruple summation, and the number of elements in their union is at most $4Nm^3$.", "corrected": "Note that $\\operatorname{Cov}[h_{ij}a_{ij}(D_i,D_j),h_{kl}a_{kl}(D_k,D_l)]\\ne0$ \\hl{only if} $h_{ij}=1$ and $h_{kl}=1$, and either $h_{ik}=1$, $h_{il}=1$, $h_{jk}=1$, or $h_{jl}=1$.", "intended_meaning": "The indicators $h_{ij}$ are the edges of the dependency graph described after Condition~5 (p.~1926). A covariance term can be nonzero only if both pairs are edges and some edge joins the two pairs. Without such an edge, the exposures of the two pairs are independent and the covariance is zero. This reading is correct, and the next sentence uses only this direction, since it bounds the number of terms that can be nonzero. The converse is false, since an edge permits dependence and does not force a nonzero covariance. With independent exposures of probability $1/2$, outcomes $y_i(1) = 1$ and $y_i(0) = 0$, and $h_{ij} = 1$ within blocks of four units, the terms $a_{12}$ and $a_{34}$ are identically zero although $h_{12} = h_{34} = h_{13} = 1$, and Condition~6 holds. The note concerns this sentence only; the other steps of A.7 are treated in \\fid{P013-E001} and \\fid{P013-E007}." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E011", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of Proposition 4.1 and the paragraph after it. A red box marks the right-hand side of the displayed inequality, at most (1 minus pi_0(d_k)) to the power R + 1. A second red box marks the rule R greater than or equal to log(b) divided by log(1 minus pi_0(d_k)), minus 1, in the sentence that follows.", "caption": "Proposition 4.1 and the rule for the number of replicates, p. 1920 (PDF p. 9). The marked bound and the marked rule hold when all $y_i(d_k)$ have the same sign and $y^T(d_k)\\ne0$, restrictions the proposition does not state.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E003.png" }, { "alt": "Crop of the end of the proof of Proposition 4.1. After the words 'And so', a display equates the absolute bias with the absolute value of the sum over i of y_i(d_k) times (1 minus pi_i(d_k)) to the power R + 1. A red box marks the next line, at most the absolute value of y^T(d_k) times (1 minus pi_0(d_k)) to the power R + 1.", "caption": "Appendix A.3, last display, p. 1938 (PDF p. 27). The marked inequality is the step that can fail for outcomes of mixed sign.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E003-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T3", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T3" }, { "id": "P013-T4", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T4" }, { "id": "P013-T12", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T12" } ], "headline": "The bound of Proposition 4.1 holds for potential outcomes of one sign with nonzero total; for general nonzero totals, multiplying its right-hand side by $\\sum_i|y_i(d_k)|/|y^T(d_k)|$ gives a valid bound, and the rule for the number of replicates changes in the same way.", "id": "P013-E003", "location": "Proposition 4.1 and the paragraph after it, p. 1920 (PDF p. 9); last inequality of Appendix A.3, p. 1938 (PDF p. 27)", "pdf_page": 9, "printed_page": "1920", "reaches": [ { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Moderate", "subject": "paper", "target_ids": [ "P013-T12", "P013-T3", "P013-T4" ], "text": { "as_printed": "Proposition 4.1 (p.~1920): ``The relative bias for $\\widehat{y^T_{\\mathrm{HT},R}}(d_k)$ is bounded as\n\\[\n\\left|\\frac{\\mathrm{E}[\\widehat{y^T_{\\mathrm{HT},R}}(d_k)]-y^T(d_k)}{y^T(d_k)}\\right|\\le\\hl{(1-\\pi_0(d_k))^{R+1}},\n\\]\nwhere $\\pi_0(d_k)=\\min_i\\{\\pi_i(d_k)\\}$.\n\nFor a relative bias target of $b$ and given some approximation of $\\pi_0(d_k)$, the bound implies selecting a number of replicates $R\\ge\\log(\\hl{b})/\\log(1-\\pi_0(d_k))-1$. Thus, for $b=0.005$ and $\\pi_0(d_k)=0.0005$, this would imply at least 10,593 replicates.''\n\nAppendix A.3, last display (p.~1938):\n\\[\n\\left|\\mathrm{E}[\\widehat{y^T_{\\mathrm{HT},R}}(d_k)]-y^T(d_k)\\right|=\\left|\\sum_{i=1}^{N}y_i(d_k)(1-\\pi_i(d_k))^{R+1}\\right|\\le\\hl{|y^T(d_k)|}(1-\\pi_0(d_k))^{R+1}.\n\\]", "corrected": "Proposition 4.1. Let the $R$ replicates be drawn independently from the randomization plan and independently of the experiment, let $\\pi_0(d_k)=\\min_i\\{\\pi_i(d_k)\\}$ with $0<\\pi_i(d_k)<1$ for every $i$, and write $T=y^T(d_k)$ and $\\hl{L=\\sum_{i=1}^{N}|y_i(d_k)|}$. Then\n\\[\n\\left|\\mathrm{E}[\\widehat{y^T_{\\mathrm{HT},R}}(d_k)]-T\\right|\\le\\hl{L}\\,(1-\\pi_0(d_k))^{R+1}.\n\\]\nIf $T\\ne0$, the relative bias for $\\widehat{y^T_{\\mathrm{HT},R}}(d_k)$ is bounded as\n\\[\n\\left|\\frac{\\mathrm{E}[\\widehat{y^T_{\\mathrm{HT},R}}(d_k)]-T}{T}\\right|\\le\\hl{\\kappa}\\,(1-\\pi_0(d_k))^{R+1},\\qquad\\hl{\\kappa=L/|T|}.\n\\]\nIf all $y_i(d_k)$ have the same sign and $T\\ne0$, then $\\kappa=1$ and the bound is $(1-\\pi_0(d_k))^{R+1}$, as printed.\n\nFor $\\hl{T\\ne0}$ and a relative bias target of $\\hl{b>0}$, it suffices to select a \\hl{nonnegative integer} number of replicates $R\\ge\\log(\\hl{b/\\kappa})/\\log(1-\\pi_0(d_k))-1$, using $\\pi_0(d_k)$ \\hl{or a positive lower bound for it}, and $\\kappa$ or an upper bound for it. For outcomes of one sign this is the printed rule, and for $b=0.005$ and $\\pi_0(d_k)=0.0005$ it gives at least 10,593 replicates.\n\nAppendix A.3, last display:\n\\[\n\\left|\\mathrm{E}[\\widehat{y^T_{\\mathrm{HT},R}}(d_k)]-y^T(d_k)\\right|=\\left|\\sum_{i=1}^{N}y_i(d_k)(1-\\pi_i(d_k))^{R+1}\\right|\\le\\hl{\\sum_{i=1}^{N}|y_i(d_k)|}\\,(1-\\pi_0(d_k))^{R+1}.\n\\]", "evidence": "The evidence is an exact counterexample and a derivation of the corrected bound. Let $N=2$, with no interference, and let the two units be exposed to $d_k$ independently with probabilities $\\pi_1(d_k)=1/3$ and $\\pi_2(d_k)=2/3$. Let $y_1(d_k)=2$ and $y_2(d_k)=-1$, and let $R=1$ replicate be drawn independently of the experiment. Then $y^T(d_k)=1$. Summing over the 16 pairs of an actual and a simulated assignment, I obtain $\\mathrm{E}[\\widehat{y^T_{\\mathrm{HT},R}}(d_k)]=2/9$. The relative bias is $7/9$, and the printed bound is $(1-1/3)^2=4/9$. For the target $b=1/2$ the printed rule selects $R=1$, and the relative bias $7/9$ exceeds the target. With outcomes $(1,-1)$ in the same design, the total is $0$ and the expected estimate is $-1/3$.\n\n\\begin{audittable}{lll}{The two-unit example with $R=1$ and exposure probabilities $(1/3,2/3)$, by exact enumeration.}\nQuantity & Outcomes $(2,-1)$ & Outcomes $(1,-1)$ \\\\\nTotal $y^T(d_k)$ & $1$ & $0$ \\\\\nExpected estimate & $2/9$ & $-1/3$ \\\\\nAbsolute bias & $7/9$ & $1/3$ \\\\\nPrinted bound on the absolute bias & $4/9$ & $0$ \\\\\nCorrected bound on the absolute bias & $4/3$ & $8/9$ \\\\\n\\end{audittable}\n\nFor the corrected bound, I write $q_i=(1-\\pi_i(d_k))^{R+1}$ and $q_0=(1-\\pi_0(d_k))^{R+1}$, so that $0\\le q_i\\le q_0$. By the identity of Appendix A.3 and the triangle inequality, I have\n\\[\n\\left|\\mathrm{E}[\\widehat{y^T_{\\mathrm{HT},R}}(d_k)]-y^T(d_k)\\right|=\\Bigl|\\sum_{i=1}^{N}y_i(d_k)q_i\\Bigr|\\le\\sum_{i=1}^{N}|y_i(d_k)|q_i\\le q_0\\sum_{i=1}^{N}|y_i(d_k)|.\n\\]\nIf all $y_i(d_k)$ have the same sign and $y^T(d_k)\\ne0$, then $\\sum_i|y_i(d_k)|=|y^T(d_k)|$ and the printed bound follows. In the example, the corrected rule with $b=1/2$ gives $R=4$, for which the corrected bound is $32/81$ and the relative bias is $7/27$.", "problem": "Proposition 4.1 bounds the relative bias of the Horvitz--Thompson total computed with simulated exposure probabilities by $(1-\\pi_0(d_k))^{R+1}$, and it places no restriction on the signs of the potential outcomes. With independent replication, the expectation identities of Appendix A.3 are correct: the bias equals $-\\sum_i y_i(d_k)(1-\\pi_i(d_k))^{R+1}$. The last step of the proof bounds the absolute value of this weighted sum by $|y^T(d_k)|(1-\\pi_0(d_k))^{R+1}$. That step can fail when the outcomes have mixed signs and the exposure probabilities differ across units, because positive and negative outcomes can cancel in the total $y^T(d_k)$ without cancelling in the weighted sum. When $y^T(d_k)=0$ the relative bias is not defined. The context does not exclude this case, since the paper considers outcomes centred on zero. For mixed signs, the rule $R\\ge\\log(b)/\\log(1-\\pi_0(d_k))-1$ does not in general guarantee a relative bias of at most $b$, and no number of replicates chosen from $\\pi_0(d_k)$ and $b$ alone can guarantee it for all signed outcomes with nonzero total.", "what_survives": "The expectation identities of Appendix A.3 hold under independent replication, and the bias of the approximate total still tends to zero as $R$ grows in a fixed finite population. The unbiasedness of the Horvitz--Thompson estimator with exact probabilities (Lemma 4.1), Proposition 4.2 and the later variance and asymptotic results do not use Proposition 4.1. The arithmetic of the printed example is right: the threshold is $10{,}592.99$, and 10,593 is the smallest admissible integer. The outcomes of the simulation (counts multiplied by positive factors) and of the application (a binary outcome) are nonnegative, and the printed bound applies when the corresponding potential-outcome total is nonzero. It need not apply to totals of signed residuals. What changes is the scope of the proposition and of the rule for $R$." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E003", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of four lines of text and the first row of expression (12). A red box marks the condition 'Cov of g of x i, xi i of d k, and Z i, equals 0', which the sentence calls a sufficient condition for unbiasedness.", "caption": "Section 7.1, p. 1927 (PDF p. 16), the sentence before expression (12). The marked condition involves the unit’s own assignment $Z_i$.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E004.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T16", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T16" } ], "headline": "Expression (12) is unbiased when the predictions $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))$ are fixed before the randomization, which is the case that Section 7.1 describes; for predictions that vary with the assignment, a sufficient condition is zero covariance with the exposure indicator $\\mathbf{I}(D_i=d_k)$, and zero covariance with $Z_i$ is not enough.", "id": "P013-E004", "location": "Section 7.1, p. 1927 (PDF p. 16), sentence before expression (12)", "pdf_page": 16, "printed_page": "1927", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [ "P013-T16" ], "text": { "as_printed": "We have some predefined function $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))\\to\\mathbb{R}$, where $\\boldsymbol{\\xi}_i$ is a parameter vector. Ideally $g(\\cdot)$ is calibrated on auxiliary data to produce values that approximate $y_i(d_k)$. We assume $\\operatorname{Cov}[g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k)),\\hl{Z_i}]=0$ as a sufficient condition for unbiasedness. Define\n\\[\n(12)\\qquad \\widehat{y^T_G}(d_k)=\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)\\frac{Y_i}{\\pi_i(d_k)}-\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)\\frac{g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))}{\\pi_i(d_k)}+\\sum_{i=1}^N g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k)),\n\\]\nwhich is unbiased for $y^T(d_k)$ by\n\\[\n\\mathrm{E}\\left[-\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)\\frac{g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))}{\\pi_i(d_k)}+\\sum_{i=1}^N g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))\\right]=0.\n\\]", "corrected": "Let Conditions 1 and 2 hold and let $\\pi_i(d_k)>0$ for every $i$. If the predictions $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))$ are \\hl{fixed}, that is, if they do not vary with the assignment $\\mathbf{Z}$, then expression (12) is unbiased for $y^T(d_k)$. If the predictions are random, then\n\\[\n\\mathrm{E}\\bigl[\\widehat{y^T_G}(d_k)\\bigr]-y^T(d_k)=-\\sum_{i=1}^N\\frac{\\operatorname{Cov}[g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k)),\\mathbf{I}(D_i=d_k)]}{\\pi_i(d_k)},\n\\]\nand a sufficient condition for unbiasedness is\n\\[\n\\operatorname{Cov}[g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k)),\\hl{\\mathbf{I}(D_i=d_k)}]=0\\quad\\text{for every } i .\n\\]\nZero covariance with $Z_i$ alone does not suffice. Neither condition is necessary, since the weighted covariances in the sum may cancel.", "intended_meaning": "Section 7.1 describes a difference estimator whose predictions are settled before the experiment. The function $g$ is ``predefined'' and ideally ``calibrated on auxiliary data'', the variance paragraph on the same page requires that $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))$ ``is fixed'', and the regression fitted to the experimental data is treated separately, with the warning that ``unbiasedness is compromised''. For fixed predictions $g_i$, expression (12) is unbiased under the standing assumptions of the paper, with no further condition. Indeed, since $\\mathrm{E}[\\mathbf{I}(D_i=d_k)]=\\pi_i(d_k)$, I have\n\\[\n\\mathrm{E}\\left[g_i-\\mathbf{I}(D_i=d_k)\\frac{g_i}{\\pi_i(d_k)}\\right]=g_i-g_i\\,\\frac{\\pi_i(d_k)}{\\pi_i(d_k)}=0 .\n\\]\nUnder this reading the printed covariance condition is redundant, and the display after (12) is correct.\n\nRead literally, as a condition that admits predictions depending on the assignment, the sentence is too weak. For random predictions $G_i$, a direct computation gives\n\\[\n\\mathrm{E}\\bigl[\\widehat{y^T_G}(d_k)\\bigr]-y^T(d_k)=-\\sum_{i=1}^N\\frac{\\operatorname{Cov}[G_i,\\mathbf{I}(D_i=d_k)]}{\\pi_i(d_k)} .\n\\]\nUnder interference the exposure $D_i=f(\\mathbf{Z},\\theta_i)$ depends on the assignments of other units, and zero covariance with $Z_i$ does not imply that the right side vanishes. An exact example shows this. Let $N=2$, let $Z_1$ and $Z_2$ be independent and uniform on $\\{1,2\\}$, and let $D_1=Z_2$ and $D_2=Z_1$. Let $d_k=2$, with potential outcomes $y_1(2)=3$, $y_2(2)=5$ and $y_1(1)=y_2(1)=0$, and let $G_1=Z_2-1$ and $G_2=Z_1-1$. Then $\\operatorname{Cov}[G_i,Z_i]=0$ for both units, while $\\operatorname{Cov}[G_i,\\mathbf{I}(D_i=2)]=1/4$. Expression (12) takes the values $0$, $5$, $9$ and $14$ with probability $1/4$ each, and its expectation is $7$. The target total is $8$. With the fixed predictions $G_1=G_2=1$ in the same design, expression (12) takes the values $2$, $6$, $10$ and $14$, with expectation $8$.\n\nThis example establishes no bias in any reported estimate; the fitted regression estimator is treated separately in expression (14). The simulation of Section 9 uses the Horvitz--Thompson, Hajek, weighted least squares and ordinary least squares estimators and the simple difference in sample means, and the application of Section 10 uses the first three." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E004", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of six lines of the first paragraph of Section 7.1, from 'Chapter 6]. Covariance adjustment of this variety' to 'perhaps obtained from analysis of auxiliary datasets.' Red boxes mark the words 'the difference estimator addresses the problem of location noninvariance that afflicts Horvitz-Thompson-type estimators', which run over two lines.", "caption": "Section 7.1, first paragraph, p. 1927 (PDF p. 16). The marked sentence says, without a condition, that the difference estimator addresses location noninvariance.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E005.png" }, { "alt": "Crop of expression (13) and of the four lines of text after it. Expression (13) writes the estimator y-hat-T-G of d_k as the sum over i of I(D_i = d_k) times epsilon_i(d_k) divided by pi_i(d_k), plus the sum over i of g(x_i, xi_i(d_k)). A red box marks the words 'is location invariant' in the sentence 'Middleton and Aronow (2011) and Aronow and Middleton (2013) demonstrate that y-hat-T-G of d_k is location invariant'. The next sentence says that variance estimation proceeds as in Section 5 as long as g(x_i, xi_i(d_k)) is fixed.", "caption": "Expression (13) and the sentences after it, p. 1927 (PDF p. 16). The marked words state the property for $\\widehat{y^T_G}(d_k)$ without a condition on the predictions.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E005-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T21", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T21" }, { "id": "P013-T22", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T22" } ], "headline": "The difference estimator of expressions (12) and (13) is location equivariant if a constant added to every outcome is also added to every prediction; Section 7.1 states the property without this condition, and it fails in general if the predictions are held numerically fixed, because the estimated total then changes by the constant times the realized sum of the weights $\\mathbf{I}(D_i=d_k)/\\pi_i(d_k)$, which need not equal the constant times $N$.", "id": "P013-E005", "location": "Section 7.1, p. 1927 (PDF p. 16): first paragraph, and the sentence after expression (13)", "pdf_page": 16, "printed_page": "1927", "reaches": [ { "effect": "uses_only_what_survives", "effect_text": "relies only on the part that survives", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P037/", "where": null, "work_id": "P037" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P013-T21", "P013-T22" ], "text": { "as_printed": "``In addition, the difference estimator \\hl{addresses the problem of location noninvariance} that afflicts Horvitz--Thompson-type estimators [Fuller (2009), pages 9--10].''\n\n``We have some predefined function $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))\\to\\mathbb{R}$, where $\\boldsymbol{\\xi}_i$ is a parameter vector. Ideally $g(\\cdot)$ is calibrated on auxiliary data to produce values that approximate $y_i(d_k)$. \\ldots\\ Define\n\\[\n(12)\\qquad \\widehat{y^T_G}(d_k)=\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)\\frac{Y_i}{\\pi_i(d_k)}-\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)\\frac{g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))}{\\pi_i(d_k)}+\\sum_{i=1}^N g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k)),\n\\]\n\\ldots''\n\n``Middleton and Aronow (2011) and Aronow and Middleton (2013) demonstrate that $\\widehat{y^T_G}(d_k)$ \\hl{is location invariant}. Variance estimation proceeds as in Section 5 using $\\varepsilon_i(d_k)$ in place of $y_i(d_k)$ as long as $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))$ is fixed.''", "corrected": "First paragraph: ``In addition, the difference estimator addresses the problem of location noninvariance that afflicts Horvitz--Thompson-type estimators [Fuller (2009), pages 9--10]\\hl{, provided that the predictions are shifted by the same constant as the outcomes}.''\n\nAfter expression (13): ``Middleton and Aronow (2011) and Aronow and Middleton (2013) demonstrate that $\\widehat{y^T_G}(d_k)$ is location \\hl{equivariant when adding a constant $c$ to all outcomes also adds $c$ to all predictions}.''\n\nIn full, the corrected statement is as follows. Let Conditions 1 and 2 hold, let $\\pi_i(d_k)>0$ and $\\pi_i(d_l)>0$ for every $i$, and let the predictions $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))$ and $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_l))$ not vary with the assignment. Suppose that a constant $c$ is added to every potential outcome and \\hl{that $c$ is also added to every prediction}. Then, on every assignment, $\\widehat{y^T_G}(d_k)$ increases by $cN$, the estimated mean $\\widehat{y^T_G}(d_k)/N$ increases by $c$, and the estimated contrast $[\\widehat{y^T_G}(d_k)-\\widehat{y^T_G}(d_l)]/N$ does not change. If instead the outcomes are shifted and the predictions are held numerically fixed, then $\\widehat{y^T_G}(d_k)$ increases by\n\\[\nc\\sum_{i=1}^N\\frac{\\mathbf{I}(D_i=d_k)}{\\pi_i(d_k)},\n\\]\nwhich is not $cN$ in general.\n\nThe condition is sufficient and not necessary. If the prediction of unit $i$ changes by $h_i$, the estimated total increases by $cN$ on an assignment exactly when $\\sum_{i=1}^N(1-w_i)(h_i-c)=0$, where $w_i=\\mathbf{I}(D_i=d_k)/\\pi_i(d_k)$. For predictions held numerically fixed and $c\\ne0$, this condition reads $\\sum_{i=1}^N w_i=N$, and the estimated contrast between $d_k$ and $d_l$ is unchanged exactly when the two realized sums of the weights are equal. For example, if a fixed number $m_k$ of the $N$ units receive exposure $d_k$ and $\\pi_i(d_k)=m_k/N$ for every $i$, the weights sum to $N$ on every assignment.", "evidence": "The evidence is a derivation of the exact change of the estimator under a shift of the outcomes, an exact counterexample with the predictions held numerically fixed, and a check of the cited works.\n\nLet $g_i=g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))$, $w_i=\\mathbf{I}(D_i=d_k)/\\pi_i(d_k)$ and $W_k=\\sum_{i=1}^N w_i$. By Condition 2, I have\n\\[\n\\widehat{y^T_G}(d_k)=\\sum_{i=1}^N w_i\\,\\bigl(y_i(d_k)-g_i\\bigr)+\\sum_{i=1}^N g_i .\n\\]\nSuppose that every potential outcome increases by $c$ and that each prediction $g_i$ changes by $h_i$, while the design, the exposures and the covariates stay the same. Substituting into the display, I obtain that the estimated total changes by\n\\[\n\\sum_{i=1}^N w_i\\,(c-h_i)+\\sum_{i=1}^N h_i=cW_k+\\sum_{i=1}^N(1-w_i)h_i .\n\\]\nSince $\\sum_{i=1}^N(1-w_i)=N-W_k$, this change differs from $cN$ by\n\\[\n\\sum_{i=1}^N(1-w_i)(h_i-c).\n\\]\nIf $h_i=c$ for every $i$, each term vanishes. The residuals $y_i(d_k)-g_i$ are unchanged, the estimated total increases by $cN$ on every assignment, and the estimated contrast $[\\widehat{y^T_G}(d_k)-\\widehat{y^T_G}(d_l)]/N$ between two exposures does not change. If $h_i=0$ for every $i$, the estimated total increases by $cW_k$, the error is $c\\,(W_k-N)$, and the estimated contrast changes by $c\\,(W_k-W_l)/N$. Since $\\mathrm{E}[w_i]=1$, the error has expectation zero, but for $c\\ne0$ it vanishes on a given assignment only if $W_k=N$.\n\nFor the counterexample, let $N=3$ with no interference. One unit is selected, with probabilities $1/4$, $1/4$ and $1/2$ for the three units. For the selected unit $D_i=d_k$, and for the other two $D_i=d_l$. Then $\\pi_i(d_k)$ equals $1/4$, $1/4$, $1/2$ and $\\pi_i(d_l)$ equals $3/4$, $3/4$, $1/2$, and Conditions 1 and 2 hold. Let the potential outcomes be $(2,3,5)$ under both exposures, and let the predictions be the fixed numbers $(2,3,5)$ for both exposures. The predictions do not vary with the assignment, their covariance with $Z_i$ is zero, and they equal the outcomes. Both estimated totals equal $10$ on every assignment, and the estimated contrast is $0$. I now add $c=1$ to every potential outcome and keep the predictions. Both true totals become $13$, and the true contrast $\\tau(d_k,d_l)$ remains $0$. Every residual equals $1$, and the estimated totals are $10+W_k$ and $10+W_l$.\n\n\\begin{audittable}{crrrrrr}{The three-unit example after $1$ is added to every outcome, with the predictions held numerically fixed at $(2,3,5)$: the realized sums of the weights, the two estimated totals and the estimated contrast. Both true totals equal $13$ and the true contrast equals $0$.}\nSelected unit & Probability & $W_k$ & $W_l$ & $\\widehat{y^T_G}(d_k)$ & $\\widehat{y^T_G}(d_l)$ & Estimated contrast \\\\\n1 & $1/4$ & $4$ & $10/3$ & $14$ & $40/3$ & $2/9$ \\\\\n2 & $1/4$ & $4$ & $10/3$ & $14$ & $40/3$ & $2/9$ \\\\\n3 & $1/2$ & $2$ & $8/3$ & $12$ & $38/3$ & $-2/9$ \\\\\n\\end{audittable}\n\nNeither estimated total increases by $cN=3$ on any assignment, and the estimated contrast is never $0$. Both estimated totals have expectation $13$ and remain unbiased. The variance of the estimated contrast rises from $0$ to $4/81$. If $1$ is also added to every prediction, both estimated totals equal $13$ and the estimated contrast equals $0$ on every assignment. Conditions 1--6 do not restore the property. Independent copies of this design, with every outcome and every prediction equal to $1$, form a sequence that satisfies them with $N\\operatorname{Var}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]=4/27$, and a shift of the outcomes alone still changes the estimated contrast.\n\nWhere I could consult the cited works, they state the property with the condition. Aronow and Middleton (2013) recode the outcomes as $Y_i^*=b_0+b_1Y_i$ (p.~139) and, in their Appendix C (p.~151), require that the ``predictive function will also change by the identical transformation''. Their equation (41) is $f(X_i;\\theta_i)^*=b_0+b_1f(X_i;\\theta_i)$. The published version of the 2011 paper (Middleton and Aronow, 2015, Section 5.2) has the same requirement: ``the same transformation must be applied to $k$'', with $k^*=b_0+b_1k$. The 2011 conference paper itself was not obtained.", "problem": "Section 7.1 says twice that the difference estimator does not depend on the origin of the outcome scale in the way that Horvitz--Thompson estimators do. The first statement is in its first paragraph. The second follows expression (13) and attributes the property to two cited works. Neither states a condition. For an estimated total the property is equivariance. If a constant $c$ is added to every potential outcome, the estimated total should increase by $cN$, and the estimated contrast between two exposures should not change. Whether expression (12) has this property depends on what happens to the predictions $g(\\mathbf{x}_i,\\boldsymbol{\\xi}_i(d_k))$ when the outcomes are shifted, and the section does not say. It calls the predictions ``predefined'' and ``fixed'', and these words refer to the randomization, not to the origin of the outcome scale. If the predictions are shifted by $c$ with the outcomes, the property holds. If they are held numerically fixed, which the printed assumptions allow, it fails in general. The estimated total then increases by $cW_k$, where $W_k=\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)/\\pi_i(d_k)$ is the realized sum of the weights. The error $c\\,(W_k-N)$ varies with the assignment. Aronow and Middleton (2013) show the property for predictions that are recoded by the same transformation as the outcomes. The sentence after expression (13) reports this conditional result without its condition.", "what_survives": "Expressions (12) and (13), the estimands and the exposure mappings are unchanged. For fixed predictions, expression (12) is unbiased, the variance calculation with $\\varepsilon_i(d_k)$ in place of $y_i(d_k)$ stands, and the adjustment can still reduce the variance. No later result of the paper uses the property. The regression estimators, the Hajek estimator and the difference in sample means that the paper reports have the property without an assumption on predefined predictions. The weighted least squares fit of expression (14) with an exposure intercept shifts its fitted values by $c$ when the fit is identified. The Hajek ratio of expression (15) and the sample means shift by $c$ when their exposure groups are not empty. In the ordinary least squares regression on exposure indicators and degree, the contrasts between exposures are unchanged when they are estimable. The simulation of Section 9 and the application of Section 10 use these estimators and the Horvitz--Thompson estimator, for which the paper does not claim the property. They do not use expression (12) with predefined predictions, and no entry of Table 1 or Table 2 is shown to be wrong. Under Conditions 3 and 4, $W_k/N$ tends to $1$ in probability. Under a shift of the outcomes alone, the change $c\\,(W_k-W_l)/N$ of an estimated contrast then tends to zero in probability, but it can remain of the order of the sampling error. What is lost is the unconditional statement that predefined predictions remove the dependence on the origin of the outcome scale." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E005", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "historical_lean_attribution", "exhibits": [ { "alt": "Crop of the lower part of Table 2 with its notes. Four Hajek rows are followed by four WLS rows, and each row gives an estimand, an estimate, a standard error and a 95 percent interval. The WLS rows read: tau(d001, d000), 0.056, 0.066, (-0.072, 0.186); tau(d011, d000), 0.156, 0.037, (0.083, 0.229); tau(d101, d000), 0.295, 0.124, (0.050, 0.536); tau(d111, d000), 0.306, 0.049, (0.212, 0.404). Red outlines surround the estimate 0.295 and the interval (0.050, 0.536) in the third WLS row, and the estimate 0.306 and the interval (0.212, 0.404) in the fourth. The last table note reads: S.E. = Estimated standard error; CI = Normal approximation confidence interval.", "caption": "Table 2, p. 1936 (PDF p. 25): the Hajek and WLS rows and the table notes. The columns are estimator, estimand, estimate, S.E. and 95% CI. The outlines mark the estimate and the interval in the WLS rows for $\\tau(d_{101},d_{000})$ and $\\tau(d_{111},d_{000})$.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E008.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T10", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T10" } ], "headline": "In the WLS rows of Table 2 for $\\tau(d_{101},d_{000})$ and $\\tau(d_{111},d_{000})$, the printed estimate and the printed 95% interval cannot both be correct: a normal-approximation interval is centred at its estimate, and the two differ by more than rounding allows.", "historical_evidence_type": "lean_proof", "id": "P013-E008", "location": "Table 2, WLS rows for $\\tau(d_{101},d_{000})$ and $\\tau(d_{111},d_{000})$, p. 1936 (PDF p. 25)", "pdf_page": 25, "printed_page": "1936", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P013-T10" ], "text": { "as_printed": "\\begin{audittable}{llrrr}{Table~2, WLS rows.}\nEstimator & Estimand & Estimate & S.E. & 95\\% CI \\\\\nWLS & $\\tau(d_{001},d_{000})$ & 0.056 & 0.066 & $(-0.072, 0.186)$ \\\\\n & $\\tau(d_{011},d_{000})$ & 0.156 & 0.037 & $(0.083, 0.229)$ \\\\\n & $\\tau(d_{101},d_{000})$ & \\hl{0.295} & 0.124 & $\\hl{(0.050, 0.536)}$ \\\\\n & $\\tau(d_{111},d_{000})$ & \\hl{0.306} & 0.049 & $\\hl{(0.212, 0.404)}$ \\\\\n\\end{audittable}\n\\emph{Table note.} S.E. = Estimated standard error; CI = Normal approximation confidence interval.", "corrected": "Table~2, WLS rows for $\\tau(d_{101},d_{000})$ and $\\tau(d_{111},d_{000})$: in each row at least one of the estimate and the two interval endpoints is misprinted. The printed estimates and endpoints cannot arise from normal-approximation intervals centred at the same estimates, even allowing for rounding to three decimals. The two rows should be checked against the original unrounded output and the wrong cells corrected. The replacement entries cannot be determined from the published table or from the public data.\n\nThe table below gives two ways of making each row consistent, with $z = 1.96$. In the first, the interval and standard error are kept at their printed precision and the estimate changes. In the second, the printed estimate and standard error are taken as exact and the interval changes. They show the size of the change, and neither is a recovered value.\n\\begin{audittable}{llll}{Two ways of restoring consistency in each row (estimate, 95\\% CI).}\nRow & As printed & Interval and S.E. kept & Estimate and S.E. kept \\\\\n$\\tau(d_{101},d_{000})$ & $0.295$, $(0.050, 0.536)$ & $\\hl{0.293}$, $(0.050, 0.536)$ & $0.295$, $\\hl{(0.052, 0.538)}$ \\\\\n$\\tau(d_{111},d_{000})$ & $0.306$, $(0.212, 0.404)$ & $\\hl{0.308}$, $(0.212, 0.404)$ & $0.306$, $\\hl{(0.210, 0.402)}$ \\\\\n\\end{audittable}", "evidence": "The evidence is a derivation in exact arithmetic and a recalculation on the public data. Let $L = e - zs$ and $U = e + zs$ be the unrounded endpoints. Then $(L + U)/2 = e$. A number printed to three decimals as $p$ lies in $[p - 0.0005, p + 0.0005]$ under any rule for ties. Then $e$ lies within $0.0005$ of the printed estimate, and $(L + U)/2$ lies within $0.0005$ of the midpoint of the printed endpoints.\n\\begin{audittable}{lrlrlr}{The ranges that rounding allows in the two WLS rows.}\nRow & Estimate & Range for $e$ & Interval midpoint & Range for $(L+U)/2$ & Gap \\\\\n$\\tau(d_{101},d_{000})$ & $0.295$ & $[0.2945, 0.2955]$ & $0.293$ & $[0.2925, 0.2935]$ & $0.001$ \\\\\n$\\tau(d_{111},d_{000})$ & $0.306$ & $[0.3055, 0.3065]$ & $0.308$ & $[0.3075, 0.3085]$ & $0.001$ \\\\\n\\end{audittable}\nIn both rows the two ranges are disjoint. Note that the argument uses neither the standard error nor the critical value. The same comparison in the other ten rows of Table~2 finds nine that are compatible and one, the WLS row for $\\tau(d_{001},d_{000})$, that is compatible only at a boundary. Its printed cells $0.056$ and $(-0.072, 0.186)$ force $e = 0.0565$, $L = -0.0725$ and $U = 0.1855$, and these values print as shown when ties are rounded to even. A third misprint is not established.\n\nThe recalculation uses the public data release of the field experiment (ICPSR study 37070, version 2). With an adapter for the public data, it simulates the exposure probabilities with the seed and the 10,000 draws of the script released with the data and with the sampler that R used before version 3.6. It keeps the eligible students whose five estimated probabilities exceed $0.01$. It fits least squares weighted by inverse exposure probabilities, with an intercept for each exposure and one slope on network degree. This reconstruction has $N=2{,}050$ students and reproduces all twelve point estimates of Table~2 to three decimals. Its WLS estimates for the two rows are approximately $0.294544$ and $0.3063$, which round to the printed $0.295$ and $0.306$ and not to the midpoints $0.293$ and $0.308$. None of its standard errors rounds to a printed one. The cut-off and the form of the regression are choices of the reconstruction. None of the released scripts computes Table~2, and the authors' own files and unrounded output are not public; ICPSR warns that some variables in the public release were changed. The recalculation supports the two printed estimates. It does not identify the misprinted cells, and it does not recover the endpoints.", "problem": "The note to Table~2 says that the intervals are normal-approximation intervals. Such an interval has endpoints $e - zs$ and $e + zs$, where $e$ is the estimate, $s$ its standard error and $z$ the critical value, and its midpoint is $e$ for every $s$ and $z$. Each printed number is rounded to three decimals, so a printed estimate and the midpoint of a printed interval can differ by at most $0.001$. In the WLS row for $\\tau(d_{101},d_{000})$ the midpoint of $(0.050, 0.536)$ is $0.293$ and the estimate is $0.295$. In the row for $\\tau(d_{111},d_{000})$ the midpoint of $(0.212, 0.404)$ is $0.308$ and the estimate is $0.306$. Both differences are $0.002$. I conclude that in each row at least one of the estimate and the two endpoints is misprinted, and that no change to the standard error removes the inconsistency. The table does not show which cell is wrong.", "what_survives": "The eight HT and Hajek rows of Table~2 and the WLS row for $\\tau(d_{011},d_{000})$ are arithmetically compatible with their intervals after rounding, and the WLS row for $\\tau(d_{001},d_{000})$ is not shown to be wrong. I find no evidence that these inconsistencies reverse the qualitative conclusions in Section~10. Under either illustrative repair below, the direct effects stay near 30 percentage points and the lower endpoints stay positive. The printed HT and Hajek intervals for the direct and indirect contrasts are positive on their own. These inconsistencies have no demonstrated consequence for the theory, the simulation of Table~1 or the exposure mapping. At least one cell changes in each of the two rows, and its value requires the original unrounded output." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E008", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "complete_derivation", "exhibits": [ { "alt": "Crop of the closing sentences of Section 4. Red boxes mark the covariance Cov[y-hat-T-HT(d_k), y-hat-T-HT(d_l)] and, on the next line, the words 'non-negative bias' in the sentence 'we can always identify estimators for Var and Cov that are guaranteed to have non-negative bias'.", "caption": "Section 4, last paragraph, p. 1921 (PDF p. 10). The marked phrase “non-negative bias” is applied to the marked covariance as well as to the variance.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E009.png" } ], "formal_release": null, "formal_statements": [], "headline": "The sentence gives the covariance estimator the same sign of bias as the variance estimators; the covariance estimators of Section 5 have non-positive bias, which is what a conservative variance for the contrast requires.", "id": "P013-E009", "location": "Section 4, last paragraph, p. 1921 (PDF p. 10)", "pdf_page": 10, "printed_page": "1921", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [], "text": { "as_printed": "``Nonetheless, we can always identify estimators for $\\operatorname{Var}[\\widehat{y^T_{\\mathrm{HT}}}(d_k)]$ and $\\hl{\\operatorname{Cov}[\\widehat{y^T_{\\mathrm{HT}}}(d_k),\\widehat{y^T_{\\mathrm{HT}}}(d_l)]}$ that are guaranteed to have \\hl{non-negative bias}. Thus, we can always identify a conservative approximation to the exact variances.''", "corrected": "Nonetheless, estimators for $\\operatorname{Var}[\\widehat{y^T_{\\mathrm{HT}}}(d_k)]$ with non-negative bias and estimators for $\\operatorname{Cov}[\\widehat{y^T_{\\mathrm{HT}}}(d_k),\\widehat{y^T_{\\mathrm{HT}}}(d_l)]$ with \\hl{non-positive bias} can always be identified. Together they give a conservative approximation to \\hl{the variance of the estimated contrast}.", "intended_meaning": "The sentence introduces the conservative estimator of the variance of a contrast. In equation (5) the covariance enters with coefficient $-2$. For the estimate of that variance to be conservative, it suffices for the variance estimators to be biased upward and the covariance estimator downward. Section 5 supplies such estimators. Propositions 5.3 and 5.5 state that $\\mathrm{E}[\\widehat{\\operatorname{Cov}}]\\le\\operatorname{Cov}$, and when all joint probabilities of distinct units are positive, the bias of estimator (8) is $-(1/2)\\sum_i(y_i(d_k)-y_i(d_l))^2\\le0$. For three units with independent fair assignments, $y_i(d_k)=1$ and $y_i(d_l)=0$, the covariance is $0$ and estimator (8) has expectation $-3/2$. Equations (5), (8), (10) and (11) and Propositions 5.3, 5.5 and 5.6 are unaffected." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E009", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "complete_derivation", "exhibits": [ { "alt": "Crop of the heading Appendix A: Proofs and the four lines below it. The text reads: A.1. Proof of Proposition 3.1. The replication procedure is equivalent to drawing a random sample without replacement from Omega, with probabilities of selection equal to those which are defined in the randomization plan. The result follows from the strong law of large numbers. A red outline surrounds the words without replacement.", "caption": "Appendix A.1, p. 1938 (PDF p. 27): the proof of Proposition 3.1 in full. The outline marks “without replacement”.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E010.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T20", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T20" } ], "headline": "The replicate assignments are independent draws from the randomization plan, that is, draws with replacement from $\\Omega$; the proof says “without replacement”.", "id": "P013-E010", "location": "Appendix A.1, proof of Proposition 3.1, first sentence, p. 1938 (PDF p. 27)", "pdf_page": 27, "printed_page": "1938", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [ "P013-T20" ], "text": { "as_printed": "A.1. Proof of Proposition 3.1. The replication procedure is equivalent to drawing a random sample \\hl{without replacement} from $\\Omega$, with probabilities of selection equal to those which are defined in the randomization plan. The result follows from the strong law of large numbers.", "corrected": "A.1. Proof of Proposition 3.1. The replication procedure is equivalent to \\hl{independent draws with replacement} from $\\Omega$, \\hl{with each assignment vector drawn according to} the randomization plan. The result follows from the strong law of large numbers.", "intended_meaning": "Section~3 (p.~1917) describes the procedure as producing ``$R$ random replicate $\\mathbf{z}$'s based on the randomization plan'', and Proposition~3.1 lets $R$ tend to infinity. The replicates are independent draws from the design, which are draws with replacement from $\\Omega$. This reading is correct. For independent replicates, the strong law implies that each of the finitely many matrix entries converges, and the additive smoothing does not affect the limit. The binomial expectation used in Appendix~A.3 holds for independent replicates as well. Read literally, the printed phrase cannot hold as $R \\to \\infty$, since $\\Omega$ is finite and a sample without replacement has at most $|\\Omega|$ draws. The sampler of the package that rejects repeated assignments is a separate implementation issue." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E010", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of Proposition 6.2 and the first lines of Section 7. A red box marks the symbol tau with subscript HT, without a hat, in the words 'will tend to cover tau HT of d k, d l at least 100 times 1 minus alpha percent of the time for large N'.", "caption": "Proposition 6.2, p. 1926 (PDF p. 15). The marked coverage target carries the subscript HT of the estimator.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E012.png" }, { "alt": "Crop of the last five lines of Appendix A.6. A red box marks the symbol mu with subscript HT, without a hat, in the difference that is said to converge to zero in probability. Earlier in the same sentence the words 'Consistency of mu hat HT for mu of d k' appear, not marked.", "caption": "Appendix A.6, p. 1942 (PDF p. 31), last lines of the proof of Proposition 6.1. The marked centering term carries the subscript HT; the same sentence names the target $\\mu(d_k)$ correctly.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E012-b.png" }, { "alt": "Crop of the definition of the statistic t and the paragraph after it. Four red boxes mark the symbol tau with subscript HT, without a hat: in the numerators of the two fractions that define t, in the numerator of the small fraction said to be asymptotically normal, and in the words 'will tend to cover tau HT of d k, d l'.", "caption": "Appendix A.7, p. 1943 (PDF p. 32), end of the proof of Proposition 6.2. The four marked terms are the targets subtracted in the statistic $t$ and in the standardized ratio, and the target covered by the intervals.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E012-c.png" }, { "alt": "Crop of the sentence 'Following Fattorini (2006), the following proposition provides guidance on choosing R in terms of a bound on the relative bias for estimating y T of d k', followed by the first line of Proposition 4.1. A red box marks the symbol y T of d k, which is printed with a wide hat.", "caption": "Sentence before Proposition 4.1, p. 1920 (PDF p. 9). The marked target total carries a hat.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E012-d.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T9", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T9" }, { "id": "P013-T18", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T18" } ], "headline": "The targets in Proposition 6.2 and Appendices A.6 and A.7 are the population quantities $\\mu(d_k)$ and $\\tau(d_k,d_l)$, which carry no subscript HT, and the relative bias before Proposition 4.1 is for estimating $y^T(d_k)$, which carries no hat.", "id": "P013-E012", "location": "Proposition 6.2, p. 1926 (PDF p. 15); Appendix A.6, p. 1942 (PDF p. 31); Appendix A.7, p. 1943 (PDF p. 32); sentence before Proposition 4.1, p. 1920 (PDF p. 9)", "pdf_page": 15, "printed_page": "1926", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P013-T18", "P013-T9" ], "text": { "as_printed": "Proposition 6.2, p.~1926: ``will tend to cover $\\tau_{\\hl{\\mathrm{HT}}}(d_k,d_l)$ at least $100(1-\\alpha)\\%$ of the time for large $N$.''\n\nAppendix A.6, p.~1942: ``as this implies that $\\widehat{\\mu_{\\mathrm{HT}}}(d_k)-\\mu_{\\hl{\\mathrm{HT}}}(d_k)\\xrightarrow{p}0$.''\n\nAppendix A.7, p.~1943: ``Define\n\\[\nt=\\frac{\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)-\\tau_{\\hl{\\mathrm{HT}}}(d_k,d_l)}{\\sqrt{\\widehat{\\operatorname{Var}}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}}\n=\\frac{\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)-\\tau_{\\hl{\\mathrm{HT}}}(d_k,d_l)}{\\sqrt{\\operatorname{Var}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}}\n\\left(\\frac{\\operatorname{Var}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}{\\widehat{\\operatorname{Var}}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}\\right)^{1/2}.\n\\]\nUnder Conditions 3, 5, and 6, then, by Chen and Shao [(2004), Theorem 2.7],\n\\[\n\\frac{\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)-\\tau_{\\hl{\\mathrm{HT}}}(d_k,d_l)}{\\sqrt{\\operatorname{Var}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}}\n\\]\nis asymptotically $\\mathrm{N}(0,1)$ \\ldots\\ will tend to cover $\\tau_{\\hl{\\mathrm{HT}}}(d_k,d_l)$ at least $100(1-\\alpha)\\%$ of the time as $N\\to\\infty$.''\n\nSentence before Proposition 4.1, p.~1920: ``a bound on the relative bias for estimating $\\hl{\\widehat{y^T}}(d_k)$.''", "corrected": "Proposition 6.2: ``will tend to cover $\\hl{\\tau}(d_k,d_l)$ at least $100(1-\\alpha)\\%$ of the time for large $N$.''\n\nAppendix A.6: ``as this implies that $\\widehat{\\mu_{\\mathrm{HT}}}(d_k)-\\hl{\\mu}(d_k)\\xrightarrow{p}0$.''\n\nAppendix A.7:\n\\[\nt=\\frac{\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)-\\hl{\\tau}(d_k,d_l)}{\\sqrt{\\widehat{\\operatorname{Var}}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}}\n=\\frac{\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)-\\hl{\\tau}(d_k,d_l)}{\\sqrt{\\operatorname{Var}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}}\n\\left(\\frac{\\operatorname{Var}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}{\\widehat{\\operatorname{Var}}[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)]}\\right)^{1/2},\n\\]\nwith $\\hl{\\tau}(d_k,d_l)$ also in the numerator of the standardized ratio that follows and in ``will tend to cover $\\hl{\\tau}(d_k,d_l)$ at least $100(1-\\alpha)\\%$ of the time as $N\\to\\infty$.''\n\nSentence before Proposition 4.1: ``a bound on the relative bias for estimating $\\hl{y^T}(d_k)$.''\n\nThe hats and the subscript HT remain on the estimators.", "evidence": "This is a check of the pages. The definitions are on pp.~1919--1920. Proposition 6.1 on p.~1925 states consistency with the target $\\tau(d_k,d_l)$, and the sentence of Appendix A.6 that contains the misprint begins ``Consistency of $\\widehat{\\mu_{\\mathrm{HT}}}(d_k)$ for $\\mu(d_k)$''. The hat on p.~1920 was confirmed on an enlargement of the page.", "problem": "Section 4 defines the population mean $\\mu(d_k)=N^{-1}\\sum_{i=1}^N y_i(d_k)$, the total $y^T(d_k)=\\sum_{i=1}^N y_i(d_k)$ and the average causal effect $\\tau(d_k,d_l)=N^{-1}\\sum_{i=1}^N y_i(d_k)-N^{-1}\\sum_{i=1}^N y_i(d_l)$. The subscript HT and the hat belong to their Horvitz--Thompson estimators. The symbols $\\tau_{\\mathrm{HT}}(d_k,d_l)$ and $\\mu_{\\mathrm{HT}}(d_k)$ without hats are not defined in the paper. The first is printed five times where $\\tau(d_k,d_l)$ is meant (once in Proposition 6.2 and four times in Appendix A.7), and the second once where $\\mu(d_k)$ is meant (at the end of Appendix A.6). In the sentence before Proposition 4.1 the target total carries a hat. These are misprints in the names of the targets.", "what_survives": "The estimands, the estimators and the intended statements of consistency, standardization and coverage are unchanged. Only the notation for the targets changes. This correction does not address the substance of Proposition 6.2 or of its proof, which are the subject of \\fid{P013-E001} and \\fid{P013-E007}." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E012", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of the first lines of the notes under Table 1. A red box marks the words 'weighted by exposure probabilities' in the line that begins 'WLS = Least squares'.", "caption": "Notes to Table 1, p. 1933 (PDF p. 22). The marked words describe the weights of the WLS estimator.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E013.png" }, { "alt": "Crop of the notes under Table 2. A red box marks the words 'weighted by exposure probabilities' in the line that begins 'WLS = Least squares'.", "caption": "Notes to Table 2, p. 1936 (PDF p. 25). The same words are marked.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E013-b.png" }, { "alt": "Crop of the display that defines the estimated parameter vector as the minimizer of a weighted sum of squared residuals, with three lines of text above and two below. A red box marks the weight, the fraction 1 over pi i of d k.", "caption": "Section 7.1, p. 1928 (PDF p. 17), the weighted least squares criterion. The marked weight is the inverse of the exposure probability.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E013-c.png" } ], "formal_release": null, "formal_statements": [], "headline": "In the notes to Tables 1 and 2, “weighted by exposure probabilities” means weighted by the inverse exposure probabilities $1/\\pi_i(d_k)$ of the least squares criterion in Section 7.1.", "id": "P013-E013", "location": "Notes to Table 1, p. 1933 (PDF p. 22), and to Table 2, p. 1936 (PDF p. 25); the weights are defined in Section 7.1, p. 1928 (PDF p. 17)", "pdf_page": 22, "printed_page": "1933", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [], "text": { "as_printed": "Notes to Table 1 and to Table 2: ``WLS = Least squares \\hl{weighted by exposure probabilities} with covariate adjustment for network degree and linearized variance estimator.''", "corrected": "In both notes: ``WLS = Least squares weighted by \\hl{inverse} exposure probabilities with covariate adjustment for network degree and linearized variance estimator.''", "intended_meaning": "Section 7.1 defines the weighted least squares fit on p.~1928 by the criterion\n\\[\n\\widehat{\\boldsymbol{\\xi}}(d_k)=\\arg\\min_{\\boldsymbol{\\xi}(d_k)}\\sum_{i:D_i=d_k}\\frac{1}{\\pi_i(d_k)}\\bigl[Y_i-g(\\mathbf{x}_i,\\boldsymbol{\\xi}(d_k))\\bigr]^2 ,\n\\]\nand says that ``weighting by $1/\\pi_i(d_k)$ ensures that the regression proceeds on a sample representative of the population of potential outcomes''. The table notes name the source of the weights and omit the word ``inverse''. The wording of the notes does not show which weights the computations used.\n\nOne of the scripts released with the data of the field experiment weights a regression by inverse exposure probabilities, but that regression belongs to a different analysis. A recalculation on the public data of the field experiment (ICPSR study 37070, version 2) supports the reading for Table 2. The recalculation simulates the exposure probabilities with the seed and the 10,000 draws of the script released with the data and with the sampler that R used before version 3.6. It keeps the eligible students whose five estimated probabilities exceed $0.01$, and it fits one slope on network degree for all exposures. This reconstruction has $N=2{,}050$ students and reproduces all twelve point estimates of Table 2 to three decimals. With inverse weights, the WLS contrasts for $d_{001}$, $d_{011}$, $d_{101}$ and $d_{111}$ against $d_{000}$ are $0.0562$, $0.1563$, $0.2945$ and $0.3063$, which round to the printed $0.056$, $0.156$, $0.295$ and $0.306$. With the probabilities themselves as weights and every other input fixed, the contrasts are $0.0771$, $0.1571$, $0.2524$ and $0.3022$, and none rounds to its printed value. Adding the residual term of expression (14) to the fit with probability weights, I obtain $0.057$, $0.155$, $0.295$ and $0.307$, of which one equals its printed value.\n\nThe $0.01$ cut-off is inferred from a different analysis, whereas the paper specifies nonzero probabilities; the common slope is also a choice of the reconstruction. The script that produced Table 2 and the authors' own files are not public, and ICPSR's notes on the release say that some variables were changed. The recalculation corroborates the inverse weights for Table 2 and does not certify them, and it says nothing about Table 1. Nothing else depends on the wording." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E013", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of five lines of text. The passage reads: Three features emerge from examination of the exposure mapping. First, the exposure mapping reflects four different sources of exposure to the program: being in a school with the program (School), having a peer who was a participation student (Indirect), and being a participating student (Direct). The next sentence begins: Second, only students selected as ``eligible have a nonzero possibility of being in all exposure con-; the crop ends in the middle of the word conditions. Red outlines surround the word four in the second line and the word participation in the third line.", "caption": "Section 10, first paragraph below Figure 2, p. 1935 (PDF p. 24). The outlines mark “four”, where three sources are listed, and “participation”, which should read “participating”.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E014.png" } ], "formal_release": null, "formal_statements": [], "headline": "The sentence lists three sources of exposure to the program, not four, and “participation student” should read “participating student”.", "id": "P013-E014", "location": "Section 10, first paragraph below Figure 2, p. 1935 (PDF p. 24)", "pdf_page": 24, "printed_page": "1935", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "First, the exposure mapping reflects \\hl{four} different sources of exposure to the program: being in a school with the program (School), having a peer who was a \\hl{participation} student (Indirect), and being a participating student (Direct).", "corrected": "First, the exposure mapping reflects \\hl{three} different sources of exposure to the program: being in a school with the program (School), having a peer who was a \\hl{participating} student (Indirect), and being a participating student (Direct).", "evidence": "The evidence is a check of the pages: the sentence on p.~1935 and the display of the exposure mapping on p.~1934.", "problem": "The sentence announces four sources and lists three: School, Indirect and Direct. The exposure mapping on p.~1934 is built from the three corresponding indicators, which generate five exposure conditions, four of them other than no exposure; ``participation student'' is a slip for ``participating student''.", "what_survives": "The exposure mapping, the five exposure conditions and every estimate of Section~10 are unaffected; two words change." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E014", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Listing of lines 233 to 243 in monospaced type with line numbers. A red box marks line 234, 'meanse is assigned function(x) mean(max(0,x) to the power .5)'. The lines below it call meanse on six vectors of variance estimates.", "caption": "Package interference, file misc/ReplicationAoAS.R, lines 233–243, typeset from the source file with line numbers added. The marked line defines the function; the same definition is repeated at line 381.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E016-b.png" }, { "alt": "Crop of the title and column headings of Table 1 and its three rows for the HT estimator. A red box marks the column heading 'Mean S.E.'. The entries of that column in the three rows shown are 0.05, 0.19 and 0.28.", "caption": "Table 1, p. 1933 (PDF p. 22), heading and first rows. The marked column is the counterpart in the paper of the script’s MeanSE output; no entry of the column has been shown wrong.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E016.png" } ], "formal_release": null, "formal_statements": [], "headline": "The function labelled “Mean SE” in the replication script distributed with the interference package returns the largest standard error across replications, not their mean.", "id": "P013-E016", "location": "interference package, misc/ReplicationAoAS.R, lines 234 and 381; counterpart in the paper: Table 1, column “Mean S.E.”, p. 1933 (PDF p. 22)", "pdf_page": 22, "printed_page": "1933", "reaches": [ { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\texttt{misc/ReplicationAoAS.R}, lines 233--234, and again lines 380--381:\n\n\\texttt{\\#\\# Mean SE}\n\n$\\texttt{meanse <- function(x) mean(}\\hl{\\texttt{max}}\\texttt{(0,x)\\textasciicircum.5)}$", "corrected": "At both lines, with the maximum taken element by element:\n\n\\texttt{meanse <- function(x) mean(sqrt(\\hl{pmax}(0,x)))}\n\nFor variance estimates $v_1,\\dots,v_B$ this returns\n\\[\n\\frac{1}{B}\\sum_{b=1}^B\\sqrt{\\max(0,v_b)} ,\n\\]\nand negative variance estimates are then set to zero separately in each replication, a rule that should be stated. For nonnegative variance estimates \\texttt{mean(sqrt(x))} gives the same value.", "evidence": "This finding rests on an exact counterexample and on running the code. For two replications with variance estimates $(1,4)$, the function returns $\\sqrt{4}=2$, and the mean standard error is $(1+2)/2=3/2$. For $(1,4,9,100)$ the values are $10$ and $4$. Both definitions in the script (lines 234 and 381) were run with their six calls, for three Horvitz--Thompson and three Hajek contrasts, on the input $(1,4)$: each returned $2$ in all six cells, and with \\texttt{pmax} in place of \\texttt{max} each returned $1.5$.", "problem": "The script calls this function on a vector of variance estimates, one for each simulated replication, and writes the result to the column \\texttt{MeanSE} of its output table. In R, \\texttt{max(0,x)} is a single number, the largest of $0$ and the elements of \\texttt{x}. The function then takes the square root of that number and averages the one value it has. For a nonempty vector of finite variance estimates $v_1,\\dots,v_B$ it returns $\\max_b\\sqrt{\\max(0,v_b)}$, the largest standard error across replications after negative variance estimates are set to zero. The heading ``Mean S.E.'' of Table 1 and the script's own label call for the mean, $B^{-1}\\sum_{b=1}^B\\sqrt{\\max(0,v_b)}$. The value returned is never smaller than this mean, and equals it only when all the standard errors are equal.", "what_survives": "The bias, standard deviation, root mean square error and coverage columns of the script do not call this function, and the estimation functions of the package do not use it. The estimators and proofs of the paper are unaffected. No entry of the Mean S.E. column of the published Table 1 is shown to be wrong or shown to be right. The script is a later implementation, its inputs are not available, and it is not established that it produced the table." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E016", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Listing of source code with line numbers. In line 71 of the replication script a red box marks the argument hop=1 in the call to estimates(). In lines 103 and 106 red boxes mark the expressions all_estimates$var_tau_ht_const_eff and all_estimates$var_tau_ht_max. Below, lines 165 to 174 of R/estimates.R show the arguments of estimates(): obs_exposure, obs_outcome, obs_prob_exposure, n_var_permutations, effect_estimators, variance_estimators, control_condition and treated_conditions. There is no argument named hop.", "caption": "Source code of the package, not a page of the article: misc/ReplicationAoAS.R, lines 71 and 103–107, above the signature of estimates() in R/estimates.R, lines 165–174. The marked argument hop=1 is not among the arguments of the function, and the two marked fields are not returned under its default settings.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E017.png" } ], "formal_release": null, "formal_statements": [], "headline": "The replication script shipped with the package calls estimates() with an argument hop that the function does not accept, which stops the script at line 71; the call should drop hop, name the control exposure and request the variance outputs that the script stores.", "id": "P013-E017", "location": "R package interference (commit c3fc7b3): misc/ReplicationAoAS.R, lines 71 and 103–107; R/estimates.R, lines 165–174. The code is not printed in the article.", "pdf_page": null, "printed_page": null, "reaches": [ { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "result_fails", "effect_text": "a result or code path there relies on it and fails with it", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\texttt{misc/ReplicationAoAS.R}, line 71 (trailing comment omitted):\n\n\\texttt{all\\_estimates <- estimates(exposure, obs\\_outcome, obs\\_prob\\_exposure, n\\_var\\_permutations = 50, \\hl{hop=1})}\n\nLines 103 and 106 (trailing comments omitted):\n\n\\texttt{var\\_estimate\\_ht\\_const\\_eff <- as.list(\\hl{all\\_estimates\\$var\\_tau\\_ht\\_const\\_eff})}\n\n\\texttt{var\\_estimate\\_ht\\_max <- as.list(\\hl{all\\_estimates\\$var\\_tau\\_ht\\_max})}\n\n\\texttt{R/estimates.R}, lines 165--174 (the signature, joined on one line):\n\n\\texttt{estimates <- function(obs\\_exposure, obs\\_outcome, obs\\_prob\\_exposure, n\\_var\\_permutations = 10, effect\\_estimators = c('hajek', 'horvitz-thompson'), variance\\_estimators = c('hajek', 'horvitz-thompson'), control\\_condition=NULL, treated\\_conditions=NULL)}", "corrected": "\\texttt{all\\_estimates <- estimates(exposure, obs\\_outcome, obs\\_prob\\_exposure, n\\_var\\_permutations = 50, \\hl{control\\_condition = 'no', variance\\_estimators = c('hajek', 'horvitz-thompson', 'max\\_ht\\_const')})}\n\nThe choice \\texttt{'max\\_ht\\_const'} also requests the constant-effect variance estimator. Alternatively, remove \\texttt{hop} and delete lines 97--107, which extract and store the derived variances that the later table statistics do not use.", "evidence": "The evidence is an exact check of the code on a small example. When I run line 71 as shipped against the function in the same package, I obtain the error quoted above. On a network of six units whose exposure probabilities are known exactly, the call without \\texttt{hop} returns variance vectors of length zero, and the script fails when it writes them to a CSV file (R 4.4.2, \\texttt{data.table} 1.18.2.1). The corrected call below passes the same steps. I did not run the whole script, because the Add Health files that it reads are not available.", "problem": "The function \\texttt{estimates()} in the same package has no argument \\texttt{hop}. R rejects the call on line 71 with the error ``unused argument (hop = 1)'' before any estimator is computed, whatever the data. It is not enough to remove \\texttt{hop}. With the default \\texttt{control\\_condition=NULL}, the outputs \\texttt{var\\_tau\\_ht} and \\texttt{var\\_tau\\_h} are empty. The default \\texttt{variance\\_estimators} does not produce \\texttt{var\\_tau\\_ht\\_const\\_eff} or \\texttt{var\\_tau\\_ht\\_max}, which lines 103--107 store, and the script then fails when it writes its results to a CSV file.", "what_survives": "The finding concerns only whether a later script runs. It bears on no statement of the article, and no entry of Table 1 is shown to be wrong. The variance fields that the script fails to store are not used in its later table statistics, which recompute the variances of the effects from the variances and covariances of the outcome totals. The check does not establish which version of the code produced the published Table 1." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E017", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Listing of source code with line numbers. Lines 109 to 112 of R/estimates.R say that yT_ht contains the Horvitz-Thompson estimator of the total as derived in Equation 1 of Aronow and Samii (2017). Lines 216 to 218 compute yT_ht with rowSums. A red box marks line 219, yT_ht[rowSums(!is.na(obs_outcome_by_exposure)) == 0] <- NA. In the replication script, line 131 is a comment saying that the aggregation removes rows for which the estimator yT is NA, and in line 136 red boxes mark the four subsetting terms [!is.na(yT_ht_dir_ind1)].", "caption": "Source code of the package, not a page of the article: R/estimates.R, lines 109–112 and 216–219, and misc/ReplicationAoAS.R, lines 131 and 136. The marked line 219 replaces the total by NA when no unit has the exposure, and the marked terms of line 136 remove the schools with a missing total from both sums of each ratio.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E018-code.png" }, { "alt": "Crop of the paragraph beginning 'The number of units in the population, N, is fixed' and of equation (1). Red boxes mark the words 'The number of units in the population, N, is fixed' and the right-hand side of equation (1), the sum over i from 1 to N of I(D_i = d_k) Y_i divided by pi_i(d_k).", "caption": "Equation (1) and the paragraph before it, p. 1919 (PDF p. 8): the estimator that the package documents yT_ht to be. The marked sum equals zero when no unit has exposure $d_k$, and the marked words fix the population. The passage is correct.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E018.png" } ], "formal_release": null, "formal_statements": [], "headline": "When no unit realizes an exposure, the package returns a missing value for the Horvitz–Thompson total that equation (1) defines as zero, and the first aggregation of the replication script then drops those schools; the total should stay zero and the aggregation should run over the fixed population.", "id": "P013-E018", "location": "R package interference (commit c3fc7b3): R/estimates.R, lines 216–219, documented on lines 109–112; misc/ReplicationAoAS.R, lines 131 and 136. Counterpart in the article: equation (1), p. 1919 (PDF p. 8).", "pdf_page": null, "printed_page": null, "reaches": [ { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": "P044-E006", "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": "P045-E004", "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "implementation_differs_from_paper", "effect_text": "the code implements something other than the printed method", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [ { "id": "P044-E006", "url": "https://pmaronow.github.io/llm-audit/works/P044/#P044-E006", "work_id": "P044" }, { "id": "P045-E004", "url": "https://pmaronow.github.io/llm-audit/works/P045/#P045-E004", "work_id": "P045" } ], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "Equation (1), p.~1919:\n\\[\n\\widehat{y^T_{\\mathrm{HT}}}(d_k)=\\sum_{i=1}^{N}\\mathbf{I}(D_i=d_k)\\frac{Y_i}{\\pi_i(d_k)}.\n\\]\n\n\\texttt{R/estimates.R}, lines 216--219:\n\n\\texttt{yT\\_ht <- rowSums(obs\\_outcome\\_by\\_exposure / obs\\_prob\\_exposure\\_individual\\_kk, na.rm = T)}\n\n\\texttt{\\hl{yT\\_ht[rowSums(!is.na(obs\\_outcome\\_by\\_exposure)) == 0] <- NA}}\n\n\\texttt{misc/ReplicationAoAS.R}, line 136 (beginning):\n\n\\texttt{tau11\\_ht <- estimates\\_all\\_schools[, sum(yT\\_ht\\_dir\\_ind1[\\hl{!is.na(yT\\_ht\\_dir\\_ind1)}])/\\ldots}", "corrected": "\\texttt{R/estimates.R}, lines 216--219, \\hl{with line 219 deleted}:\n\n\\texttt{yT\\_ht <- rowSums(obs\\_outcome\\_by\\_exposure / obs\\_prob\\_exposure\\_individual\\_kk, na.rm = T)}\n\nIn the replication script, aggregate the Horvitz--Thompson contrasts, and the effects they are compared with, over all schools with the fixed population in the denominator, and do not delete schools according to the exposures realized. Keep the Horvitz--Thompson variance and covariance expressions as defined when an exposure is empty, and treat an empty Hajek denominator separately.", "evidence": "The evidence is an exact counterexample, which I confirmed by running the unchanged functions and aggregation lines on every assignment. The example has two schools and 12 units, the four exposures of Section 9, and a strictly positive probability of every exposure for every unit. The average effect over the fixed population is $4/3$, and the estimator built from equation (1) has expectation $4/3$. The estimator that deletes schools as the script does has expectation $14/9$, which is a bias of $2/9$. The population to which it refers has average effect $4/3$ or $1$, depending on the assignment.", "problem": "The documentation of \\texttt{estimates()} identifies its output \\texttt{yT\\_ht} with equation (1). Equation (1) is a sum over the units with $D_i=d_k$. It equals zero when no unit has that exposure, and the unbiasedness stated after it counts such assignments with the value zero. Line 219 replaces that zero by \\texttt{NA}, and the missing value passes to the contrasts. The first aggregation of the replication script, from line 136, then removes the affected schools. The denominator of the estimate, and the population to which it refers, then change from one assignment to the next.", "what_survives": "Equations (1) and (3), Lemma 4.1 and Proposition 4.2 are correct as printed, and line 219 changes no point estimate when every exposure is realized. The package uses a missing value for an empty Hajek denominator; the article adopts the convention $0/0=0$ on p.~1922. The available files do not identify the code that produced Table 1, and no entry of Table 1 or Table 2 is shown to be wrong." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E018", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of the top of Table 1, titled Results from school friends' network simulated experiment. The column headings are Estimator, Estimand, Bias, S.D., RMSE, Mean S.E., 95 percent CI Coverage and 90 percent CI Coverage. Red outlines surround the two headings 95 percent CI Coverage and 90 percent CI Coverage. Below the headings are the three HT rows. For tau(d01, d00) the entries are 0.00, 0.04, 0.04, 0.05, 0.960 and 0.924. For tau(d10, d00) they are 0.00, 0.10, 0.10, 0.19, 0.986 and 0.970. For tau(d11, d00) they are 0.00, 0.13, 0.13, 0.28, 0.990 and 0.970.", "caption": "Table 1, p. 1933 (PDF p. 22): the title, the column headings and the three HT rows. The outlines mark the two coverage columns, which the coverage function of the script is written to reproduce.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-E019.png" } ], "formal_release": null, "formal_statements": [], "headline": "The coverage function of the replication script takes the square root of the absolute value of each variance estimate and leaves missing comparisons out of the denominator, which departs from the interval of Proposition 6.2 when a variance estimate is negative or a comparison is missing; no entry of the published Table 1 is shown to be affected.", "id": "P013-E019", "location": "Replication script misc/ReplicationAoAS.R of the R package interference, lines 247, 261, 394 and 408; the paper’s counterparts are Table 1, columns “95% CI Coverage” and “90% CI Coverage”, p. 1933 (PDF p. 22), and the interval of Proposition 6.2, p. 1926 (PDF p. 15)", "pdf_page": null, "printed_page": null, "reaches": [ { "effect": "implementation_differs_from_paper", "effect_text": "the code implements something other than the printed method", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "implementation_differs_from_paper", "effect_text": "the code implements something other than the printed method", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "implementation_differs_from_paper", "effect_text": "the code implements something other than the printed method", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\emph{Script, lines 247, 261, 394 and 408 (one line of code, broken here after the argument list).}\n\\[\n\\begin{aligned}\n&\\texttt{coverage <- function(est,var,tau)}\\\\\n&\\qquad\\texttt{mean(abs(est-tau) < alphanorm*}\\hl{\\texttt{abs(var)}}\\texttt{\\textasciicircum.5,}\\hl{\\texttt{na.rm=TRUE}}\\texttt{)}\n\\end{aligned}\n\\]\n\n\\emph{Proposition~6.2, p.~1926.} Given Conditions 3, 5, and 6, Wald-type intervals constructed as\n\\[\n\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l) \\pm z_{1-\\alpha/2}\\sqrt{\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]}\n\\]\nwill tend to cover $\\tau_{\\mathrm{HT}}(d_k,d_l)$ at least $100(1-\\alpha)\\%$ of the time for large $N$.\n\n\\emph{Table~1, note, p.~1933.} CI = Normal approximation confidence interval.", "corrected": "To report the coverage of the interval of Proposition~6.2 as printed, the function should \\hl{flag} each replication whose variance estimate is negative or not finite, in place of taking its absolute value, and flag \\hl{any other undefined coverage comparison}. The script should then report the number of replications, the number evaluated, the number with a negative variance estimate, and the number missing for another reason. A rate over the evaluated replications is a conditional rate and should be labelled as one; a rate over all replications needs a stated rule for those in which the interval is undefined. Whichever variance rule is used, state the endpoint convention: use \\hl{\\texttt{<=} for a closed interval}, or specify an open interval if \\texttt{<} is retained.\n\nAlternatively, the absolute value may be kept, if the script states that its interval is\n\\[\n\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l) \\pm z_{1-\\alpha/2}\\sqrt{\\hl{\\bigl|\\widehat{\\operatorname{Var}}[\\widehat{\\tau}_{\\mathrm{HT}}(d_k,d_l)]\\bigr|}} ,\n\\]\nreports how often the absolute value changes the estimate, and reports the denominator. Removing \\texttt{abs()} and keeping \\texttt{na.rm=TRUE} is not sufficient, since negative estimates would then be dropped without notice. Truncation at zero is a third possible rule; I do not select one here.", "evidence": "The evidence is an exact counterexample and an evaluation of the script's own definitions. The example has four units and two exposures with $D_i = Z_i$, where $1$ stands for $d_k$ and $0$ for $d_l$. The pair $(Z_1, Z_2)$ equals $(0,0)$ or $(1,1)$ with probability $1/100$ each, and $(0,1)$ or $(1,0)$ with probability $49/100$ each; $Z_3$ and $Z_4$ are independent fair coins, independent of $(Z_1, Z_2)$. The potential outcomes are $(y_1(d_l), \\ldots, y_4(d_l)) = (1, 1, 0, 0)$ and $(y_1(d_k), \\ldots, y_4(d_k)) = (3/2, 3/2, 0, 0)$, and the average effect is $\\tau = 1/4$. Every exposure probability is $1/2$, and every joint exposure probability of two distinct units is positive. Computing $\\widehat\\tau$ and the variance estimate (11) for each of the 16 assignments, I find three cases.\n\\begin{audittable}{lrrr}{The estimate and the variance estimate (11) in the four-unit example, by the assignment of the first two units.}\n$(Z_1, Z_2)$ & Probability & $\\widehat\\tau$ & $\\widehat V$ \\\\\n$(1,1)$ & $1/100$ & $3/2$ & $-207/8$ \\\\\n$(0,0)$ & $1/100$ & $-1$ & $-23/2$ \\\\\n$(0,1)$ or $(1,0)$ & $49/50$ & $1/4$ & $349/784$ \\\\\n\\end{audittable}\nIn the first two cases the squared error is $25/16$. With $z = 2$, the squared radius $z^2 |\\widehat V|$ used by the function is $207/2$ and $46$. Both cases count as covered, and the coverage over the design is $1$. If the variance estimate is truncated at zero instead, both cases fail and the coverage is $49/50$. The example agrees with Proposition~5.6, since $\\mathbb{E}[\\widehat V] = 1/16$ and $\\operatorname{Var}(\\widehat\\tau) = 1/32$.\n\nOn the assignment $(1,1,0,0)$, the estimation functions of the package return $\\widehat\\tau = 1.5$ and $\\widehat V = -25.875$, and the square root of $\\widehat V$ is not a number. Each of the four copies of the function counts this assignment as covered at both critical values of the script. For the second departure, I take the inputs $\\widehat\\tau = (1/2, 1/2, 1/2)$, $\\tau = 0$ and $\\widehat V = (1, -1, \\mathrm{NA})$. The comparisons are TRUE, TRUE and NA, and the function returns $1$ from two of the three replications; counting the missing one as a failure gives $2/3$. The four-unit example has no interference, a special case that the setting of the paper allows. It is not the simulation design of Section~9.", "problem": "The script says in its first line that it replicates Table~1, and it labels the output of this function as 95\\% and 90\\% coverage. Let $\\widehat V$ be the variance estimate in one replication of the simulated experiment, $\\widehat\\tau$ its estimate and $z$ the critical value. The function returns the share of replications such that $|\\widehat\\tau - \\tau| < z\\sqrt{|\\widehat V|}$, and it takes this share over the replications whose comparison is not missing. Two departures are established here. First, the variance estimator (11) can be negative, and the printed interval is then undefined; the function replaces $\\widehat V$ by $|\\widehat V|$ and evaluates the replication. Second, a replication whose comparison is missing is removed from the numerator and from the denominator. The function then returns a rate among the evaluated replications and not a rate among all replications. The script reports neither how often these cases occur nor the denominator.\n\nThe strict \\texttt{<} also excludes exact endpoint matches, even when the variance estimate is nonnegative and the comparison is defined. The printed $\\pm$ formula leaves the endpoint convention unstated, so I do not treat this alone as a further confirmed error.\n\nTaking the absolute value is not in itself invalid. Since $|\\widehat V| \\ge \\widehat V$, it defines a different variance estimator, which is nonnegative and remains conservative in expectation. The defect is that the function is presented as the coverage of the printed interval, while it computes the coverage of a modified interval among the evaluated replications. Its effect on a reported number is not established. The network and outcome files that the script reads are not available. No negative variance estimate or missing comparison is shown to have occurred in the published simulation. Nothing identifies this script as the code that produced the published Table~1: the script computes the HT and Hajek rows only, and the table reports five estimators.", "what_survives": "The function feeds only the two coverage columns of the two tables that the script writes. The point estimates, the bias, the standard deviation and the root mean square error of the script do not pass through it. Its ``Mean SE'' function is a separate matter, recorded as \\fid{P013-E016}. No entry of the published Table~1 is shown to be wrong, and no replacement value is available. The estimands, the estimators, Proposition~5.6 and the asymptotic results of the paper are unaffected." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-E019", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "complete_derivation", "exhibits": [ { "alt": "Crop showing expression (8) on three lines: the first line is a double sum of indicator products divided by pi_ij(d_k, d_l) times Y_i over pi_i(d_k) times Y_j over pi_j(d_l); the second line is the factor pi_ij(d_k, d_l) minus pi_i(d_k) pi_j(d_l); the third line subtracts a sum over i of two terms in Y_i squared. Below are Proposition 5.3 and the sentence beginning 'A proof for Proposition 5.3 follows from noting that the term on the second line in expression (8)', in which red boxes mark the words 'second line'.", "caption": "Expression (8), Proposition 5.3 and the sentence after it, p. 1923 (PDF p. 12). Expression (8) is set on three lines; the marked words point to its second line, and the term meant is on its third.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N001.png" } ], "formal_release": null, "formal_statements": [], "headline": "“The term on the second line in expression (8)” should read “the term on the last line in expression (8)”.", "id": "P013-N001", "location": "Sentence after Proposition 5.3, p. 1923 (PDF p. 12)", "pdf_page": 12, "printed_page": "1923", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "``A proof for Proposition 5.3 follows from noting that the term on the \\hl{second} line in expression (8) has expected value less than or equal to the quantity in the last line of expression (6), again via Young's inequality; see Aronow and Samii [(2013), Proposition 2, reproduced in the Appendix below] for greater detail.''", "corrected": "``A proof for Proposition 5.3 follows from noting that the term on the \\hl{last} line in expression (8) has expected value less than or equal to the quantity in the last line of expression (6), again via Young's inequality.''", "evidence": "The evidence is a check of the page and a short derivation. The last line of (8) has expectation $-(1/2)\\sum_i[y_i(d_k)^2+y_i(d_l)^2]$, and the last line of (6) is $-\\sum_i y_i(d_k)y_i(d_l)$. Their difference is $-(1/2)\\sum_i[y_i(d_k)-y_i(d_l)]^2\\le0$, which is the inequality that the sentence states.", "problem": "Expression (8) is set on three lines. Its second line is the factor $\\times[\\pi_{ij}(d_k,d_l)-\\pi_i(d_k)\\pi_j(d_l)]$ of the double sum. The term that the sentence means is the sum on the third and last line,\n\\[\n-\\sum_{i\\in U}\\left[\\frac{\\mathbf{I}(D_i=d_k)Y_i^2}{2\\pi_i(d_k)}+\\frac{\\mathbf{I}(D_i=d_l)Y_i^2}{2\\pi_i(d_l)}\\right].\n\\]", "what_survives": "Proposition 5.3 and its proof are unaffected. Only the reference to the line changes." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N001", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "complete_derivation", "exhibits": [ { "alt": "Crop of Corollary 8.1 and the line after it. A red box marks the weight w with subscript i, z prime in the second inner sum, which runs over assignments z prime with f of z prime and theta i equal to d l.", "caption": "Corollary 8.1, p. 1930 (PDF p. 19). The marked weight belongs to the sum over assignments that induce exposure $d_l$.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N003.png" }, { "alt": "Crop of expression (16) and the three lines after it. A red box marks the words 'w i, z equals p z divided by pi i of d k' after the word 'where'.", "caption": "Proposition 8.1, p. 1929 (PDF p. 18), expression (16) and the definition of the weights. The marked definition has $\\pi_i(d_k)$ in the denominator.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N003-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T17", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T17" } ], "headline": "In Corollary 8.1 the weight in the second sum is $w_{i,\\mathbf{z}'}=p_{\\mathbf{z}'}/\\pi_i(d_l)$: each weight divides by the probability of the exposure that its assignment induces.", "id": "P013-N003", "location": "Corollary 8.1, p. 1930 (PDF p. 19); definition of the weights after expression (16), p. 1929 (PDF p. 18)", "pdf_page": 19, "printed_page": "1930", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [ "P013-T17" ], "text": { "as_printed": "Proposition 8.1, p.~1929: ``where $w_{i,\\mathbf{z}}=p_{\\mathbf{z}}/\\hl{\\pi_i(d_k)}$.''\n\nCorollary 8.1, p.~1930: Under Condition 7,\n\\[\n\\mathrm{E}\\bigl[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)\\bigr]=\\frac{1}{N}\\sum_{i=1}^N\\left[\\sum_{\\mathbf{z}:f(\\mathbf{z},\\theta_i)=d_k}w_{i,\\mathbf{z}}\\,y_i^r(\\mathbf{z})-\\sum_{\\mathbf{z}':f(\\mathbf{z}',\\theta_i)=d_l}\\hl{w_{i,\\mathbf{z}'}}\\,y_i^r(\\mathbf{z}')\\right].\n\\]", "corrected": "Proposition 8.1: ``where $w_{i,\\mathbf{z}}=p_{\\mathbf{z}}/\\hl{\\pi_i(f(\\mathbf{z},\\theta_i))}$'', for assignments $\\mathbf{z}$ whose induced exposure $f(\\mathbf{z},\\theta_i)$ has positive probability for unit $i$. With this definition, Corollary 8.1 states that\n\\[\n\\mathrm{E}\\bigl[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_l)\\bigr]=\\frac{1}{N}\\sum_{i=1}^N\\left[\\sum_{\\mathbf{z}:f(\\mathbf{z},\\theta_i)=d_k}\\frac{p_{\\mathbf{z}}}{\\pi_i(d_k)}\\,y_i^r(\\mathbf{z})-\\sum_{\\mathbf{z}':f(\\mathbf{z}',\\theta_i)=d_l}\\frac{p_{\\mathbf{z}'}}{\\hl{\\pi_i(d_l)}}\\,y_i^r(\\mathbf{z}')\\right].\n\\]", "intended_meaning": "Proposition 8.1 concerns one exposure $d_k$, and its weight $p_{\\mathbf{z}}/\\pi_i(d_k)$ is the probability of the assignment $\\mathbf{z}$ given that unit $i$ receives exposure $d_k$. Corollary 8.1 applies the proposition once to $d_k$ and once to $d_l$, and the weight in its second sum is $p_{\\mathbf{z}'}/\\pi_i(d_l)$. This exposure-specific reading is correct. By Condition 7, for any exposure $d$ such that $\\pi_i(d)>0$, I have\n\\[\n\\mathrm{E}\\left[\\mathbf{I}(D_i=d)\\frac{Y_i}{\\pi_i(d)}\\right]=\\sum_{\\mathbf{z}:f(\\mathbf{z},\\theta_i)=d}\\frac{p_{\\mathbf{z}}}{\\pi_i(d)}\\,y_i^r(\\mathbf{z}) .\n\\]\nSubtracting this identity for $d_l$ from the identity for $d_k$, summing over $i$ and dividing by $N$, I obtain the corollary. If the printed definition is kept literally in the second sum, the formula is false in general. For one unit and four assignments with probabilities $1/8$, $1/8$, $3/8$, $3/8$ and outcomes $0$, $1$, $2$, $3$, of which the first two induce $d_k$ and the last two $d_l$, the bracket equals $1/2-5/2=-2$ with the weights as intended and $1/2-15/2=-7$ with the denominator $\\pi_i(d_k)$ in both sums. Proposition 8.1, the corollary as intended and the derivation in Appendix A.8 are unaffected." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N003", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "historical_lean_attribution", "exhibits": [ { "alt": "Crop of the last eight lines of the paragraph that begins 'This experiment is repeated independently', with the first two lines of the next paragraph. One red box marks the number 0.00067 in the sentence about the student with degree of 39. Two more red boxes mark the words 'there were no cases with probabilities less than 0.01 for either the indirect- or no-exposure conditions', which run over two lines.", "caption": "Section 9, p. 1931 (PDF p. 20). The two marked statements cannot both describe exposure probabilities of the design.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N004.png" }, { "alt": "Crop of six lines of the first paragraph of Section 9. Red boxes mark the words 'randomly assigned without replacement and with uniform probability to 1/10 of individuals in a school network', which run over two lines.", "caption": "Section 9, p. 1930 (PDF p. 19). The marked words state the design of the simulation.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N004-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T11", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T11" } ], "headline": "The value 0.00067 for the student of degree 39 and the statement that no student has a no-exposure probability below 0.01 cannot both be exposure probabilities of the stated design: at least one of them is inaccurate, or both are estimates from a finite number of simulated assignments.", "historical_evidence_type": "lean_proof", "id": "P013-N004", "location": "Section 9, p. 1931 (PDF p. 20), paragraph beginning “This experiment is repeated independently”; the design is stated on p. 1930 (PDF p. 19)", "pdf_page": 20, "printed_page": "1931", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P013-T11" ], "text": { "as_printed": "p.~1930: ``We simulate experiments in which a treatment, $\\mathbf{Z}$, is randomly assigned without replacement and with uniform probability to 1/10 of individuals in a school network.''\n\np.~1931: ``To give an idea of the range of exposure probabilities, for the student with degree of 39, the probability of isolated direct exposure was \\hl{0.00067}. In Figure 3 of the Appendix, we display the cumulative distribution functions for the four exposure probabilities. About 3\\% of students have an exposure probability of less than 0.01 for the direct + indirect exposure condition, 0.5\\% for isolated direct exposure, and then there were \\hl{no cases with probabilities less than 0.01 for either the indirect- or no-exposure conditions}.''", "corrected": "For a student with 39 friends in a school of $N$ students, of whom $n\\ge 1$ are treated uniformly at random without replacement with $N\\le 10n+5$, the probabilities of the design satisfy\n\\[\n\\pi_i(d_{00})=\\frac{N-39-n}{n}\\,\\pi_i(d_{10}),\\qquad\\text{and hence}\\qquad \\pi_i(d_{00})<9\\,\\pi_i(d_{10})\\quad\\text{when } \\pi_i(d_{10})>0 .\n\\]\nThe passage can be corrected in one of two ways.\n\nIf the probabilities were estimated by simulation: ``for the student with degree of 39, the \\hl{estimated} probability of isolated direct exposure was 0.00067 \\ldots\\ there were no cases with \\hl{estimated} probabilities less than 0.01 for either the indirect- or no-exposure conditions'', together with the estimator used, the number $R$ of simulated assignments and the precision of the estimates, and with Figure 3 labelled as showing estimated probabilities. A statement about estimated probabilities does not show that no probability of the design is below 0.01.\n\nIf probabilities of the design are meant, the two binomial expressions above are to be evaluated for the student's school, and at least one of the two statements replaced. I cannot give the replacement values, because the size of the school and the number treated are not in the available materials.", "evidence": "This finding rests on a derivation by counting. Let the student's school have $N$ students, of whom $n$ are treated, every set of $n$ being equally likely, and let $p$ and $q$ be the student's probabilities of $d_{10}$ and $d_{00}$. There are $N-40$ students who are neither the student nor one of the 39 friends, and\n\\[\np=\\frac{\\binom{N-40}{n-1}}{\\binom{N}{n}},\\qquad q=\\frac{\\binom{N-40}{n}}{\\binom{N}{n}},\\qquad \\frac{q}{p}=\\frac{N-39-n}{n}\\quad\\text{when }p>0 .\n\\]\nIf $n=N/10$, the ratio is $9-39/n<9$. If $n$ is $N/10$ rounded to the nearest integer, then $N\\le 10n+5$ and the ratio is at most $9-34/n<9$; the ratio is also below 9 if the number treated is rounded down or up. If 0.00067 is a rounded value, then $p\\le 0.000675$, and I obtain $q\\le 9p\\le 0.006075<0.01$. Conversely, $q\\ge 0.01$ implies that $p\\ge 1/900>0.000675$. In an exhaustive count for $N=44$ and $n=4$ (135,751 assignments), I find $p=4/135751$ and $q=1/135751$, in agreement with the formula. The counts, the identity $q=((N-39-n)/n)\\,p$ and the bounds have Lean proofs for the uniform design, with $1\\le n\\le N-39$ and, for the bounds, $N\\le10n+5$ (\\fid{P013-T11}).\n\nEstimates behave differently. With $R=1500$ and no simulated assignment placing the student in $d_{10}$, the estimate of $p$ is $1/1501=0.000666\\ldots$, which rounds to 0.00067, and there exist 1500 admissible assignments in a school with $N=300$ and $n=30$ for which the estimates are $1/1501$ and $21/1501>0.01$. The replication script distributed with the \\texttt{interference} package uses 1500 assignments and this estimator. It is not established that this script produced the published numbers; it describes 139 networks, and the paper 144 school classes.", "problem": "In the exposure mapping of Section 9, a student is in isolated direct exposure $d_{10}$ when the student is treated and no friend is, and in no exposure $d_{00}$ when neither the student nor any friend is treated. For a student with 39 friends these two probabilities are tied together under the stated design: the second is less than nine times the first. If the probability of $d_{10}$ is 0.00067, then the probability of $d_{00}$ for the same student is below 0.01, and the paragraph says that there were no such cases. The paragraph does not say how the probabilities were computed. Section 4 (pp.~1919--1920) describes estimates of the form $(X_i+1)/(R+1)$ from $R$ simulated assignments, and two such estimates can agree with both printed statements. Which reading applies to the published numbers is not established.", "what_survives": "The theoretical results are stated for given exposure probabilities and do not depend on these two descriptive statements, and no later result uses the value 0.00067. The entries of Table 1 were not recomputed, and this finding neither shows them wrong nor confirms them. Whether the published simulation used estimated probabilities remains unresolved. The summaries of 3\\% and 0.5\\% are not in question individually, and the application of Section 10 uses another design and exposure mapping." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N004", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of the first three items of the bulleted list of estimators. A red box marks the words 'The Hajek ratio estimator given in expression (15)' in the second item.", "caption": "Section 9, p. 1932 (PDF p. 21), list of the estimators of the simulation. The marked item names expression (15) as the Hajek estimator.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N006.png" }, { "alt": "Crop of expression (15) with two lines of text above and two below. A red box marks the formula: mu hat H of d k equals the sum over i from 1 to N of the indicator of D i equal to d k times Y i over pi i of d k, divided by the sum over i from 1 to N of the indicator times 1 over pi i of d k.", "caption": "Expression (15), p. 1928 (PDF p. 17). The marked ratio has sums over the whole population in its numerator and its denominator.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N006-b.png" }, { "alt": "Listing of lines 225 to 235 in monospaced type with line numbers. One red box marks lines 226 to 228, which assign to mu_h the Horvitz-Thompson total divided by the row sums of exposure indicators over exposure probabilities. A second red box marks line 231, 'yT_h is assigned mu_h times N'.", "caption": "Package interference, file R/estimates.R, lines 225–235, typeset from the source file with line numbers added. The marked lines compute a school’s Hajek ratio and multiply it by the size of the school.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N006-c.png" }, { "alt": "Listing of lines 143 to 152 in monospaced type with line numbers. One red box marks line 143, which assigns to tau01_h the sum of yT_h_ind1 over the sum of N minus the sum of yT_h_no over the sum of N. A second red box marks lines 150 to 152, which assign weighted means of the school-level Hajek contrasts with weights N.", "caption": "Package interference, file misc/ReplicationAoAS.R, lines 143–152, typeset from the source file with line numbers added. The marked lines combine the school-level Hajek quantities across schools.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N006-d.png" } ], "formal_release": null, "formal_statements": [], "headline": "The replication script distributed with the interference package averages the Hajek ratios of the schools with weights proportional to school size, and this differs in general from the single pooled ratio of expression (15), which Section 9 names.", "id": "P013-N006", "location": "Section 9, list of estimators, p. 1932 (PDF p. 21); expression (15), p. 1928 (PDF p. 17); interference package, misc/ReplicationAoAS.R lines 141–143 and 150–152, R/estimates.R lines 226–231", "pdf_page": 21, "printed_page": "1932", "reaches": [ { "effect": "implementation_differs_from_paper", "effect_text": "the code implements something other than the printed method", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "implementation_differs_from_paper", "effect_text": "the code implements something other than the printed method", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "implementation_differs_from_paper", "effect_text": "the code implements something other than the printed method", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "Section 9, p.~1932: ``The Hajek ratio estimator \\hl{given in expression (15)}, with the associated linearized variance estimator;''\n\nExpression (15), p.~1928:\n\\[\n\\widehat{\\mu_H}(d_k)=\\frac{\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)\\frac{Y_i}{\\pi_i(d_k)}}{\\sum_{i=1}^N\\mathbf{I}(D_i=d_k)\\frac{1}{\\pi_i(d_k)}} .\n\\]\n\n\\texttt{R/estimates.R}, lines 226--228 (line breaks joined) and 231, run once for each school with \\texttt{N} the size of the school:\n\n\\texttt{mu\\_h <- yT\\_ht / rowSums(t(obs\\_exposure) / obs\\_prob\\_exposure\\_individual\\_kk, na.rm = T)}\n\n\\texttt{yT\\_h <- mu\\_h * N}\n\n\\texttt{misc/ReplicationAoAS.R}, line 143:\n\n\\texttt{tau01\\_h <- estimates\\_all\\_schools[, sum(yT\\_h\\_ind1)/sum(N)-sum(yT\\_h\\_no)/sum(N), by=tr]}", "corrected": "Either of the following removes the discrepancy. Expression (15) needs no change.\n\nTo compute expression (15) for the simulation, pool the schools before taking the ratio:\n\\[\n\\widehat{\\mu_H}(d_k)=\\frac{\\hl{\\sum_s A_{sk}}}{\\hl{\\sum_s B_{sk}}},\\qquad A_{sk}=\\sum_{i\\in s}\\mathbf{I}(D_i=d_k)\\frac{Y_i}{\\pi_i(d_k)},\\qquad B_{sk}=\\sum_{i\\in s}\\mathbf{I}(D_i=d_k)\\frac{1}{\\pi_i(d_k)},\n\\]\nwith the residuals of the linearized variance estimator taken about this pooled mean. A school in which no student receives exposure $d_k$ contributes zero to both sums, and $\\sum_s B_{sk}$ must be positive.\n\nAlternatively, describe the estimator that the script computes: ``The Hajek ratio estimator given in expression (15), \\hl{applied within each school and averaged over schools with weights proportional to the number of students}, with the associated linearized variance estimator''. In symbols, this is $\\sum_s (N_s/N)\\,A_{sk}/B_{sk}$ when every $B_{sk}>0$. The script marks an empty school exposure as missing and either propagates or omits it in its different aggregations.\n\nA correction to the published Table 1 cannot be given without the original computation.", "evidence": "This finding rests on an exact counterexample and on running the package code on it. There are two schools of 30 students: school A is a cycle and school B is a path, each on the vertices $1,\\dots,30$ in order. In each school three students are treated uniformly at random, as in the design of Section 9, and the treated set is $\\{2,3,10\\}$ in both. Outcomes under no exposure are 1 in A and 2 in B, and they are multiplied by $2$, $3/2$ and $5/4$ under $d_{11}$, $d_{10}$ and $d_{01}$, as in Section 9. All exposure probabilities are positive, and all four exposure groups are nonempty in both schools. In exact arithmetic, I find the following contrasts for the two computations.\n\\begin{audittable}{lccc}{Estimated contrasts in the two-school example: the script's aggregation and pooled expression (15).}\nContrast & Script & Pooled (15) & Pooled minus script \\\\\n$\\tau(d_{11},d_{00})$ & $3/2$ & $1929/1285$ & $3/2570$ \\\\\n$\\tau(d_{10},d_{00})$ & $3/4$ & $3861/5140$ & $3/2570$ \\\\\n$\\tau(d_{01},d_{00})$ & $3/8$ & $5669/12850$ & $3401/51400$ \\\\\n\\end{audittable}\nFor the indirect effect, the value from the script is $0.375$ and the pooled ratio is about $0.4412$. The package's school-level function and lines 141--143 and 150--152 of the script, run on this example, all return the values in the column headed Script.", "problem": "Section 9 names expression (15) as the Hajek estimator of the simulation, and expression (15) is a ratio of two sums over the whole population, here the students of all schools. The script computes another quantity. It calls the estimation function once for each school, obtains the school's Hajek ratio multiplied by the size of the school, sums these products over schools and divides by the total number of students. For school $s$ of size $N_s$, let $A_{sk}=\\sum_{i\\in s}\\mathbf{I}(D_i=d_k)Y_i/\\pi_i(d_k)$ and $B_{sk}=\\sum_{i\\in s}\\mathbf{I}(D_i=d_k)/\\pi_i(d_k)$, and let $N=\\sum_s N_s$. When every school has a positive denominator $B_{sk}$, the script computes\n\\[\n\\sum_s\\frac{N_s}{N}\\,\\frac{A_{sk}}{B_{sk}},\\qquad\\text{and expression (15) is}\\qquad \\frac{\\sum_s A_{sk}}{\\sum_s B_{sk}}=\\sum_s\\frac{B_{sk}}{\\sum_t B_{tk}}\\,\\frac{A_{sk}}{B_{sk}} .\n\\]\nThe first weights the school ratios by $N_s/N$, and the second by the realized $B_{sk}/\\sum_t B_{tk}$. Although $\\mathrm{E}[B_{sk}]=N_s$, the realized $B_{sk}$ can differ from $N_s$, and the two estimates can differ. Both are ratio estimators of the same mean $\\mu(d_k)$, and the paper does not state a formula for combining schools. The discrepancy is between the script and the pooled ratio; expression (15) is not in error.", "what_survives": "Expression (15) is correct as printed, and an average of the Hajek ratios of the schools, weighted by school size, is a coherent estimator of the same mean. The exposure mappings, the theoretical results and the other estimators are unaffected. No entry of Table 1 is shown to be wrong: the original simulation code and inputs are not available, and which aggregation produced the published table is unresolved. The application of Section 10 and Table 2 are not affected." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N006", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of nine reference entries, from Angrist and Krueger (1999) to Bowers, Fredrickson and Panagopolous (2013). Red outlines surround four items. The first is the editor's name Ahsenfelter in the entry for Angrist and Krueger. The second is the page range 135 to 144 in the entry for Aronow and Middleton (2013), Journal of Causal Inference 1. The third is the page range 546 to 577 in the entry for Aronow, Samii and Assenova (2015), Polit. Anal. 23. The fourth is the name Panagopolous in the entry for Bowers, Fredrickson and Panagopolous (2013).", "caption": "References, p. 1945 (PDF p. 34), from Angrist and Krueger (1999) to Bowers et al. (2013). The outlines mark four of the six slips: “Ahsenfelter”, the page ranges 135–144 and 546–577, and “Panagopolous”. The entries for Paluck (2011) and Neyman (1990) are on pp. 1946 and 1947.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N011.png" } ], "formal_release": null, "formal_statements": [], "headline": "Five reference entries have a misspelled name, a wrong page range or a wrong journal title, one of the misspellings recurs in the text, and the entry for Neyman (1990) is out of alphabetical order.", "id": "P013-N011", "location": "References, pp. 1945–1947 (PDF pp. 34–36); Section 2, p. 1914 (PDF p. 3)", "pdf_page": 34, "printed_page": "1945–1947", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\begin{itemize}\n\\item Angrist and Krueger (1999), p.~1945: In \\emph{Handbook of Labor Economics} (O. C. \\hl{Ahsenfelter} and D. Card, eds.) 3.\n\\item Aronow and Middleton (2013), p.~1945: \\emph{Journal of Causal Inference} 1 135--\\hl{144}.\n\\item Aronow, Samii and Assenova (2015), p.~1945: \\emph{Polit. Anal.} 23 \\hl{546}--577.\n\\item Bowers et al.\\ (2013), p.~1945: BOWERS, J., FREDRICKSON, M. M. and \\hl{PANAGOPOLOUS}, C. (2013). The text on p.~1914 reads: Bowers, Fredrickson and \\hl{Panagopolous} (2013) apply exact tests to evaluate parameters in models of spillover processes.\n\\item Paluck (2011), p.~1946: \\emph{\\hl{J. Exp. Psychol.}} 47 350--358.\n\\item Neyman (1990), p.~1947, author headings in printed order: SOBEL, M. E. (2006). \\hl{NEYMAN, J. S. (1990).} TCHETGEN TCHETGEN, E. J. and VANDERWEELE, T. J. (2012).\n\\end{itemize}", "corrected": "\\begin{itemize}\n\\item Angrist and Krueger (1999): In \\emph{Handbook of Labor Economics} (O. C. \\hl{Ashenfelter} and D. Card, eds.) 3.\n\\item Aronow and Middleton (2013): \\emph{Journal of Causal Inference} 1 135--\\hl{154}.\n\\item Aronow, Samii and Assenova (2015): \\emph{Polit. Anal.} 23 \\hl{564}--577.\n\\item Bowers et al.\\ (2013): BOWERS, J., FREDRICKSON, M. M. and \\hl{PANAGOPOULOS}, C. (2013). On p.~1914: Bowers, Fredrickson and \\hl{Panagopoulos} (2013) apply exact tests to evaluate parameters in models of spillover processes.\n\\item Paluck (2011): \\emph{\\hl{J. Exp. Soc. Psychol.}} 47 350--358.\n\\item Neyman (1990): the entry is placed \\hl{after Middleton and Aronow (2011) and before Ogburn and VanderWeele (2014)}.\n\\end{itemize}", "evidence": "The evidence is a check of the pages for the order of the list, and a check of the publishers' records for the other five items. The records are those of Elsevier for volume 3A of the \\emph{Handbook of Labor Economics} and for Paluck (2011), of Cambridge University Press for the two articles in \\emph{Political Analysis}, and of De Gruyter for the \\emph{Journal of Causal Inference}.", "problem": "The records of the publishers give Ashenfelter and Panagopoulos for the two names, pp.~135--154 for Aronow and Middleton (2013), pp.~564--577 for Aronow, Samii and Assenova (2015), and the \\emph{Journal of Experimental Social Psychology} for Paluck (2011). The reference list is otherwise alphabetical, and the entry for Neyman (1990) belongs on p.~1946, between Middleton and Aronow (2011) and Ogburn and VanderWeele (2014).", "what_survives": "Each cited work remains identifiable from its authors and title, and no result or estimate depends on these details." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N011", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Listing of source code with line numbers. In lines 169 to 177 of R/make_tr_vec_bernoulli.R, a red box marks the condition R > 2 ^ N of an if statement that stops the function with the message 'R must be smaller than 2 ^ N, the number of possible treatment assignements'. In lines 179 to 187, red boxes mark line 181, a while loop whose condition is that the new vector duplicates one of the vectors already sampled, and line 183, which draws a new vector with rbinom. Below, lines 83 and 88 of R/make_exposure_prob.R divide matrix products of exposure indicators by R + 1 and by R.", "caption": "Source code of the package, not a page of the article: the check on the number of replicates (lines 169–177) and the sampling loop (lines 179–187) of make_tr_vec_bernoulli_indiv, and the two frequency formulas of R/make_exposure_prob.R (lines 83 and 88). The marked lines 181 and 183 redraw an assignment vector until it differs from every vector already kept, and the marked condition on line 169 limits $R$ to the number of distinct vectors.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N012-code.png" }, { "alt": "Crop of the paragraph about approximating exposure probabilities by simulation. A red box marks the words 'produce R random replicate z's based on the randomization plan'. The paragraph ends with the estimators (I-hat-k I-hat-k-prime plus the identity matrix) divided by (R + 1), and I-hat-k I-hat-l-prime divided by R.", "caption": "Section 3, p. 1917 (PDF p. 6): the original formulas used by the later package. The marked words describe replicates drawn from the randomization plan; the two estimators at the end of the paragraph weight the replicates equally. The formulas are correct for independent repetitions of the whole design; Appendix A.1’s conflicting without-replacement wording is addressed separately in the clarity note P013-E010.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N012.png" } ], "formal_release": null, "formal_statements": [], "headline": "The sampler for Bernoulli designs in the later package rejects every assignment vector that repeats an earlier one, and the exposure probabilities are then computed with equal weights on the vectors kept, which distorts them (for $N=100$, $p=0.01$ and $R=1000$ the estimated probabilities of treatment average at least $0.01898$, against $0.01$ under the design); the sampler should keep every independent draw.", "id": "P013-N012", "location": "R package interference (commit c3fc7b3): R/make_tr_vec_bernoulli.R, function make_tr_vec_bernoulli_indiv, lines 169–187; R/make_exposure_prob.R, lines 83 and 88. The package DESCRIPTION credits Stephanie Zonszein, Cyrus Samii and P. M. Aronow; the corresponding formulas appear in Section 3, p. 1917 (PDF p. 6).", "pdf_page": null, "printed_page": null, "reaches": [ { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": "P044-N014", "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [ { "id": "P044-N014", "url": "https://pmaronow.github.io/llm-audit/works/P044/#P044-N014", "work_id": "P044" } ], "severity": "Moderate", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\texttt{R/make\\_tr\\_vec\\_bernoulli.R}, lines 169--177 (written on two lines):\n\n$\\texttt{if (}\\hl{\\texttt{R > 2 \\textasciicircum{} N}}\\texttt{) \\{ stop(paste(\"R must be smaller than\", 2 \\textasciicircum{} N,}$\n\n$\\texttt{\", the number of possible treatment assignements\")) \\}}$\n\nLines 179--187 (the inner loop joined on one line):\n\n\\texttt{for (i in 1:R) \\{}\n\n\\texttt{vec <- rbinom(N, 1, p)}\n\n\\texttt{\\hl{while (any(duplicated(rbind(vec, tr\\_vec\\_sampled[1:i - 1,]))))} \\{ \\hl{vec <- rbinom(N, 1, p)} \\}}\n\n\\texttt{tr\\_vec\\_sampled[i,] <- vec}\n\n\\texttt{\\}}\n\n\\texttt{R/make\\_exposure\\_prob.R}, lines 83 and 88:\n\n\\texttt{(I\\_exposure[[i]] \\%*\\% t(I\\_exposure[[i]]) + diag(N)) / (R + 1)}\n\n\\texttt{(I\\_exposure[[i]] \\%*\\% t(I\\_exposure[[j]])) / R}\n\nSection 3, p.~1917: ``One may nonetheless approximate the $\\boldsymbol{\\pi}_i$'s and joint probabilities with arbitrary precision through simulation, that is, produce $R$ random replicate $\\mathbf{z}$'s based on the randomization plan.''", "corrected": "\\texttt{R/make\\_tr\\_vec\\_bernoulli.R}, lines 169--187, \\hl{with the check of lines 169--177 and the inner loop of lines 181--185 deleted}:\n\n\\texttt{for (i in 1:R) \\{}\n\n\\texttt{vec <- rbinom(N, 1, p)}\n\n\\texttt{tr\\_vec\\_sampled[i,] <- vec}\n\n\\texttt{\\}}\n\nThe sampler then keeps every independent draw, repeated vectors included, and $R$ is no longer limited by $2^N$. The unit-level sampler's documentation should say that repeated vectors are kept and remove its stated bound on $R$. The two formulas of \\texttt{R/make\\_exposure\\_prob.R} stay as they are. For any other design, each replicate should be an independent draw of the whole randomization, with all of its blocks and clusters, and the replicates should be drawn independently of the assignment observed in the experiment. If all distinct assignment vectors are enumerated instead, each must be weighted by its probability under the design, as in $\\mathbf{I}_k\\mathbf{P}\\mathbf{I}_l'$. These weights alone do not correct a random subset of the vectors. Appendix A.1 itself says ``without replacement''; that wording is addressed separately in the clarity note \\fid{P013-E010} and does not establish use of this package for the publication.", "evidence": "The evidence is an exact calculation for a design of ordinary size, a run of the unchanged code on that design, and an exact counterexample with two units.\n\nLet $N=100$ and $p=1/100$, and let the exposure of a unit be its own treatment. Among $R$ distinct assignment vectors, at most one has no treated unit and at most $100$ have exactly one treated unit. Every further vector has at least two. For $R=1000$ the vectors kept then contain at least $100+2\\cdot899=1898$ treated entries. I call the fraction of the vectors kept in which a unit is treated its raw frequency of treatment. Averaged over the units, the raw frequency of treatment is at least $1898/(100\\cdot1000)=0.01898$. This bound holds for every sample that the sampler can return, whatever the seed. The same count gives the other rows of the table.\n\n\\begin{audittable}{rrrr}{Bernoulli design with $N=100$ and $p=0.01$: lower bounds that hold for every sample of $R$ distinct assignment vectors. The probability of treatment under the design is $0.01$. Values that are not exact are rounded down.}\n$R$ & Treated entries & Average raw frequency of treatment & Relative excess over $0.01$ \\\\\n$1{,}000$ & $1{,}898$ & $0.018980$ & $89.80\\%$ \\\\\n$1{,}500$ & $2{,}898$ & $0.019320$ & $93.20\\%$ \\\\\n$10{,}000$ & $24{,}847$ & $0.024847$ & $148.47\\%$ \\\\\n$10{,}593$ & $26{,}626$ & $0.025135$ & $151.35\\%$ \\\\\n$100{,}000$ & $294{,}847$ & $0.029484$ & $194.84\\%$ \\\\\n\\end{audittable}\n\nSince the sampler treats the units symmetrically, the expected raw frequency of each unit obeys the same bounds. The diagonal entries that the package returns add one to each count and to $R$, and for $R=1000$ their average is at least $1998/100100\\approx0.01996$. The raw joint frequency of treatment of a pair of units is the fraction of the vectors kept in which both units are treated. Counting the ordered pairs of treated units in the same way, I find that its average over the ordered pairs of units is at least $1798/(1000\\cdot100\\cdot99)\\approx0.0001816$ for $R=1000$, against $p^2=0.0001$. The value $R=1500$ is the number of replicates in the replication script of the package, and $R=10{,}593$ is the number in the example after Proposition 4.1. The rows apply these numbers to the Bernoulli design. They do not recalculate any result of the article. For small $R$ the bound is not informative: for $R=100$ it is $0.0099$. The distortion is present there as well. For $R=2$ the expected raw frequency of treatment is $0.011057$ to six decimals, which is $10.57\\%$ above $0.01$.\n\nI ran the unchanged sampler with $N=100$, $p=0.01$, $R=1000$ and seed $101$. The $1000$ vectors kept are distinct and contain $2240$ treated entries. The average raw frequency of treatment is $0.0224$, and every unit is treated in at least $12$ of the vectors. On a graph of $50$ disjoint edges, the first-order exposure mapping of the package, which has the four exposures of Section 9, gives the following frequencies.\n\n\\begin{audittable}{lrr}{One run of the unchanged sampler ($N=100$, $p=0.01$, $R=1000$, seed 101) on $50$ disjoint edges: probabilities of the four exposures under the design and their raw frequencies in the run, averaged over the units.}\nExposure & Probability under the design & Average raw frequency \\\\\n$d_{11}$ (direct and indirect exposure) & $0.0001$ & $0.00032$ \\\\\n$d_{10}$ (isolated direct exposure) & $0.0099$ & $0.02208$ \\\\\n$d_{01}$ (indirect exposure) & $0.0099$ & $0.02208$ \\\\\n$d_{00}$ (no exposure) & $0.9801$ & $0.95552$ \\\\\n\\end{audittable}\n\nLet every outcome under treatment equal $1$, so that the total under treatment is $100$. Suppose that this total is estimated by the Horvitz--Thompson estimator with the probabilities that the package returns for this run. Given the vectors kept, the expectation of the estimator over the Bernoulli design is $44.31$, with an empty sum counted as zero. With independent draws, the identity of Appendix A.3 implies that the expectation over the replicates and the design is $100[1-(99/100)^{1001}]$.\n\nFor an exact counterexample with two units, let $N=2$ and $R=4$, and let each unit be treated with probability $p=1/5$. The sampler then keeps each of the four assignment vectors once. Each unit is treated in half of them, although its probability of treatment under the design is $1/5$. When the exposure of a unit is its own treatment, the package returns $3/5$ and $1/5$ for the marginal and joint probabilities of treatment. Their values under the design are $1/5$ and $1/25$. The distortion does not require that the sampler exhaust the assignment vectors: for $R=2$ the expected raw frequency of treatment is $383/1260$. At the observed assignment $(1,0)$ with both outcomes equal to $1$, \\texttt{estimates()} returns a Horvitz--Thompson total of $5/3$ for the treated exposure with variance estimate $10/9$, and the probabilities under the design give $5$ and $20$. I confirmed these values by running the unchanged code.\n\nThe size of the distortion depends on the design. For $p=1/2$ all assignment vectors are equally likely, and the raw frequencies are unbiased. If the first $R$ independent draws are distinct, the sampler returns exactly these draws. For $N=1000$, $p=0.01$ and $R=1500$, the probability that two of them coincide is at most $0.0024$, and the expectations of the raw frequencies differ from those under independent draws by at most the same amount.", "problem": "I could not establish that this implementation was used for computations in the 2017 article. The loop on lines 181--185 redraws any assignment vector that repeats an earlier one. The $R$ vectors kept are then distinct, and they are not independent draws from the Bernoulli design. The function \\texttt{make\\_exposure\\_prob} estimates the exposure probabilities by the unweighted frequencies over these vectors, as in the estimators $(\\widehat{\\mathbf{I}}_k\\widehat{\\mathbf{I}}_k'+\\iota_N)/(R+1)$ and $\\widehat{\\mathbf{I}}_k\\widehat{\\mathbf{I}}_l'/R$ of Section 3. Unweighted frequencies estimate the probabilities under the design when the draws are independent, or when all assignment vectors are equally likely. Under a Bernoulli design with $p\\ne1/2$ the assignment vectors are not equally likely, and the sampler keeps each of them at most once. A vector whose probability exceeds $1/R$ is therefore under-represented in every sample, and less likely vectors take its place. This happens in populations of ordinary size. For $N=100$ and $p=0.01$ there are $2^{100}$ assignment vectors, but the vector with no treated unit has probability $(99/100)^{100}\\approx0.366$. Two independent draws coincide with probability about $0.135$. In this design the less likely vectors have more treated units, and the estimated probabilities of treatment are too large for the numbers of replicates examined in the evidence. The estimated probabilities enter the Horvitz--Thompson estimates and their variance estimates as weights. The check on lines 169--177 follows from the same choice. The distinct vectors are exhausted at $R=2^N$, and the limit $R\\to\\infty$ of Proposition 3.1 is not available to this sampler. The article names Bernoulli designs on a graph of bounded degree as a setting for its asymptotic results (p.~1926). Conditions 1--6 do not exclude the examples below, since a sequence of Bernoulli designs with $p$ fixed at $0.01$ and bounded outcomes can satisfy them, with or without first-order interference on disjoint edges.", "what_survives": "Section 3 and Proposition 3.1 are unaffected when the replicates are drawn independently from the randomization plan, and the two estimators of Section 3 are implemented as printed. The designs, the exposure mappings, the estimands and the results that take the exposure probabilities as known are unchanged. The simulation of Section 9 treats a fixed number of units chosen uniformly; the later package's Table 1 replication script uses the permutation sampler, not this function, and does not establish which program generated the published table. That sampler also rejects repeats by default, but this distortion does not arise when equally likely vectors are sampled without replacement. No entry of Table 1 or Table 2 is shown to be affected by this error. I could not establish when the loop was introduced, who introduced it, or a causal connection between the code and Appendix A.1. I could not identify the code used to compute the exposure probabilities for the application of Section 10 or determine how its replicates were drawn. What is lost is the assurance that the probabilities computed from this sampler are those of the Bernoulli design. In sparse designs such as the one above they are not, and the Horvitz--Thompson estimates and variance estimates that use them inherit the error." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N012", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Listing of source code with line numbers. Lines 44 to 48 document control_condition as the name of the single condition considered the pure control condition. Line 172 declares control_condition=NULL. In line 314 red boxes mark yT_ht['no'] and != 'no' in the definition of tau_ht. In line 318 red boxes mark mu_h['no'] and != 'no' in the definition of tau_h.", "caption": "Source code of the package, not a page of the article: R/estimates.R, the documentation of control_condition (lines 44–48), its declaration (line 172) and the point contrasts (lines 313–320). The marked terms refer to the exposure named 'no' and not to control_condition.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N013.png" } ], "formal_release": null, "formal_statements": [], "headline": "The function estimates() subtracts the exposure named 'no' whatever control_condition is supplied; the two point contrasts should use the supplied control condition.", "id": "P013-N013", "location": "R package interference (commit c3fc7b3): R/estimates.R, lines 313–318; the argument control_condition is declared on line 172 and documented on lines 44–48. The code is not printed in the article; the contrast it computes is equation (3), p. 1920 (PDF p. 9).", "pdf_page": null, "printed_page": null, "reaches": [ { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": "P044-E008", "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P044/", "where": null, "work_id": "P044" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P045/", "where": null, "work_id": "P045" }, { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": "interference", "recorded_as": null, "to": "package", "url": null, "where": null, "work_id": "P013" } ], "related_entries": [], "revisions": [], "same_defect_as": [ { "id": "P044-E008", "url": "https://pmaronow.github.io/llm-audit/works/P044/#P044-E008", "work_id": "P044" } ], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\texttt{R/estimates.R}, lines 314 and 318:\n\n\\texttt{tau\\_ht <- (1 / N) * (yT\\_ht - yT\\_ht[\\hl{'no'}])[names(yT\\_ht) != \\hl{'no'}]}\n\n\\texttt{tau\\_h <- (mu\\_h - mu\\_h[\\hl{'no'}])[names(mu\\_h) != \\hl{'no'}]}", "corrected": "\\texttt{tau\\_ht <- (1 / N) * (yT\\_ht - yT\\_ht[\\hl{control\\_condition}])[names(yT\\_ht) != \\hl{control\\_condition}]}\n\n\\texttt{tau\\_h <- (mu\\_h - mu\\_h[\\hl{control\\_condition}])[names(mu\\_h) != \\hl{control\\_condition}]}\n\nIf \\texttt{control\\_condition} is \\texttt{NULL}, set it to \\texttt{'no'} before these lines, which keeps the earlier point-estimate behaviour. The same literal in \\texttt{tau\\_dsm} should be replaced in the same way.", "evidence": "The evidence is an exact counterexample, run on the unchanged function. Let two units have independent fair binary assignments, and suppose that the observed assignment is $(1,0)$, that the exposures are named \\texttt{treated} and \\texttt{baseline}, that the outcomes are $(3,1)$ and that \\texttt{control\\_condition = 'baseline'}. The Horvitz--Thompson totals are $6$ and $2$. Both contrasts equal $2$, since $(6-2)/2=2$ and $3-1=2$. The function returns \\texttt{NA} for both, together with the variance estimates $10$ (Horvitz--Thompson) and $0$ (Hajek) for \\texttt{treated}. In a second design with three units and a third exposure named \\texttt{'no'}, the Horvitz--Thompson contrasts $2/3$ and $16/3$ against \\texttt{baseline} are returned as $-14/3$ and $-16/3$, and the Hajek contrasts $2$ and $4$ are returned as $-2$ and $-4$.", "problem": "The function documents \\texttt{control\\_condition} as the name of the pure control exposure and uses it when it computes variances (lines 279--306). The Horvitz--Thompson and Hajek point contrasts on lines 314 and 318 instead subtract, and exclude, the exposure literally named \\texttt{'no'}. With any other control label the contrasts are missing, or, if some other exposure is named \\texttt{'no'}, they are taken against the wrong exposure; the point estimates then do not correspond to the variances returned with them.", "what_survives": "Equation (3) and the Hajek estimator of equation (15) are unaffected, and so is every use of the function with the control exposure named \\texttt{'no'}, which is the label of the package's own exposure mappings. The simulation and the application of the article compare each exposure with no exposure ($d_{00}$ and $d_{000}$), and no published result is shown to change. I read the same literals in \\texttt{R/estimates\\_batch.R} but did not run that function." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N013", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Crop of five lines of the first paragraph of Section 10. The passage reads: asking students to nominate up to 10 students in their school that they had chosen to spend time with, face to face or online, in the last two weeks. These nominations were used to construct an undirected adjacency graph so that students were considered peers if either student nominated one another. A red box marks the words 'in the last two weeks.'", "caption": "Section 10, first paragraph, p. 1934 (PDF p. 23). The marked words give the nomination question a recall period of two weeks; the questionnaire says “the last few weeks”.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N015.png" } ], "formal_release": null, "formal_statements": [], "headline": "“In the last two weeks” should read “in the last few weeks”, which is the wording of the nomination question in the questionnaire released with the data.", "id": "P013-N015", "location": "Section 10, first paragraph, p. 1934 (PDF p. 23)", "pdf_page": 23, "printed_page": "1934", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "``At the beginning of the school year, the research team measured students' social networks, asking students to nominate up to 10 students in their school that they had chosen to spend time with, face to face or online, in the last \\hl{two} weeks.''", "corrected": "``At the beginning of the school year, the research team measured students' social networks, asking students to nominate up to 10 students in their school that they had chosen to spend time with, face to face or online, in the last \\hl{few} weeks.''", "evidence": "I checked the published page against the questionnaire and the codebook. Both forms of the questionnaire print the question with the words ``the last few weeks''. The codebook labels the ten nomination variables of the first-wave survey with the same words. The check concerns the written question. It does not establish which period the students had in mind, and the sources record no spoken instructions.", "problem": "The sentence describes the question that measured the networks at the beginning of the school year, and it gives the question a recall period of two weeks. The questionnaire released with the data of the field experiment (ICPSR study 37070) reads ``In the last few weeks I decided to spend time with these students at my school: (in school, out of school, or online)'', followed by ten numbered lines. The written question names no number of weeks.", "what_survives": "The network of Section 10 is built from the recorded nominations: the released script forms an undirected graph from the ten nomination variables. With the nominations held fixed, no exposure, estimate or conclusion of Section 10 depends on the word. One word changes." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N015", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Listing of source code with line numbers. Line 57 assigns to ind.perms the result of genperms(Zraw, blockvar = withinblock, maxiter = niter). In line 58, which assigns to sch.perms, a red box marks the call genperms(schtreat, clustvar = schid, maxiter = niter). Below, lines 87 to 90 of the second script are a loop over i from 1 to 14 that draws a random order of the labels A, A, B, B and assigns it to the schools of block i.", "caption": "Source code of the field experiment’s scripts, not a page of the article: lines 57–58 of 1. Sanitize and generate perms, probs.R and lines 87–90 of 7. Blocking, randomization, and summary statistics.R. The marked call draws the school lottery with the schools as clusters and with no block variable. Lines 87–90 are the lottery that assigned the schools, two of the four schools in each of 14 blocks.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N016-code.png" }, { "alt": "Crop of five lines of the first paragraph of Section 10. The passage reads: The experiment involved two levels of randomization. First, 28 of 56 schools were randomly selected to host the anti-conflict program, via block randomization. Within all schools, a group of between 40 to 64 students were nonrandomly selected as eligible to participate in the program. A red box marks the words 'via block randomization'.", "caption": "Section 10, first paragraph, p. 1934 (PDF p. 23). The marked words describe the school lottery. They are correct, and the marked call of the script does not implement them.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N016.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T25", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T25" }, { "id": "P013-T26", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T26" } ], "headline": "The script that simulates the assignments of the field experiment draws the 28 host schools from all 56 schools and ignores the 14 blocks of four in which the schools were randomized; the call should pass the school blocks, and the sentence of the article, which says “via block randomization”, is correct.", "id": "P013-N016", "location": "Section 10, first paragraph, p. 1934 (PDF p. 23). Code: scripts released with the data of the field experiment (ICPSR study 37070, version 2), 1. Sanitize and generate perms, probs.R, line 58, compared with 7. Blocking, randomization, and summary statistics.R, lines 73–74 and 87–90. The code is not printed in the article.", "pdf_page": 23, "printed_page": "1934", "reaches": [ { "effect": "implementation_wrong", "effect_text": "the package or replication code has the reported defect", "explanation": null, "package": null, "recorded_as": "P037-N031", "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P037/", "where": null, "work_id": "P037" } ], "related_entries": [], "revisions": [], "same_defect_as": [ { "id": "P037-N031", "url": "https://pmaronow.github.io/llm-audit/works/P037/#P037-N031", "work_id": "P037" } ], "severity": "Minor", "subject": "paper", "target_ids": [ "P013-T25", "P013-T26" ], "text": { "as_printed": "Section 10, p.~1934: ``The experiment involved two levels of randomization. First, 28 of 56 schools were randomly selected to host the anti-conflict program, via block randomization.''\n\n\\texttt{1. Sanitize and generate perms, probs.R}, lines 57--58 (trailing comments omitted):\n\n\\texttt{ind.perms <- genperms(Zraw, blockvar = withinblock, maxiter = niter)}\n\n\\texttt{sch.perms <- \\hl{genperms(schtreat, clustvar = schid, maxiter = niter)}}\n\n\\texttt{7. Blocking, randomization, and summary statistics.R}, lines 87--90:\n\n\\texttt{for(i in 1:14) \\{}\n\n\\texttt{Zblock <- sample(c(\"A\",\"A\",\"B\",\"B\"))}\n\n\\texttt{treatment.blind[blockvar == i] <- Zblock}\n\n\\texttt{\\}}", "corrected": "\\texttt{1. Sanitize and generate perms, probs.R}, line 58 (written on two lines):\n\n\\texttt{sch.perms <- genperms(schtreat, \\hl{blockvar = schrb,} clustvar = schid,}\n\n\\texttt{maxiter = niter)}\n\nHere \\texttt{schrb} is the school-block variable of the data, and each school lies in one block. The sentence of Section 10 stays as printed.", "evidence": "The evidence is a reading of the two scripts, an evaluation of the function that the script calls, and an exact calculation. With the arguments of line 58 and the schools of the public data, the design routine of \\texttt{ri} (version 0.9) returns one block of 56 schools with 28 hosts. With the block variable added, it returns 14 blocks of four schools with two hosts each. I ran \\texttt{genperms} to exhaustion on eight synthetic schools in two blocks of four. The call without the block variable enumerates 70 assignments, of which 34 do not have two hosts in each block. The call with the block variable enumerates 36 assignments, and all of them do.\n\nFor the 56 schools, the blocked design has $\\binom{4}{2}^{14}=6^{14}$ equally likely assignments, and the lottery of the script has $\\binom{56}{28}$. Counting the assignments in which two given schools both host, I obtain\n\\[\n\\frac{6^{13}}{6^{14}}=\\frac{1}{6},\\qquad \\frac{3^2\\cdot6^{12}}{6^{14}}=\\frac{1}{4},\\qquad \\frac{\\binom{54}{26}}{\\binom{56}{28}}=\\frac{28\\cdot27}{56\\cdot55}=\\frac{27}{110}\n\\]\nfor two schools of one block under the blocked design, for two schools of different blocks under the blocked design, and for any two schools under the script.\n\nAn exact example shows the consequence for a variance estimate. I take the 56 schools as the units and hosting as the exposure. Let the outcome under hosting be $1$ at two schools and $0$ at the others, and let every outcome without hosting be $0$. Let $p$ be the probability under the design that both schools host. The Horvitz--Thompson estimator of the total under hosting has variance $8p$. Suppose that the estimator of expression (7) is computed with a joint probability $q$ in place of $p$. A direct computation shows that its expectation under the design is $2+8p(q-1/4)/q$, which equals $8p$ for $q=p$ and $2-4p/27$ for $q=27/110$. For two schools of one block, the expectation is $160/81$ and the variance is $4/3$. For two schools of different blocks, the expectation is $53/27$ and the variance is $2$. With the joint probability of the script, the expected variance estimate is too large by $52/81$ in the first case and too small by $1/27$ in the second.", "problem": "The sentence of the article is correct. The script that assigned the schools forms 14 blocks and draws two hosts among the four schools of each block (lines 87--90). The public data show 14 blocks of four schools with two hosts in each. The defect is in the script that simulates the assignments of this experiment and computes the exposure probabilities from them. Its student lottery on line 57 passes the student blocks. Its school lottery on line 58 passes the schools as clusters and passes no block variable. The function \\texttt{genperms} of the R package \\texttt{ri} then treats the 56 schools as one block and draws 28 of them uniformly. Note that each school hosts the program with probability $1/2$ under both designs. The probability that two given schools both host is $27/110$ under the script. Under the blocked design it is $1/6$ for two schools of one block and $1/4$ for two schools of different blocks. The simulated draws then give joint exposure probabilities for students of different schools that are not those of the design. These joint probabilities enter the variance estimators of expressions (7), (10) and (11).", "what_survives": "The sentence of the article needs no change. Under the exposure mapping of Section 10, the exposure of a student depends on the lottery of the student's own school and on the student lottery within that school. This implies that the exact probabilities of the five exposures are the same under the two school designs. Point estimates computed with exact probabilities on a fixed sample therefore do not depend on the defect. Simulated probabilities from a finite number of draws, and a sample chosen from them, need not agree between the two designs. None of the eleven scripts released with the data computes Table 2. It is not established that the variance estimates of Table 2 used these draws, and no number of Table 2 is shown to be wrong on this account. In a separate reconstruction on the public data (ICPSR 37070 V2), an inferred screen requiring all five exposure probabilities to exceed $0.01$ and the legacy R sampler give 2,050 students and match all twelve printed point estimates to three decimals. On that sample, the Horvitz--Thompson standard errors for the $d_{011}$ and $d_{111}$ contrasts against $d_{000}$ are $0.0571$ and $0.0801$ with exact blocked probabilities, and $0.0569$ and $0.0785$ with the script's 10,000 unblocked draws. ICPSR reports that some released variables were changed; the original sample and variance code are unavailable, so these are conditional reconstruction values. Neither pair reproduces the printed $0.029$ and $0.020$. The results of Sections 3 to 8 and the simulation of Section 9 are unaffected. What changes is the script. After the correction, the exposure matrices must be generated again, and every calculation that resamples from them must be repeated." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N016", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "recalculation_from_source", "exhibits": [ { "alt": "Crop of seven lines of text after Table 2. The passage reads: The outcome of interest, y i in the set 0, 1, is a binary indicator for whether the subject wore a program wristband. The effect estimates characterize, for eligible students, the average increase in the probability of wearing a wristband relative to the average in the no exposure condition. (The average for eligible students in the no exposure condition was essentially zero, at 0.000.) Red boxes mark the words 'whether the subject wore a program wristband' and, across a line break, the words 'the probability of wearing a wristband'.", "caption": "Section 10, first paragraph after Table 2, p. 1936 (PDF p. 25). The marked words define the outcome as whether the subject wore a program wristband, and the estimand as the probability of wearing a wristband. The passage gives no rule for a student whose report is absent.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N020.png" }, { "alt": "Dot plot with twelve positions on the horizontal axis: the exposures d001, d011, d101 and d111, each compared with no exposure, for the HT, Hajek and WLS estimators. The vertical axis is the point estimate, with ticks from 0 to 0.4. At each position a black horizontal bar shows the value printed in Table 2, and a blue circle near the bar shows the estimate with students with no recorded general-item answer kept with their released outcome 0, N = 2,050. An orange square, higher, shows the estimate without the 151 students with no recorded second-wave survey, N = 1,899, and a green triangle, higher still, shows the estimate without the 342 students with no recorded general-item answer, N = 1,708. For example, for the HT estimate of d101 the printed value is 0.305, and the circle, square and triangle round to 0.305, 0.329 and 0.366. The title reads: Table 2 is reproduced on the public data only when students with no recorded answer are kept and scored as not wearing the wristband.", "caption": "The twelve point estimates of Table 2, recalculated on the public data under three treatments of the students with no recorded answer to the general wristband item. The horizontal bars are the printed values. Among these three policies, only keeping these students with their released outcome zero ($N=2{,}050$) reproduces all twelve estimates after rounding. Without the 151 students with no recorded second-wave survey ($N=1{,}899$), or without all 342 students with no recorded general-item answer ($N=1{,}708$), none does. The values are in the second table in the Evidence section above.", "url": "https://pmaronow.github.io/llm-audit/works/P013/exhibits/P013-N020-outcome-coding.png" } ], "formal_release": null, "formal_statements": [ { "id": "P013-T23", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T23" }, { "id": "P013-T24", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T24" }, { "id": "P013-T27", "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T27" } ], "headline": "In a reconstruction on the public data, all twelve point estimates of Table 2 are reproduced with 342 of 2,050 students who have no recorded general wristband answer kept and scored zero; the article should state whether it uses this recorded orange-wristband outcome and disclose its missing-answer rule.", "id": "P013-N020", "location": "Section 10, first paragraph after Table 2, p. 1936 (PDF p. 25); Table 2, all rows", "pdf_page": 25, "printed_page": "1936", "reaches": [ { "effect": "cannot_determine", "effect_text": "could not be determined", "explanation": null, "package": null, "recorded_as": null, "to": "work", "url": "https://pmaronow.github.io/llm-audit/works/P037/", "where": null, "work_id": "P037" } ], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Moderate", "subject": "paper", "target_ids": [ "P013-T23", "P013-T24", "P013-T27" ], "text": { "as_printed": "``The outcome of interest, $y_i\\in\\{0,1\\}$, is a binary indicator for \\hl{whether the subject wore a program wristband}. The effect estimates characterize, for eligible students, the average increase in \\hl{the probability of wearing a wristband} relative to the average in the no exposure condition.''", "corrected": "``The outcome of interest, $y_i\\in\\{0,1\\}$, is a binary indicator for \\hl{whether the subject reported wearing an orange program wristband; a student with no recorded answer to the general wristband item, including a student with no recorded second-wave survey, is kept in the sample with $y_i=0$}. \\hl{Under the exposure-mapping assumptions for this recorded outcome,} the effect estimates characterize, for eligible students, the average increase in \\hl{the probability of such a recorded report} relative to the average in the no exposure condition.''\n\nIn the public-data reconstruction with the strict $0.01$ Monte Carlo screen described above, the students with no recorded general-item answer are 342 of 2,050, and 151 of them have no recorded second-wave survey. This wording is the correction if the authors' own outcome file codes these students as the public file does, which should be checked against that file. Under the outcome model above, retaining the wearing contrast requires $L(d_k)=L(d_{000})$ for each contrast; otherwise it requires information or assumptions identifying the unrecorded outcomes and a new analysis with its standard errors.", "evidence": "The evidence is a recalculation on the public data, a derivation and an exact example. The recalculation uses the released code's unblocked school-assignment generator, seed 123 and 10,000 draws, with the sampler that R used before version 3.6; the article describes school assignment as blocked. It keeps the eligible students whose five unsmoothed estimated exposure probabilities exceed $0.01$, and it fits the WLS rows with one slope on network degree. This reconstruction has $N=2{,}050$ students and, with the orange-wristband indicator as released, reproduces all twelve point estimates of Table 2 to three decimals. None of the eleven scripts released with the data computes Table 2, and the reconstruction is not the authors' calculation. Recall that the article limits the analysis to students with nonzero probabilities and does not state the cut-off of $0.01$. Whether the published analysis used it is not determined.\n\n\\begin{audittable}{lrrrr}{Public data, reconstructed sample: the students with no recorded answer to the general wristband item, by observed exposure, with their share of $N$ and the number among them with no recorded second-wave survey. All 342 have the value 0 on the orange-wristband indicator.}\nObserved exposure & $N$ & No answer & Share & No survey \\\\\n$d_{000}$ (no exposure) & 1,034 & 198 & 19.1\\% & 78 \\\\\n$d_{001}$ (school only) & 157 & 25 & 15.9\\% & 11 \\\\\n$d_{011}$ (indirect) & 348 & 51 & 14.7\\% & 26 \\\\\n$d_{101}$ (isolated direct) & 199 & 25 & 12.6\\% & 13 \\\\\n$d_{111}$ (direct and indirect) & 312 & 43 & 13.8\\% & 23 \\\\\nAll & 2,050 & 342 & 16.7\\% & 151 \\\\\n\\end{audittable}\n\n\\begin{audittable}{llrrrr}{Point estimates of Table 2 as printed and as recalculated on the public data under three treatments of the students with no recorded general-item answer: kept and scored 0 ($N=2{,}050$), without the 151 students with no recorded second-wave survey ($N=1{,}899$), and without all 342 ($N=1{,}708$).}\nEstimator & Contrast & Printed & Kept as 0 & Without 151 & Without 342 \\\\\nHT & $\\tau(d_{001},d_{000})$ & 0.057 & 0.057 & 0.062 & 0.069 \\\\\n & $\\tau(d_{011},d_{000})$ & 0.154 & 0.154 & 0.166 & 0.185 \\\\\n & $\\tau(d_{101},d_{000})$ & 0.305 & 0.305 & 0.329 & 0.366 \\\\\n & $\\tau(d_{111},d_{000})$ & 0.299 & 0.299 & 0.323 & 0.359 \\\\\nHajek & $\\tau(d_{001},d_{000})$ & 0.058 & 0.058 & 0.062 & 0.069 \\\\\n & $\\tau(d_{011},d_{000})$ & 0.154 & 0.154 & 0.167 & 0.181 \\\\\n & $\\tau(d_{101},d_{000})$ & 0.292 & 0.292 & 0.310 & 0.332 \\\\\n & $\\tau(d_{111},d_{000})$ & 0.307 & 0.307 & 0.330 & 0.355 \\\\\nWLS & $\\tau(d_{001},d_{000})$ & 0.056 & 0.056 & 0.060 & 0.068 \\\\\n & $\\tau(d_{011},d_{000})$ & 0.156 & 0.156 & 0.169 & 0.184 \\\\\n & $\\tau(d_{101},d_{000})$ & 0.295 & 0.295 & 0.312 & 0.336 \\\\\n & $\\tau(d_{111},d_{000})$ & 0.306 & 0.306 & 0.329 & 0.353 \\\\\n\\end{audittable}\n\nEvery student removed in the last two columns has outcome $0$. With the exposure probabilities held fixed, the numerators of the Horvitz--Thompson means are then unchanged, and only the divisor $N$ changes. Each HT contrast with these students kept equals $1708/2050=854/1025$ of its value without the 342 students, and $1899/2050$ of its value without the 151. These ratios describe a change of divisor. They do not measure a bias.\n\nTo isolate the effect of absent reports, let $R_i(d)$ indicate that usable wearing information for student $i$ is recorded under exposure $d$, let $y_i(d)$ be the wearing outcome, and suppose the analyzed outcome is $g_i(d)=R_i(d)\\,y_i(d)$. This model assumes that recorded reports equal wearing. Suppose also that $y_i$ and $R_i$ depend on the assignment only through the exposure and that the exposure probabilities used in the estimator are exact and positive. By Lemma 4.1 applied to $g$, I have\n\\[\n\\mathrm{E}\\bigl[\\widehat{\\mu_{\\mathrm{HT}}}(d)\\bigr]=\\frac{1}{N}\\sum_{i=1}^{N}g_i(d)=\\mu(d)-L(d),\\qquad L(d)=\\frac{1}{N}\\sum_{i=1}^{N}\\bigl(1-R_i(d)\\bigr)\\,y_i(d),\n\\]\nwhere the estimator is computed from the recorded outcomes and $\\mu(d)$ is the mean of the wearing outcomes $y_i(d)$. Subtracting the same identity for $d_{000}$, I obtain\n\\[\n\\mathrm{E}\\bigl[\\widehat{\\tau_{\\mathrm{HT}}}(d_k,d_{000})\\bigr]-\\tau(d_k,d_{000})=L(d_{000})-L(d_k).\n\\]\nThis identity implies that the estimator with absent reports counted as zero is unbiased for the effect on wearing if and only if $L(d_k)=L(d_{000})$. It suffices that under both exposures no student whose report would be absent would wear the wristband. Randomization, bounded outcomes and positive exposure probabilities do not imply this condition.\n\nThe exact example has two units with independent fair assignments and no interference. The first unit always reports, and its wearing outcome is $0$ without treatment and $1$ with it. The second unit never reports. Its wearing outcomes without and with treatment may be $(0,0)$, $(0,1)$ or $(1,0)$, and the recorded data are the same in the three cases under all four assignments. The effect on wearing is $1/2$, $1$ or $0$. The estimator with the absent report counted as zero has expectation $1/2$ in each case, and its bias is $0$, $-1/2$ or $1/2$. The estimator on the reporting unit alone has expectation $1$. The recorded data therefore do not determine the sign of the bias.\n\nTwo limits apply. In the whole public file, 4,108 of the 24,471 records have no recorded answer to the general item, and four of them have the value $1$ on the orange-wristband indicator. An absent answer to the general item does not prove that no information on colour was recorded. The released scripts do not show how the zeros arose: the cleaning script sets codes above 4 to missing and otherwise leaves the indicator as it is.", "problem": "The article describes the outcome as whether the student wore a program wristband, measured by self-report (p.~1934), and it gives no rule for a student whose report is absent. On the public data release of the field experiment (ICPSR study 37070, version 2), the variable that reproduces Table 2 is an indicator for a recorded orange-wristband report. It is $0$ or $1$ for every student. The general wristband item of the second-wave survey, ``I wore a Roots wristband this past week'', has no recorded answer for 342 of the 2,050 students of the reconstructed sample. Of these 342 students, 151 have no recorded second-wave survey, and 191 have a survey and no answer to the item. All 342 have the value $0$ on the indicator. Among the three policies compared here, the twelve point estimates of Table 2 are reproduced to three decimals only when these students are kept in the sample with outcome $0$. Without the 342 students the sample has 1,708 students, and without the 151 alone it has 1,899. In both cases none of the twelve estimates is reproduced. The article does not distinguish an observed negative answer from a zero recorded without an answer. For one student in six in this reconstruction, the zero stands where no answer to the general wristband item is recorded. Under the exposure assumptions for this recorded outcome, the reconstructed point estimates concern effects on a recorded report of wearing, with an absent report counted as zero. In the outcome model below, effects on wearing require a further condition that the article does not state: the students whose reports would be absent must contribute equally to the mean of wearing under the two exposures compared. The sizes that the text quotes, about 30 percentage points for direct exposure and 15 to 16 for indirect exposure, depend on the outcome definition. ICPSR notes that some released variables were changed. The authors' own outcome file is not public, and I could not check whether it codes these students as the public file does.", "what_survives": "All twelve point estimates of Table 2 are reproduced on the public data under this coding, and so is the mean of zero in the no-exposure condition (0 of 1,034 students). The broad order of the effects is the same under each of the three treatments and for each estimator: school-only exposure, then indirect exposure, then the two direct exposures. This outcome-coding issue does not affect the estimators or the results of Sections 3 to 8, and the simulation of Section 9 has no absent outcomes. What changes is the definition of the outcome, and with it the estimand of Table 2 and the meaning of the sizes quoted on pp.~1936--1937. No bias in the effects on wearing is identified. This finding concerns point estimates only; a different treatment of these students would need its own standard errors." }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-N020", "work": { "arxiv_id": "1305.6156", "doi": "10.1214/16-aoas1005", "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "url": "https://pmaronow.github.io/llm-audit/works/P013/" }, "work_id": "P013" } ], "lean": { "applicable": true, "guide": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/docs/works/P013.md", "repository": "https://github.com/pmaronow/llm-audit-lean", "targets": [ { "declarations": [ { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairSix_exact_enumeration", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2728", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairSix_formulas", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2713", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairSix_joint", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2703", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairSix_on_split", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2748", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairSix_pi", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2694", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pair_score_moments", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1960", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "The second equality is a raw second moment, not a variance; the 8/ε term is essential for the target’s diverging scaled-variance calculation.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pair_variance_score_moments", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1969", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "This is the exact witness family requested by T1, including strict undercoverage for every 0<α<1; no full all-integer formal negation is claimed by this row. The even-size restriction is in the target itself, and expression (11) is everywhere positive in the family, so Real.sqrt adds no spurious coverage cases.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.paired_asymptotic_counterexample_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GLP013P012B1.lean#L796", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "The printed dependency antecedent is weak because of its nonmembership clauses, but genuine full-vector dependency is separately established, so the witness does not rely on that weakness.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.paired_finite_hypotheses", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1995", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "This row proves the finite portion of the witness, not the asymptotic assertions on its own; those are supplied by row 01. The zero same-unit cross-exposure joints are included in covarianceA’s correction.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.paired_finite_identities_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GLP013P012B1.lean#L830", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_probability_product", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1383", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_probability_by_units", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1406", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.paired_pi", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1879", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.paired_joint", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1890", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.paired_row_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1925", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:9e63dcde927782dd8301d183ef6f96a18daf4c0ee1a46c1796fdac0871b1dce6", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T1", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T1" }, "html": { "body": "

Proposition 6.2, as printed, p. 1926 (PDF p. 15), and Appendix A.7, p. 1943 (PDF p. 32). The paired family satisfies the standing conditions and genuine degree-two dependence, but its positive unbiased variance estimate has scaled probability limit six instead of eight and its Wald intervals strictly undercover. All finite identities, asymptotic limits and the decimal coverage at level 95 percent are included.

", "headline": "Proposition 6.2, as printed", "location": "p. 1926 (PDF p. 15); proof in Appendix A.7, p. 1943 (PDF p. 32)" }, "id": "P013-T1", "text": { "body": "Proposition 6.2, as printed, p. 1926 (PDF p. 15), and Appendix A.7, p. 1943 (PDF p. 32). The paired family satisfies the standing conditions and genuine degree-two dependence, but its positive unbiased variance estimate has scaled probability limit six instead of eight and its Wald intervals strictly undercover. All finite identities, asymptotic limits and the decimal coverage at level 95 percent are included.", "headline": "Proposition 6.2, as printed", "location": "p. 1926 (PDF p. 15); proof in Appendix A.7, p. 1943 (PDF p. 32)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T1" }, { "declarations": [ { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "This is an acceptable generalization, not a loss of cases or an assumption of the implication. Claim-stage verdict unchanged.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.centered_concentration_implies_lower_tail", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2627", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.condition3_implies_comparison", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1696", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.conservative_lower_tail", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1772", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "{'source': 'adjudication+second review', 'adjudicator': \"Wave1/Gap/GLP013P012B2.lean:13 includes hnested : NestedPopulationSetup S, importing the common outcome table imposed at P013_Audit.lean:1657 and P013_Audit.lean:1662. Consequently the two coverage declarations cover only arrays in which an existing unit's outcomes at both comparison exposures stay unchanged as N grows. T2's per-N fixed outcomes and nested unit sets allow bounded outcome arrays that vary with N, including y_{N,i}(0)=y_{N,i}(1)=1+i/N under independent fair binary assignment, which satisfies all the corrected coverage premises. Indexing by N=n+1 and the convention for negative estimated variance cause no material loss; this extra cross-size restriction does.\", 'suggested_fix': 'Use a nesting premise that does not require potential outcomes to agree across different population sizes, and extend the underlying coverage bridge accordingly. Each finite experiment should retain its own fixed potential outcomes, comparison positivity, C3, genuine C5, C6 and (L).', 'second_review': \"P013/Wave1/Gap/GLP013P012B2.lean:13 includes hnested, and NestedPopulationSetup at P013/P013_Audit.lean:1657, especially its equality at line 1662, requires one outcome table across all sizes at the fixed comparison exposures. T2's setting allows 'For each N' fixed potential outcomes satisfying Conditions 1 and 2, without requiring that the same unit's outcomes remain equal as N changes. Independent fair binary assignments with y_{N,i}(0)=y_{N,i}(1)=1+i/N have uniformly bounded outcomes and inverse marginals, an empty genuine dependency graph, and N Vhat_N=N V_N→28/3, so they meet all corrected coverage premises but fail hnested. Extending this coverage declaration and its bridge to those arrays requires a statement repair, independently of the accurately disclosed restriction to the coverage component.\", 'severity': 'major'}", "credit_status": "narrower", "fidelity_status_baseline": "narrower", "lean_name": "AronowAudit.P013.Wave1.corrected_wald_coverage_proof", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GLP013P012B2.lean#L13", "status": "proved", "support_scope": "limited", "used_in_solution": false }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "{'source': 'adjudication+second review', 'adjudicator': \"Wave1/Gap/GLP013P012B2.lean:580 adds hnested : NestedPopulationSetup S to the shrinking-joint rate, thereby requiring the common outcome table of P013_Audit.lean:1657 and P013_Audit.lean:1662. T2(R') assumes C3 and genuine C5 in the per-population setting, with no equality requirement on an existing unit's potential outcomes across different N. Independent fair binary assignment with y_{N,i}(0)=y_{N,i}(1)=1+i/N meets those assumptions and has b_N=1/4 for N>=2, but cannot satisfy the added common-table premise. The reviewer's convention for an empty initial minimum domain is harmless because that domain is eventually nonempty; the new problem is the restriction on outcome arrays.\", 'suggested_fix': 'Remove hnested from the rate theorem, or weaken it to retain only nesting of unit sets and exposure-label identification. Keep the attained-minimum condition on the nonempty positive-joint domain and the stated rate and implication to (L).', 'second_review': \"P013/Wave1/Gap/GLP013P012B2.lean:580 requires hnested, importing the cross-size outcome equality in P013/P013_Audit.lean:1662. The credited T2(R') component assumes C3 and genuine C5 for the per-population setting and the smallest positive distinct-unit joint b_N, without imposing that equality. Independent fair binary own-assignment exposures with y_{N,i}(0)=y_{N,i}(1)=1+i/N satisfy those premises and have b_N=1/4 for N≥2, yet fail the common-table condition. Remove or weaken hnested to cover these arrays; the conditional minimum convention at line 582 causes no asymptotic loss because PositiveJointDomain is eventually nonempty under the stated C5 and positivity assumptions.\", 'severity': 'major'}", "credit_status": "narrower", "fidelity_status_baseline": "narrower", "lean_name": "AronowAudit.P013.Wave1.shrinking_joints_variance_rate_proof", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GLP013P012B2.lean#L574", "status": "proved", "support_scope": "limited", "used_in_solution": false }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "Compared against the corrected R component, so its explicit uniform joint lower bound is a target hypothesis, not an extra restriction. Claim-stage verdict unchanged.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.uniform_joints_variance_rate_proof", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GLP013P012B2.lean#L609", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "No limit of the expectation, variance ratio or studentized statistic is assumed. Claim-stage verdict unchanged.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.vanishing_variance_implies_lower_tail", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2655", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairTerm_second_moment_identity", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3684", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.positive_joint_second_moment_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3702", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairTerm_second_moment_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3736", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "conditional_on": [], "credit_reason": "The selected coverage proof already applies this bridge and carries the same disclosed common-outcome-table premise.", "credit_status": "narrower", "fidelity_status_baseline": "narrower", "lean_name": "AronowAudit.P013.Wave1.corrected_wald_coverage_bridge", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/CorrectedCoverage.lean#L24", "status": "proved", "support_scope": "limited", "used_in_solution": false } ], "formalization": { "claim_revision_id": "catalog:f1a96a1f3f53c00b4184a600dc39fca07c56f869be3bceb6dc4508c63f0e0b52", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T2", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T2" }, "html": { "body": "

corrected Proposition 6.2, p. 1926 (PDF p. 15), and Appendix A.7, p. 1943 (PDF p. 32). Comparison boundedness, genuine bounded-degree dependence, positive limiting scaled variance and the added lower-tail condition ensure nominal lower-limit coverage of the population contrast. Centered concentration suffices; both joint-probability rates and their concentration consequences hold without outcome stability across sizes. The three original conditions alone permit strict undercoverage.

", "headline": "Proposition 6.2, corrected", "location": "p. 1926 (PDF p. 15); proof in Appendix A.7, p. 1943 (PDF p. 32)" }, "id": "P013-T2", "text": { "body": "corrected Proposition 6.2, p. 1926 (PDF p. 15), and Appendix A.7, p. 1943 (PDF p. 32). Comparison boundedness, genuine bounded-degree dependence, positive limiting scaled variance and the added lower-tail condition ensure nominal lower-limit coverage of the population contrast. Centered concentration suffices; both joint-probability rates and their concentration consequences hold without outcome stability across sizes. The three original conditions alone permit strict undercoverage.", "headline": "Proposition 6.2, corrected", "location": "p. 1926 (PDF p. 15); proof in Appendix A.7, p. 1943 (PDF p. 32)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T2" }, { "declarations": [ { "claim_revision_id": "catalog:973ba3a32d5afe427e91e621de3990a827fa5217f51abcc4d394e3eb53562fea", "conditional_on": [], "credit_reason": "This is a witness for T3(b), with logarithms evaluated strictly inside their ordinary positive domain and with population total 1. The only public description supplied is accurate.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.signed_log_rule_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2941", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:973ba3a32d5afe427e91e621de3990a827fa5217f51abcc4d394e3eb53562fea", "conditional_on": [], "credit_reason": "This is the nonzero-total witness for T3(a,b); its two power inequalities also establish that R=1 is the smallest passing nonnegative integer.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.signed_replication_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1567", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:973ba3a32d5afe427e91e621de3990a827fa5217f51abcc4d394e3eb53562fea", "conditional_on": [], "credit_reason": "This refutes the absolute-bias step of Appendix A.3 at zero total; it does not claim to refute a relative-bias inequality on a zero denominator.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.zero_total_replication_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1590", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:973ba3a32d5afe427e91e621de3990a827fa5217f51abcc4d394e3eb53562fea", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T3", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T3" }, "html": { "body": "

Proposition 4.1 and the replicate rule, as printed; p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27). The two-unit signed-outcome experiment refutes the printed relative-bias bound and replicate rule, and its zero-total variant refutes the absolute-bias inequality.

", "headline": "Proposition 4.1 and the replicate rule, as printed", "location": "p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27)" }, "id": "P013-T3", "text": { "body": "Proposition 4.1 and the replicate rule, as printed; p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27). The two-unit signed-outcome experiment refutes the printed relative-bias bound and replicate rule, and its zero-total variant refutes the absolute-bias inequality.", "headline": "Proposition 4.1 and the replicate rule, as printed", "location": "p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T3" }, { "declarations": [ { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.binomial_inverse_moment", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2853", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.cancellation_factor_ge_one", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2904", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.common_sign_cancellation_factor", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2914", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.relative_bias_from_absolute", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2929", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_log_rule_equivalence", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2885", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "The two tested values are exactly corrected part (f); no estimator bias guarantee for unrestricted signed outcomes is inferred from this numerical row alone.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_rule_numerical", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1536", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.signed_bias_absolute_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1503", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Covers exactly corrected parts (b)–(e); the exact expectation and the numerical part are covered by R092-12 and R092-09. N>0 is inherent in the minimum-probability/mean setup, and R≥1 is explicit in the corrected claim.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.simulated_bias_repaired", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3639", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Covers corrected part (a). Its positivity and R≥1 premises match the target, rather than introducing extra assumptions.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.simulated_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3594", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.simulated_power_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1518", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_reindex", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1252", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_cons_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1270", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.smoothed_inverse_success", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1285", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_independent_product", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3576", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:fb78e540be90bc8e211c5428de51a23ff880921980f5b2b80deb4b7f898ff6ec", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T4", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T4" }, "html": { "body": "

Proposition 4.1 and the replicate rule, corrected; p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27). Independent simulated assignments yield the corrected exact expectation, absolute and relative bias bounds, sign-qualified replicate rule and numerical threshold.

", "headline": "Proposition 4.1 and the replicate rule, corrected", "location": "p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27)" }, "id": "P013-T4", "text": { "body": "Proposition 4.1 and the replicate rule, corrected; p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27). Independent simulated assignments yield the corrected exact expectation, absolute and relative bias bounds, sign-qualified replicate rule and numerical threshold.", "headline": "Proposition 4.1 and the replicate rule, corrected", "location": "p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T4" }, { "declarations": [ { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P013_Audit.lean:726 asserts PrintedDependency and the row bound one for the identity indicator for every finite exposure law. PrintedNoEdge at P013_Audit.lean:669 retains exactly the four printed nonmembership clauses, so the vacuous independence antecedent is the defect that T5(a) asks this row to expose. N=0 is only an additional empty case, and an identity row has exactly one entry for each actual unit.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.printed_identity_dependency", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L725", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': \"P013_Audit.lean:707 states precisely T5(a)'s equivalence between the printed identity-indicator antecedent and both disjoint sets being empty. Disjointness is required by the printed definition itself. The extension to N=0 excludes no claimed case, and the theorem characterizes the printed antecedent's vacuity rather than having inconsistent hypotheses.\"}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.printed_identity_vacuous", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L706", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': \"P013_Audit.lean:902 uses exactly the loop-free within-triple indicator of T5(b), and concludes both the empty-set characterization of PrintedNoEdge and row sum two. Symmetry and zero diagonal follow immediately from its explicit definition. Allowing q=0 adds an empty case; for q>=1 the triangles are nonempty, and the earlier record's identity-loop explanation is inapplicable to this row.\"}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.printed_triangles_vacuous", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L901", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.shared_condition4_failure", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2448", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P013_Audit.lean:868 concludes printed dependency with row bound one, all marginals equal to 1/2, all exposure-1 joints equal to 1/2, and failure of pair independence. The design and population at P013_Audit.lean:854 and P013_Audit.lean:861 encode the two shared fair-Bernoulli assignments in T5(c), so N>=2 gives the required dependent-distinct-unit witness. Zero outcomes and the additional N=0 or N=1 instances do not remove any case of this finite supporting component.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.shared_exposure_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L867", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P013_Audit.lean:890 assumes exactly that each false Boolean entry of g implies pair independence and concludes that its indicator sum is N^2. As there are N^2 entries each bounded by one, this equality is equivalent to every entry being true, matching the stated supporting component of T5(c). It supplies the exact count for the shared-exposure family; the harmless N=0 extension and the absence of a separate asymptotic conclusion do not narrow that component.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.shared_pair_count", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L889", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "conditional_on": [], "credit_reason": "This refutes precisely the literal printed C5-to-C4 assertion; it is not a counterexample to a different theorem requiring C6. Metadata: the target asymptotic flag is false despite this sequence component.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.shared_sequence_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3041", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:2c0cdaad8ee5245afdaad78dd13790333b4d04770d921381b336757b081017a9", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T5", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T5" }, "html": { "body": "

Definition of the dependency indicator hijh_{ij}hij​ and Condition 5, as printed, with the claim that Condition 5 subsumes Condition 4; p. 1925 (PDF p. 14); p. 1926 (PDF p. 15). The printed dependency condition is vacuous for identity and triangle indicators with row sums exactly one and two, respectively, and permits the fully shared Bernoulli counterexample to Condition 4.

", "headline": "Definition of the dependency indicator hijh_{ij}hij​ and Condition 5, as printed, with the claim that Condition 5 subsumes Condition 4", "location": "p. 1925 (PDF p. 14); p. 1926 (PDF p. 15)" }, "id": "P013-T5", "text": { "body": "Definition of the dependency indicator $h_{ij}$ and Condition 5, as printed, with the claim that Condition 5 subsumes Condition 4; p. 1925 (PDF p. 14); p. 1926 (PDF p. 15). The printed dependency condition is vacuous for identity and triangle indicators with row sums exactly one and two, respectively, and permits the fully shared Bernoulli counterexample to Condition 4.", "headline": "Definition of the dependency indicator $h_{ij}$ and Condition 5, as printed, with the claim that Condition 5 subsumes Condition 4", "location": "p. 1925 (PDF p. 14); p. 1926 (PDF p. 15)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T5" }, { "declarations": [ { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "conditional_on": [], "credit_reason": "This is a harmless logical generalization, not a loss of directed cases.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.corrected_edge_equivalence", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L739", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "conditional_on": [], "credit_reason": "Symmetry belongs to the symmetric equivalence expressly asserted by T6(a); it is not added to the directed statement.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.corrected_symmetric_equivalence", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L762", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "conditional_on": [], "credit_reason": "Scope is the sequence implication, not a conjunction of all finite T6 clauses; those are independently covered here. The predicate assumes full subvector independence, not the conclusion Condition 4.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.directed_condition5_implies_condition4", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/FidP013.lean#L25", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "conditional_on": [], "credit_reason": "Finite exposure-vector point-event factorization is equivalent to ordinary vector independence here.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.genuine_implies_pair_independent", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L772", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "conditional_on": [], "credit_reason": "Natural-number row bounds lose no case because the counted rows are integer-valued and any uniform finite real bound has a natural upper bound.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.graph_pair_count", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L807", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "conditional_on": [], "credit_reason": "The diagonal is always counted by graphPairIndicator, even if h has no loops, exactly as specified by T6(c).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.graph_pair_count_symmetric", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L832", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.coordinate_population_genuine_dependency", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1427", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false } ], "formalization": { "claim_revision_id": "catalog:a31f6aab028f61017aeba8405739dcc67cf3cc09482a6bba6e05d121468c7f03", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T6", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T6" }, "html": { "body": "

Definition of the dependency indicator hijh_{ij}hij​, corrected, with Condition 5 and the claim that Condition 5 subsumes Condition 4; p. 1925 (PDF p. 14); p. 1926 (PDF p. 15). The corrected edge condition supplies pairwise independence, the directed and symmetric pair-count bounds, and Condition 5 implies Condition 4.

", "headline": "Definition of the dependency indicator hijh_{ij}hij​, corrected, with Condition 5 and the claim that Condition 5 subsumes Condition 4", "location": "p. 1925 (PDF p. 14); p. 1926 (PDF p. 15)" }, "id": "P013-T6", "text": { "body": "Definition of the dependency indicator $h_{ij}$, corrected, with Condition 5 and the claim that Condition 5 subsumes Condition 4; p. 1925 (PDF p. 14); p. 1926 (PDF p. 15). The corrected edge condition supplies pairwise independence, the directed and symmetric pair-count bounds, and Condition 5 implies Condition 4.", "headline": "Definition of the dependency indicator $h_{ij}$, corrected, with Condition 5 and the claim that Condition 5 subsumes Condition 4", "location": "p. 1925 (PDF p. 14); p. 1926 (PDF p. 15)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T6" }, { "declarations": [ { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "This is the asymptotic witness component, not an assertion about nominal liminf coverage. Together with the finite certificates in R094-03 and R094-04 it satisfies the printed hypotheses, even under genuine dependency. For every N>=1, unit 1 has Y_1=1, so the exported identity Vhat=4 sum_i Y_i^2/N^2 gives Vhat>=4/N^2>0: division and square-root totalization do not change the witness. No conditional theorem is assumed. Claim-stage verdict retained.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.block_asymptotic_counterexample_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GLP013P012B3.lean#L602", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_effect_error", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2985", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Auxiliary finite part of T7(c); no asymptotic assertion is credited to this row alone.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.block_endpoint_bounds_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GP013P012B1.lean#L143", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Exact finite witness component T7(a,b). The potential-outcome and exposure definitions directly implement Conditions 1 and 2 and persist across population sizes; pi=1/2 gives positivity and c_2=2, and the outcome bound gives c_1=2. This row does not claim the separate no-limit results.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.block_finite_identities_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GLP013P012B3.lean#L726", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_joint", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2964", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_outcome_properties", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2974", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_pi", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2957", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_unit_score_identity", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2069", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_unit_square_identities", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2081", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.rademacher_moments", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2075", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.rademacher_coordinate_mean", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3749", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.rademacher_coordinate_covariance", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3756", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.rademacher_sum_variance", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3770", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.variance_sub_const", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3779", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_true_variance", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3790", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_condition6_limit", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3807", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_genuine_dependency", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3822", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:d108820a7307812bf6bb4f8eceb9253ddba1bb709f1969577e2e422843dd20fb", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T7", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T7" }, "html": { "body": "

Appendix A.7 (proof of Proposition 6.2), convergence claims of the first and last paragraphs, as printed; p. 1943 (PDF p. 32). The fixed block family has a concentrating scaled variance estimate with two subsequential limits and no limiting variance ratio or studentized law.

", "headline": "Appendix A.7 (proof of Proposition 6.2), convergence claims of the first and last paragraphs, as printed", "location": "p. 1943 (PDF p. 32)" }, "id": "P013-T7", "text": { "body": "Appendix A.7 (proof of Proposition 6.2), convergence claims of the first and last paragraphs, as printed; p. 1943 (PDF p. 32). The fixed block family has a concentrating scaled variance estimate with two subsequential limits and no limiting variance ratio or studentized law.", "headline": "Appendix A.7 (proof of Proposition 6.2), convergence claims of the first and last paragraphs, as printed", "location": "p. 1943 (PDF p. 32)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T7" }, { "declarations": [ { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "No upper restriction epsilon<1 or nonnegativity of W is needed for the one-sided Chebyshev bound.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.conservative_relative_lower_tail", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1790", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "This row covers T8(a,c), not the derivation of its premises from primitive paper conditions; T8 itself explicitly assumes those premises. Coverage probabilities are bounded, so real liminf has its ordinary meaning.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.corrected_concentration_coverage_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GP013P012B1.lean#L336", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "The extra expectation limit is in the corrected claim T8(d); c≤s is also implied by conservativeness and the two finite limits. The totalization of the finite-index ratio/studentized statistic is asymptotically irrelevant.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.corrected_studentized_limit_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/Gap/GP013P012B1.lean#L595", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.finite_wald_coverage_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2326", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.probability_inter_lower", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2294", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.probability_mono", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2281", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.probability_nonneg", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2272", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "This covers the vanishing-tail part of T8(b); R093-10 supplies the finite inequality.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.relative_lower_tail_tends_zero", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2515", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.variance_concentration", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2462", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.wald_event_inclusion", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2305", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:4300636e7b1dc7fff7031a037cbc7433e5da039746e3939fab182a4595399f08", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T8", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T8" }, "html": { "body": "

corrected concentration and coverage argument, Appendix A.7, p. 1943 (PDF p. 32). Conservativeness and vanishing variance of the scaled estimate give centered concentration, the finite relative lower-tail bound and both coverage lower limits. If its expectation converges to a finite value, the scaled estimate, variance ratio and studentized error have the stated limits. Ratios and studentization are extended by zero where the estimated variance is nonpositive; that event has probability tending to zero.

", "headline": "Appendix A.7 (proof of Proposition 6.2), concentration and coverage steps, corrected", "location": "p. 1943 (PDF p. 32)" }, "id": "P013-T8", "text": { "body": "corrected concentration and coverage argument, Appendix A.7, p. 1943 (PDF p. 32). Conservativeness and vanishing variance of the scaled estimate give centered concentration, the finite relative lower-tail bound and both coverage lower limits. If its expectation converges to a finite value, the scaled estimate, variance ratio and studentized error have the stated limits. Ratios and studentization are extended by zero where the estimated variance is nonpositive; that event has probability tending to zero.", "headline": "Appendix A.7 (proof of Proposition 6.2), concentration and coverage steps, corrected", "location": "p. 1943 (PDF p. 32)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T8" }, { "declarations": [ { "claim_revision_id": "catalog:dc25e2c735c52edb1829c7c98303db4cf6c8fd7767aa1fc40ff4112b71fe277f", "conditional_on": [], "credit_reason": "The nesting and all-exposure positivity premises come from the claim’s setting, rather than from the row description. The conclusion covers only the mean-consistency component of T9(i).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.corrected_mean_consistency", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2600", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:dc25e2c735c52edb1829c7c98303db4cf6c8fd7767aa1fc40ff4112b71fe277f", "conditional_on": [], "credit_reason": "The generalization is harmless: substitute the paper’s HT estimate, population effect, true variance, and realized variance estimate to recover the corrected display under exactly its two positivity conditions.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.studentized_identity", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1747", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:dc25e2c735c52edb1829c7c98303db4cf6c8fd7767aa1fc40ff4112b71fe277f", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T9", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T9" }, "html": { "body": "

corrected targets in Appendix A.6, p. 1942 (PDF p. 31), Proposition 6.2, p. 1926 (PDF p. 15), Appendix A.7, p. 1943 (PDF p. 32), and the sentence preceding Proposition 4.1, p. 1920 (PDF p. 9). HT means and their contrast are consistent for the population quantities under Conditions 3 and 4. The corrected Wald conclusion covers the population contrast, the positive-variance studentization display is an identity, and simulation guidance bounds relative bias for the fixed population total.

", "headline": "Centering and coverage targets in Appendix A.6, Proposition 6.2, Appendix A.7 and the sentence before Proposition 4.1, corrected", "location": "p. 1942 (PDF p. 31); p. 1926 (PDF p. 15); p. 1943 (PDF p. 32); p. 1920 (PDF p. 9)" }, "id": "P013-T9", "text": { "body": "corrected targets in Appendix A.6, p. 1942 (PDF p. 31), Proposition 6.2, p. 1926 (PDF p. 15), Appendix A.7, p. 1943 (PDF p. 32), and the sentence preceding Proposition 4.1, p. 1920 (PDF p. 9). HT means and their contrast are consistent for the population quantities under Conditions 3 and 4. The corrected Wald conclusion covers the population contrast, the positive-variance studentization display is an identity, and simulation guidance bounds relative bias for the fixed population total.", "headline": "Centering and coverage targets in Appendix A.6, Proposition 6.2, Appendix A.7 and the sentence before Proposition 4.1, corrected", "location": "p. 1942 (PDF p. 31); p. 1926 (PDF p. 15); p. 1943 (PDF p. 32); p. 1920 (PDF p. 9)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T9" }, { "declarations": [ { "claim_revision_id": "catalog:050e6d3f1ccb624266801af1cb902aca1323123918c68a994edf10d2a218e693", "conditional_on": [], "credit_reason": "This proves the target’s impossibility consequence for both rows; it does not invent replacement table entries.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.table2_two_rows", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L36", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:050e6d3f1ccb624266801af1cb902aca1323123918c68a994edf10d2a218e693", "conditional_on": [], "credit_reason": "Covers the target’s explicitly stated universal algebraic part (a); no rounding hypothesis is part of this declaration.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.wald_center", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L16", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:050e6d3f1ccb624266801af1cb902aca1323123918c68a994edf10d2a218e693", "conditional_on": [], "credit_reason": "Intentional impossibility statement, not an accidentally empty positive theorem; it is exactly corrected part (b).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.wls_d101_impossible", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L20", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:050e6d3f1ccb624266801af1cb902aca1323123918c68a994edf10d2a218e693", "conditional_on": [], "credit_reason": "Intentional impossibility statement, not an accidentally empty positive theorem; it is exactly corrected part (c).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.wls_d111_impossible", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L28", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:050e6d3f1ccb624266801af1cb902aca1323123918c68a994edf10d2a218e693", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T10", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T10" }, "html": { "body": "

Table 2, WLS rows τ(d101,d000)\\tau(d_{101}, d_{000})τ(d101​,d000​) and τ(d111,d000)\\tau(d_{111}, d_{000})τ(d111​,d000​): estimate and 95% CI, corrected claim; p. 1936 (PDF p. 25). Both printed WLS triples are incompatible with a symmetric interval even allowing closed three-decimal rounding bins.

", "headline": "Table 2, WLS rows τ(d101,d000)\\tau(d_{101}, d_{000})τ(d101​,d000​) and τ(d111,d000)\\tau(d_{111}, d_{000})τ(d111​,d000​): estimate and 95% CI, corrected claim", "location": "p. 1936 (PDF p. 25)" }, "id": "P013-T10", "text": { "body": "Table 2, WLS rows $\\tau(d_{101}, d_{000})$ and $\\tau(d_{111}, d_{000})$: estimate and 95% CI, corrected claim; p. 1936 (PDF p. 25). Both printed WLS triples are incompatible with a symmetric interval even allowing closed three-decimal rounding bins.", "headline": "Table 2, WLS rows $\\tau(d_{101}, d_{000})$ and $\\tau(d_{111}, d_{000})$: estimate and 95% CI, corrected claim", "location": "p. 1936 (PDF p. 25)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T10" }, { "declarations": [ { "claim_revision_id": "catalog:c784297681b5f26c35bae143741374e155a9fb257569e6e8d5643e418de76229", "conditional_on": [], "credit_reason": "This proves the actual counts, not merely a formula assumed for abstract p,q. The target-level record note saying parts (a) and (b) remain unformalized is stale relative to this declaration and R095-02.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.complete_randomization_degree39_counts", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L67", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c784297681b5f26c35bae143741374e155a9fb257569e6e8d5643e418de76229", "conditional_on": [], "credit_reason": "N≥40 follows from the target’s focal unit plus 39 distinct other neighbours; it is not an extra school-size restriction. At n=N−39 the no-exposure binomial numerator is zero and the denominator remains positive. The earlier target note about part (b) being unformalized is stale.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.degree39_choose_ratio", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L104", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c784297681b5f26c35bae143741374e155a9fb257569e6e8d5643e418de76229", "conditional_on": [], "credit_reason": "The selected counting, ratio and decimal-bound declarations provide both directions of incompatibility directly.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.degree39_design_incompatibility", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L157", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:c784297681b5f26c35bae143741374e155a9fb257569e6e8d5643e418de76229", "conditional_on": [], "credit_reason": "Auxiliary scalar implication for part (d), not a claim that this theorem alone constructs the randomization probabilities. Its assumed ratio is proved for the actual probabilities by R095-01 and R095-02.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.degree39_large_q", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L147", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c784297681b5f26c35bae143741374e155a9fb257569e6e8d5643e418de76229", "conditional_on": [], "credit_reason": "Auxiliary scalar bound for part (c); the ratio premise is not an assumption of the desired inequality and has its own separate counting derivation.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.degree39_ratio_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L131", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c784297681b5f26c35bae143741374e155a9fb257569e6e8d5643e418de76229", "conditional_on": [], "credit_reason": "Auxiliary scalar implication for the forward part of (d); the complete finite-design application is R095-03.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.degree39_small_p", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Reporting.lean#L139", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:c784297681b5f26c35bae143741374e155a9fb257569e6e8d5643e418de76229", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T11", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T11" }, "html": { "body": "

Section 9, the two descriptions of the simulation design's exposure probabilities (the degree-39 student; no no-exposure probability below 0.01), corrected claim; p. 1931 (PDF p. 20); design and exposure mapping on p. 1930 (PDF p. 19). Exact complete-randomization counting links the degree-39 exposure probabilities and proves that the two printed probability descriptions are incompatible.

", "headline": "Section 9, the two descriptions of the simulation design's exposure probabilities (the degree-39 student; no no-exposure probability below 0.01), corrected claim", "location": "p. 1931 (PDF p. 20); design and exposure mapping on p. 1930 (PDF p. 19)" }, "id": "P013-T11", "text": { "body": "Section 9, the two descriptions of the simulation design's exposure probabilities (the degree-39 student; no no-exposure probability below 0.01), corrected claim; p. 1931 (PDF p. 20); design and exposure mapping on p. 1930 (PDF p. 19). Exact complete-randomization counting links the degree-39 exposure probabilities and proves that the two printed probability descriptions are incompatible.", "headline": "Section 9, the two descriptions of the simulation design's exposure probabilities (the degree-39 student; no no-exposure probability below 0.01), corrected claim", "location": "p. 1931 (PDF p. 20); design and exposure mapping on p. 1930 (PDF p. 19)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T11" }, { "declarations": [ { "claim_revision_id": "catalog:8009c9bd33c86487e2a476901e18b1025386968f513538ca1db3d92307e393b9", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P013_ConfidenceCompletion.lean:60 negates the printed universal power bound using the actual simulated estimator on the experiment and independent replicate product designs defined at P013_Audit.lean:1487. Requiring a nonzero total in the negated assertion still refutes the claim on its ordinary defined domain; it introduces no sign assumption that would avoid the signed-outcome failure. The four-atom population at P013_Audit.lean:1559 has treatment total one and, for R=1, relative bias 7/9 against the proposed 4/9 bound, so the refutation concerns a genuine positive-probability experiment. ULift and finite relabeling change no probabilities or target cases.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ConfidenceFollowup02.simulated_bias_as_printed_refuted", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_ConfidenceCompletion.lean#L60", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:8009c9bd33c86487e2a476901e18b1025386968f513538ca1db3d92307e393b9", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'AronowAudit/P013/P013_ConfidenceCompletion.lean:74 negates the printed logarithmic replicate guarantee on the domain b > 0 and nonzero population total. The signed experiment defined in AronowAudit/P013/P013_Audit.lean:1552 has all exposure marginals strictly between zero and one, total 1, and R = 1 satisfying the logarithmic rule for b = 1/2, while its relative bias is 7/9. These domain guards do not manufacture a violation: a counterexample within this domain refutes the printed universal guarantee, irrespective of its undefined zero-total cases or whether zero replicates are allowed.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ConfidenceFollowup02.simulated_rule_as_printed_refuted", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_ConfidenceCompletion.lean#L74", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:8009c9bd33c86487e2a476901e18b1025386968f513538ca1db3d92307e393b9", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T12", "decision": "refuted", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T12" }, "html": { "body": "

The printed claim is false: the unrestricted relative-bias bound and the logarithmic replicate guarantee are both refuted on the nonzero-total domain. Proposition 4.1; p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27).

", "headline": "Proposition 4.1", "location": "p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27)" }, "id": "P013-T12", "text": { "body": "The printed claim is false: the unrestricted relative-bias bound and the logarithmic replicate guarantee are both refuted on the nonzero-total domain. Proposition 4.1; p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27).", "headline": "Proposition 4.1", "location": "p. 1920 (PDF p. 9); proof in Appendix A.3, p. 1938 (PDF p. 27)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T12" }, { "declarations": [ { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covariance_mul_const", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L177", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covariance_sum_sum", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L168", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covariance_symm", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L162", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_add", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L103", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_const", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L97", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_const_mul", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L121", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_finset_sum", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L381", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_mono", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L139", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_mul_const", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L115", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_nonneg", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L134", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_sub", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L109", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.expectation_sum", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L127", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "The observed-outcome estimator agrees with the masked fixed-potential-outcome sum by Conditions 1–2; any paper model specializes to this representation. This is an acceptable broader result, not a loss of coverage, and the claim-stage verdict is unchanged.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htEffect_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L229", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Without positivity the formula concerns the totalized real htTotal variables; under the paper’s positivity it is precisely expression (5). The extension to undefined-in-the-paper zero denominators does not alter that specialization, and the claim-stage verdict is unchanged.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htEffect_variance", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L238", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "The canonical statement asks for total and contrast expectations; mean unbiasedness is an auxiliary specialization.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htMean_unbiased", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L222", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htTotal_observed", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L199", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Exactly the printed covariance component on its stated domain; no joint-positivity or independence restriction is added. Claim-stage verdict unchanged after reading the record.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ht_covariance_formula", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L312", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ht_covariance_full", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L246", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Exactly the total-expectation component of Lemma 4.1 on the paper’s domain, with weaker assumptions on unused exposures. Claim-stage verdict unchanged after reading the record.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ht_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L209", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "This row directly establishes the variance component of Lemma 4.1; it is not merely an estimator bound or an unbiased-variance-estimator claim. Claim-stage verdict unchanged after reading the record.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ht_variance_formula", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L297", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.jointPi_cross_self", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L273", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.jointPi_nonneg", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L388", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.jointPi_self", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L263", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.sum_diag_off", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L287", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.variance_centered", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L145", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.variance_nonneg", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L156", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.variance_sub_div", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L186", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:0a287d08c1b655ce44ff5ad6ac30ea94ab7f2fbb3dcd674616787a62add92f84", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T13", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T13" }, "html": { "body": "

Lemma 4.1 and Proposition 4.2; p. 1919 (PDF p. 8); pp. 1920–1921 (PDF pp. 9–10); proofs in Appendix A.2 and A.4, p. 1938 (PDF p. 27). The exposure total and contrast have the printed expectations, variance formulas and covariance expansion.

", "headline": "Lemma 4.1 and Proposition 4.2", "location": "p. 1919 (PDF p. 8); pp. 1920–1921 (PDF pp. 9–10); proofs in Appendix A.2 and A.4, p. 1938 (PDF p. 27)" }, "id": "P013-T13", "text": { "body": "Lemma 4.1 and Proposition 4.2; p. 1919 (PDF p. 8); pp. 1920–1921 (PDF pp. 9–10); proofs in Appendix A.2 and A.4, p. 1938 (PDF p. 27). The exposure total and contrast have the printed expectations, variance formulas and covariance expansion.", "headline": "Lemma 4.1 and Proposition 4.2", "location": "p. 1919 (PDF p. 8); pp. 1920–1921 (PDF pp. 9–10); proofs in Appendix A.2 and A.4, p. 1938 (PDF p. 27)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T13" }, { "declarations": [ { "claim_revision_id": "catalog:c37f3ad99b77ab757f73c1a21304a69766d4826b3cebe67415717ca80a25d36b", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.correctionA2_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L474", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c37f3ad99b77ab757f73c1a21304a69766d4826b3cebe67415717ca80a25d36b", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pairTerm_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L397", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c37f3ad99b77ab757f73c1a21304a69766d4826b3cebe67415717ca80a25d36b", "conditional_on": [], "credit_reason": "Supporting component is already included in a selected composite declaration; no separate application is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.squareTerm_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L464", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c37f3ad99b77ab757f73c1a21304a69766d4826b3cebe67415717ca80a25d36b", "conditional_on": [], "credit_reason": "The credited component is the exposure-design bias identity of Proposition 5.1, which applies Appendix A.1 to I(D_i=d). This is not a formalization of every general-sampling formulation in the appendix.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.varianceHT_bias", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L434", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c37f3ad99b77ab757f73c1a21304a69766d4826b3cebe67415717ca80a25d36b", "conditional_on": [], "credit_reason": "The credited component is Proposition 5.2 and its squared-outcome Appendix A.5 specialization in the exposure setting; the arbitrary-conjugate-exponent version of Appendix Proposition A.2 is not claimed by this declaration.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.varianceHT_conservative", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L493", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:c37f3ad99b77ab757f73c1a21304a69766d4826b3cebe67415717ca80a25d36b", "conditional_on": [], "credit_reason": "Joint positivity includes i=j and therefore supplies positive marginal denominators; no omitted nondegeneracy premise is needed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.varianceHT_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L415", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:c37f3ad99b77ab757f73c1a21304a69766d4826b3cebe67415717ca80a25d36b", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T14", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T14" }, "html": { "body": "

Lemma 5.1 and Propositions 5.1–5.2, pp. 1921–1922 (PDF pp. 10–11), and Appendix A.5, Propositions A.1–A.2 and Corollary A.2, pp. 1939–1942 (PDF pp. 28–31). The exposure variance estimate has the stated unbiasedness, missing-joint bias and quadratic conservativeness. For general binary inclusion sampling with positive marginals, including certainty units, the same bias identity and both conjugate-power and quadratic conservative corrections hold. All missing-pair sums are ordered and outcomes may have either sign.

", "headline": "Lemma 5.1, Proposition 5.1 and Proposition 5.2", "location": "p. 1922 (PDF p. 11), with expression (7) on p. 1921 (PDF p. 10); Appendix A.5, pp. 1939–1942 (PDF pp. 28–31)" }, "id": "P013-T14", "text": { "body": "Lemma 5.1 and Propositions 5.1–5.2, pp. 1921–1922 (PDF pp. 10–11), and Appendix A.5, Propositions A.1–A.2 and Corollary A.2, pp. 1939–1942 (PDF pp. 28–31). The exposure variance estimate has the stated unbiasedness, missing-joint bias and quadratic conservativeness. For general binary inclusion sampling with positive marginals, including certainty units, the same bias identity and both conjugate-power and quadratic conservative corrections hold. All missing-pair sums are ordered and outcomes may have either sign.", "headline": "Lemma 5.1, Proposition 5.1 and Proposition 5.2", "location": "p. 1922 (PDF p. 11), with expression (7) on p. 1921 (PDF p. 10); Appendix A.5, pp. 1939–1942 (PDF pp. 28–31)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T14" }, { "declarations": [ { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "conditional_on": [], "credit_reason": "Both exposure marginal positivity assumptions and d≠e are printed target hypotheses. Conservativeness is a lower bound in expectation for covariance, with no required sign of the covariance itself.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covarianceA_conservative", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L624", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covarianceA_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L592", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "conditional_on": [], "credit_reason": "Off-diagonal cross-joint positivity is part of Proposition 5.3 itself; it is not a hidden restriction added to the unrestricted expression-(10) result.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covarianceHT_conservative", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L536", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covarianceHT_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L512", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "conditional_on": [], "credit_reason": "The no-effect premise is unitwise equality, exactly as printed, rather than merely a zero average causal effect.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covarianceHT_unbiased_no_effect", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L550", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covariance_split_zeros", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L563", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "conditional_on": [], "credit_reason": "No positive lower bound on the true variance, joint positivity, or outcome sign is imposed. The public description accurately states the finite-design expectation inequality. The N=0 extension is harmless and does not make the theorem vacuous.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.effectVariance_conservative", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L643", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:ca36946f69d121007e56393f338789469c0bafc8e09d393c1dd751c82d40c880", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T15", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T15" }, "html": { "body": "

Propositions 5.3–5.6, pp. 1923–1924 (PDF pp. 12–13). When every off-diagonal cross-exposure joint probability is positive, the first covariance estimator has expectation at most the true covariance, with equality when each unit's two selected potential outcomes agree. For arbitrary joint probabilities, the adjusted covariance estimator has expectation at most the true covariance, and the effect variance estimator has expectation at least the true variance.

", "headline": "Propositions 5.3, 5.4, 5.5 and 5.6", "location": "pp. 1923–1924 (PDF pp. 12–13)" }, "id": "P013-T15", "text": { "body": "Propositions 5.3–5.6, pp. 1923–1924 (PDF pp. 12–13). When every off-diagonal cross-exposure joint probability is positive, the first covariance estimator has expectation at most the true covariance, with equality when each unit's two selected potential outcomes agree. For arbitrary joint probabilities, the adjusted covariance estimator has expectation at most the true covariance, and the effect variance estimator has expectation at least the true variance.", "headline": "Propositions 5.3, 5.4, 5.5 and 5.6", "location": "pp. 1923–1924 (PDF pp. 12–13)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T15" }, { "declarations": [ { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.auxiliary_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L980", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covariance_abs_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1078", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.covariance_centered", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1066", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Harmless broadening, with no lost claim cases. Zero-probability exposures add only totalized identities; the statement still agrees with (12)–(13) under the standing positivity assumptions.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceTotal_residual", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1002", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Fixed prediction values are specified in T16(a1) itself, so this is not a description-only restriction. The component exports the required expectation equality.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceTotal_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L992", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "N>0 is inherent in the paper’s population mean. The target explicitly selects the finite bound; no convergence of the variances or bias ratio is asserted by this row.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.hajek_bias_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1111", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.hajek_bias_covariance", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1095", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.hajek_product_identity", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1055", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "This establishes the denominator expectation in T16(b1), not an independent proof of numerator unbiasedness or the full bias bound.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htCount_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1020", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htTotal_zero_of_count_zero", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1038", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.indicator_nonneg", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1033", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:320d0001366db24ce4ab471a66e889516256b2af0f6d1a65f67dbcc3ee430e3a", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T16", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T16" }, "html": { "body": "

Section 7, expressions (12)–(15), pp. 1927–1929 (PDF pp. 16–18). Fixed predictions give an unbiased difference total and its residual form, the inverse-probability count is unbiased, and the zero-convention Hajek ratio has the stated bias bound.

", "headline": "Section 7: unbiasedness of the difference estimator, expressions (12)–(13), and the bias bound for the Hajek estimator, expression (15)", "location": "p. 1927 (PDF p. 16); pp. 1928–1929 (PDF pp. 17–18)" }, "id": "P013-T16", "text": { "body": "Section 7, expressions (12)–(15), pp. 1927–1929 (PDF pp. 16–18). Fixed predictions give an unbiased difference total and its residual form, the inverse-probability count is unbiased, and the zero-convention Hajek ratio has the stated bias bound.", "headline": "Section 7: unbiasedness of the difference estimator, expressions (12)–(13), and the bias bound for the Hajek estimator, expression (15)", "location": "p. 1927 (PDF p. 16); pp. 1928–1929 (PDF pp. 17–18)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T16" }, { "declarations": [ { "claim_revision_id": "catalog:94d6a5b289f24e53b303973f06ee1942160fd866d8dbfcb2db481809fc844938", "conditional_on": [], "credit_reason": "The paper’s reused weight notation is specialized to the exposure of each sum; this matches the conditional-expectation meaning already fixed by the target. No verdict changed after reading the record.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.misspecified_contrast", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1161", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:94d6a5b289f24e53b303973f06ee1942160fd866d8dbfcb2db481809fc844938", "conditional_on": [], "credit_reason": "Condition 7 implies N>0. The statement retains both Condition 7 and positivity; whether a proof could omit them does not change the exported theorem type.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.misspecified_expectation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1144", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:94d6a5b289f24e53b303973f06ee1942160fd866d8dbfcb2db481809fc844938", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T17", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T17" }, "html": { "body": "

Proposition 8.1 and Corollary 8.1, pp. 1929–1930 (PDF pp. 18–19). With misspecified exposures the HT total and contrast have the stated weighted randomization-potential-outcome expectations.

", "headline": "Proposition 8.1 and Corollary 8.1", "location": "p. 1929 (PDF p. 18); p. 1930 (PDF p. 19); proof in Appendix A.8, pp. 1943–1944 (PDF pp. 32–33)" }, "id": "P013-T17", "text": { "body": "Proposition 8.1 and Corollary 8.1, pp. 1929–1930 (PDF pp. 18–19). With misspecified exposures the HT total and contrast have the stated weighted randomization-potential-outcome expectations.", "headline": "Proposition 8.1 and Corollary 8.1", "location": "p. 1929 (PDF p. 18); p. 1930 (PDF p. 19); proof in Appendix A.8, pp. 1943–1944 (PDF pp. 32–33)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T17" }, { "declarations": [ { "claim_revision_id": "catalog:91240060387d39737426df3268caf879c20735865fac0d5b0f01dea3a8671ff6", "conditional_on": [], "credit_reason": "Final wrong_credit: the symmetric counting lemma does not supply the printed directed counting claim; the independent refutation row suffices.", "credit_status": "wrong_credit", "fidelity_status_baseline": "wrong_credit", "lean_name": "AronowAudit.P013.graph_quadruple_count", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2760", "status": "proved", "support_scope": "withdrawn", "used_in_solution": false }, { "claim_revision_id": "catalog:91240060387d39737426df3268caf879c20735865fac0d5b0f01dea3a8671ff6", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': \"AronowAudit/P013/Wave1/PrintedWald.lean:34 negates the universal coverage bound with nesting, standing positivity, Conditions 3 and 6, and the literal printed Condition 5. Reading the undefined un-hatted τ_HT as the population effect and expressing asymptotic 'at least' coverage by a liminf, as encoded at AronowAudit/P013/P013_Audit.lean:1747, preserves the natural content of Proposition 6.2. Although that definition counts negative estimated variance as noncoverage, the explicit family in AronowAudit/P013/Wave1/PrintedWaldBlueprint.lean:87 has positive estimated variance on even-size events whose probability tends to one and variance ratio tending to 3/4. Thus the undercoverage does not depend on that convention, and the refutation is faithful to the target rather than to an unrelated reading.\"}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.Wave1.printed_wald_coverage_refuted", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/Wave1/PrintedWald.lean#L34", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:91240060387d39737426df3268caf879c20735865fac0d5b0f01dea3a8671ff6", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T18", "decision": "refuted", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T18" }, "html": { "body": "

The printed claim is false: Conditions 3, printed 5 and 6 do not guarantee the asserted limiting Wald coverage for every nested population sequence. Proposition 6.2, p. 1926 (PDF p. 15), Appendix A.7, p. 1943 (PDF p. 32). Coverage counts a negative variance estimate as noncoverage. In the source paired witness, its probability along the even-size subsequence showing undercoverage is at most 1/N and tends to zero; the refutation does not rely on undefined intervals.

", "headline": "Proposition 6.2", "location": "p. 1926 (PDF p. 15); proof in Appendix A.7, p. 1943 (PDF p. 32)" }, "id": "P013-T18", "text": { "body": "The printed claim is false: Conditions 3, printed 5 and 6 do not guarantee the asserted limiting Wald coverage for every nested population sequence. Proposition 6.2, p. 1926 (PDF p. 15), Appendix A.7, p. 1943 (PDF p. 32). Coverage counts a negative variance estimate as noncoverage. In the source paired witness, its probability along the even-size subsequence showing undercoverage is at most 1/N and tends to zero; the refutation does not rely on undefined intervals.", "headline": "Proposition 6.2", "location": "p. 1926 (PDF p. 15); proof in Appendix A.7, p. 1943 (PDF p. 32)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T18" }, { "declarations": [ { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.both_means_consistent", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2533", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.bounded_htMean_consistency", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2475", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.convergence_difference", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2586", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.difference_probability_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2562", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.finite_chebyshev", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1755", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "The statement itself is faithful and its claim-stage verdict is unchanged. All-exposure positivity and nesting are paper assumptions, even though stronger auxiliary statements do not need them. The original description's broader-coverage sentence should refer to those auxiliaries or to a separately stated generalization.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htEffect_consistency", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2610", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htMean_chebyshev", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2423", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.htMean_variance_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2383", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.jointPi_bounds", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2351", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.joint_covariance_abs_le_one", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2361", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.linear_count_is_little_o", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2431", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.pi_bounds", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2341", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.variance_div", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2372", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:498004e1d638f565e927b0d06ff01d7c18325632301587471bce1341a3ca235e", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T19", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T19" }, "html": { "body": "

Proposition 6.1, p. 1925 (PDF p. 14), Appendix A.6, p. 1942 (PDF p. 31). Under uniform Condition 3 and Condition 4 the HT effect estimation error converges in probability to zero.

", "headline": "Proposition 6.1", "location": "p. 1925 (PDF p. 14); proof in Appendix A.6, p. 1942 (PDF p. 31)" }, "id": "P013-T19", "text": { "body": "Proposition 6.1, p. 1925 (PDF p. 14), Appendix A.6, p. 1942 (PDF p. 31). Under uniform Condition 3 and Condition 4 the HT effect estimation error converges in probability to zero.", "headline": "Proposition 6.1", "location": "p. 1925 (PDF p. 14); proof in Appendix A.6, p. 1942 (PDF p. 31)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T19" }, { "declarations": [ { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: supporting finite infrastructure is already incorporated in the selected IID matrix-limit theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.design_matrix_entry", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2108", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: supporting finite infrastructure is already incorporated in the selected IID matrix-limit theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.integral_designMeasure", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2172", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: supporting finite infrastructure is already incorporated in the selected IID matrix-limit theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_count_expectation", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2142", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: the selected simultaneous matrix-limit row incorporates this strong law under the same IID assumptions.", "credit_status": "wrong_credit", "fidelity_status_baseline": "wrong_credit", "lean_name": "AronowAudit.P013.replicate_joint_strong_law", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2191", "status": "proved", "support_scope": "withdrawn", "used_in_solution": true }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: supporting finite infrastructure is already incorporated in the selected IID matrix-limit theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_marginal", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1192", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: supporting finite infrastructure is already incorporated in the selected IID matrix-limit theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_matrix_entry", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2134", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: supporting finite infrastructure is already incorporated in the selected IID matrix-limit theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_matrix_unbiased", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2157", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: supporting finite infrastructure is already incorporated in the selected IID matrix-limit theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.replicate_pair_marginal", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L1219", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "{'source': 'adjudication+second review', 'adjudicator': 'AronowAudit/P013/P013_Audit.lean:2464 requires hi : ProbabilityTheory.iIndepFun Z μ, whereas T20 specifies random replicate assignment vectors drawn according to the randomization plan without specifying independence. Exact design marginals alone are insufficient: repeat one random assignment forever in a two-unit independent fair binary design, and a smoothed diagonal entry converges to a random zero or one rather than its design probability 1/2. The paper also describes sampling without replacement in Appendix A.1, which cannot generate an infinite sequence from its fixed finite support. The record expressly adopts an IID interpretation, but that repair appears only outside the supplied claim text; under the required comparison it does not authorize the additional hypothesis. Both matrix limits and their simultaneous finite-entry formulation otherwise match the claim.', 'suggested_fix': 'Reassign the row to an explicitly corrected Proposition 3.1 stating that the replicates are independent draws with replacement, each having the exact design law. Retain the Lean statement for that correction and replace the appendix sampling description accordingly; removing hi does not establish the supplied unrestricted target.', 'second_review': \"P013/P013_Audit.lean:2464 requires mutual independence of the infinite replicate sequence, while T20's claim text requires design-law random replicates without stating independence. In a two-unit independent fair binary design, reusing one random whole assignment forever gives each replicate the correct design law, but the smoothed diagonal at line 2467 tends to a random 0 or 1 rather than 1/2. The [paper's Appendix A.1](https://arxiv.org/pdf/1305.6156#page=29) describes sampling without replacement from finite Ω, which cannot justify this infinite IID sequence; both matrix limits otherwise match the credited conclusion. Reassign the declaration to an explicitly corrected target specifying independent draws with replacement, since deleting the Lean independence premise would leave a false unrestricted claim.\", 'severity': 'major'}", "credit_status": "wrong_credit", "fidelity_status_baseline": "wrong_credit", "lean_name": "AronowAudit.P013.simulation_matrices_almost_sure", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2248", "status": "proved", "support_scope": "withdrawn", "used_in_solution": true }, { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "conditional_on": [], "credit_reason": "Not separately invoked: the selected matrix-limit row already includes same-exposure identity smoothing.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.smoothed_average_limit", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L2229", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:91046cdc16bfcba5f3d54c99a03a5d0e2a38b69fa3b0505017be40fe0171d214", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T20", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T20" }, "html": { "body": "

Proposition 3.1, p. 1917 (PDF p. 6), Appendix A.1, p. 1938 (PDF p. 27). On a fixed finite population, an infinite sequence of mutually independent replicate assignments, each drawn from the exact design law, gives almost surely, simultaneously for every exposure `d_k` and every entry, convergence of the smoothed same-exposure matrix `(Î_kÎ_k' + ι_N)/(R+1)` to `I_kPI_k'` and, for every exposure `d_l ≠ d_k`, of the cross-exposure matrix `Î_kÎ_l'/R` to `I_kPI_l'`. Both limits hold on one probability-one event for every exposure and entry, including the same-exposure limit when there is only one exposure. The usual measure-theoretic conditions are assumed: measurable replicates on a probability space and measurable singletons in the finite assignment space. The zero-replicate cross ratio has no effect on the limit; its denominator is positive for every R ≥ 1.

", "headline": "Proposition 3.1", "location": "p. 1917 (PDF p. 6); proof in Appendix A.1, p. 1938 (PDF p. 27)" }, "id": "P013-T20", "text": { "body": "Proposition 3.1, p. 1917 (PDF p. 6), Appendix A.1, p. 1938 (PDF p. 27). On a fixed finite population, an infinite sequence of mutually independent replicate assignments, each drawn from the exact design law, gives almost surely, simultaneously for every exposure `d_k` and every entry, convergence of the smoothed same-exposure matrix `(Î_kÎ_k' + ι_N)/(R+1)` to `I_kPI_k'` and, for every exposure `d_l ≠ d_k`, of the cross-exposure matrix `Î_kÎ_l'/R` to `I_kPI_l'`. Both limits hold on one probability-one event for every exposure and entry, including the same-exposure limit when there is only one exposure. The usual measure-theoretic conditions are assumed: measurable replicates on a probability space and measurable singletons in the finite assignment space. The zero-replicate cross ratio has no effect on the limit; its denominator is positive for every R ≥ 1.", "headline": "Proposition 3.1", "location": "p. 1917 (PDF p. 6); proof in Appendix A.1, p. 1938 (PDF p. 27)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T20" }, { "declarations": [ { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Exposure preservation and consistent outcome translation are encoded directly by the SAP shift definition.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.shift_population_objects", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3100", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceTotal_weighted", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3108", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (a), first identity. The positive population size is implicit in the claim’s population means; the relevant positivity is explicitly present.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceTotal_shift_change", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3181", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (a), second identity; translationError is the exact displayed sum and is not an assumption of zero error.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceTotal_shift_error", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3190", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Single-exposure component of part (c); the other comparison exposure is irrelevant to this conclusion.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceTotal_equivariance_iff", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3198", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (b) total component; applying the same universally quantified theorem at the other label supplies its total equality too.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceTotal_joint_translation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3206", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (b). Population-mean shifts are algebraically equivalent to the exported population-total shifts after unfolding potentialMean and using N>0; no independent conclusion is missing.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.difference_joint_translation_properties", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3216", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "This is a proof of the corrected claim’s part (c), not a refutation of the paper’s unqualified printed sentence.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.differenceEffect_shift_error", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3249", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (d); c≠0 is assumed only for the two calibration equivalences, as in the target.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.difference_fixed_prediction_translation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3275", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Supporting component of part (e). The weaker unitwise hypothesis is a harmless generalization. Claim-stage verdict retained.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.exposureWeight_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3313", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (e), all four single-exposure expectation conclusions. Fixed vectors cannot depend on the randomized assignment. Allowing N=0 adds only trivial empty-sum instances to a nontrivial theorem. Claim-stage verdict retained.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.translation_error_mean_zero", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3347", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Fixed-prediction specializations in part (e); these are expectation equalities, not pointwise invariance claims.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.fixed_translation_mean_zero", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3389", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (f). The count hypothesis implies m≤N, and m≥1,N>0 imply positive marginals. With N=2,m=1,c=1 and unchanged predictions this also exhibits why joint prediction translation is sufficient but unnecessary. Claim-stage verdict retained.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.fixed_count_calibration", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3404", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "conditional_on": [], "credit_reason": "Part (d), Horvitz–Thompson specialization. Zero predictions identify the HT estimators; the statement does not require outcomes or the shift to vanish.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ht_fixed_translation", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3324", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:43c5e3639127746eda9feaf89480dd77d3ef20e8543886eff3494c7fc5eed924", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T22", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T22" }, "html": { "body": "

corrected Section 7.1, p. 1927 (PDF p. 16). Exact translation formulas give equivariance, invariance and their characterizations, zero mean translation errors, and calibration with fixed predictions, including a two-unit example.

", "headline": "Section 7.1, location equivariance of the difference estimator of expressions (12)–(13), corrected", "location": "p. 1927 (PDF p. 16)" }, "id": "P013-T22", "text": { "body": "corrected Section 7.1, p. 1927 (PDF p. 16). Exact translation formulas give equivariance, invariance and their characterizations, zero mean translation errors, and calibration with fixed predictions, including a two-unit example.", "headline": "Section 7.1, location equivariance of the difference estimator of expressions (12)–(13), corrected", "location": "p. 1927 (PDF p. 16)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T22" }, { "declarations": [ { "claim_revision_id": "catalog:41b1c9eb73cee9b6f15c7d44dc4b4a37c2f4bbe9a4dcfec11bbf40147aa75e8d", "conditional_on": [], "credit_reason": "Hypothesis-certification component of the counterexample, not a standalone refutation. Both exposure-specific predictions are the same fixed vector, and its exposure-1 indicator is precisely own treatment.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.location_counterexample_hypotheses", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3448", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:41b1c9eb73cee9b6f15c7d44dc4b4a37c2f4bbe9a4dcfec11bbf40147aa75e8d", "conditional_on": [], "credit_reason": "Baseline and true-target component of T21(b)–(c); no translation failure is claimed from this lemma alone.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.location_counterexample_targets", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3474", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:41b1c9eb73cee9b6f15c7d44dc4b4a37c2f4bbe9a4dcfec11bbf40147aa75e8d", "conditional_on": [], "credit_reason": "Assignmentwise arithmetic component of T21(c)–(d); the same concrete construction is checked against all printed hypotheses, and R096-17 supplies the violation.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.location_counterexample_enumeration", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3497", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:41b1c9eb73cee9b6f15c7d44dc4b4a37c2f4bbe9a4dcfec11bbf40147aa75e8d", "conditional_on": [], "credit_reason": "The printed phrase “location invariant” is not formally defined in the paper; the witness is exact for the fixed-numerical-prediction reading explicitly included in T21. It uses positive-probability assignments, positive denominators and nonempty groups, and R096-14 confirms restoration under joint recoding. The record does not change this saved judgment.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.location_translation_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3515", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:41b1c9eb73cee9b6f15c7d44dc4b4a37c2f4bbe9a4dcfec11bbf40147aa75e8d", "conditional_on": [], "credit_reason": "Moment component of the counterexample; the unshifted means also follow from the constant estimators exported by R096-16. The theorem need not separately restate those means to establish T21(e).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.location_counterexample_moments", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3542", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:41b1c9eb73cee9b6f15c7d44dc4b4a37c2f4bbe9a4dcfec11bbf40147aa75e8d", "conditional_on": [], "credit_reason": "This is T21(f), a concrete repair component, not a purported counterexample to joint recoding or a universal repair theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.location_counterexample_joint_repair", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3560", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:41b1c9eb73cee9b6f15c7d44dc4b4a37c2f4bbe9a4dcfec11bbf40147aa75e8d", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T21", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T21" }, "html": { "body": "

Section 7.1, p. 1927 (PDF p. 16), fixed-numerical-prediction reading. The specified three-unit design gives all baseline, shifted, error, moment and joint-repair facts refuting the asserted location property when predictions are held fixed.

", "headline": "Section 7.1, the two statements that the difference estimator of expressions (12)–(13) is location invariant, as printed", "location": "p. 1927 (PDF p. 16)" }, "id": "P013-T21", "text": { "body": "Section 7.1, p. 1927 (PDF p. 16), fixed-numerical-prediction reading. The specified three-unit design gives all baseline, shifted, error, moment and joint-repair facts refuting the asserted location property when predictions are held fixed.", "headline": "Section 7.1, the two statements that the difference estimator of expressions (12)–(13) is location invariant, as printed", "location": "p. 1927 (PDF p. 16)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T21" }, { "declarations": [ { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.recorded_pi", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3867", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.observed_exposure_sum", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3873", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.ht_positive_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3881", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.recorded_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3897", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.recorded_mean_decomposition", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3917", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.recorded_wearing_identification", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3938", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.lost_mean_zero_iff", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3958", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "The nonnegativity assumptions are explicitly those of target (d), not an extra restriction on the target’s arbitrary-real-outcome parts (a)–(c).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.no_lost_wearing_suffices", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L3980", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "All four exported parts match (e), including the conditional positive-retained-denominator Hajek-mean formula. This row does not purport to prove a universal factor for Hajek contrasts.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.dropping_zero_outcomes", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4009", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "The inaccurate model boilerplate is cosmetic and does not alter the correctly described mathematical credit. Arithmetic does not prove the public-data provenance, a limitation already present in the saved claim-stage assessment; no verdict or severity has changed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.public_reconstruction_factors", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4041", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "Covers the corrected mathematical assertions (a)–(d), not the empirical reconstruction or exclusions. Identification means the expectation equalities explicitly specified in (c); no recovery functional for arbitrary observed-data distributions is claimed. No claim-stage verdict changed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.corrected_recorded_outcome", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4257", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "The count and zero-exclusion premises are facts explicitly supplied by the target’s reconstruction description, so they are not additional exclusions from this row’s conditional-algebra component. Their empirical verification remains uncovered at target level. conditional_on is empty: this is not a hypothesis invoking an external published theorem.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.public_reconstruction_HT_scaling", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4287", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "conditional_on": [], "credit_reason": "A witness for the positive corrected assertion that no scalar works in general, not a refutation of the corrected target. The record reports a separate manifest label proved versus the packet status counterexample_proved; the mathematical witness purpose is clear and consistent with target (e).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.hajek_exclusion_no_common_factor", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4308", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:6164eab09dfaa55578c5144cc6d24b2ad8a4491eebf698a73f36d0aac21a7f00", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T24", "decision": "restricted", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T24" }, "html": { "body": "

corrected Section 10 and Table 2, pp. 1934–1936 (PDF pp. 23–25). Recorded-outcome HT estimation has the stated loss decomposition, identification conditions, sufficient condition, exclusion formulas, unequal Hajek factors and conditional numeric scaling. Restricted: the scaling clause assumes item and survey are subsets of S, with |S| = 2050, |item| = 1708 and |survey| = 1899, positive weights on S, and retention of every nonzero recorded score in both sets. It does not assume or establish item ⊆ survey: containment of survey nonrespondents among item nonrespondents is omitted. The reconstruction's actual counts and zero coding, its link to the released orange-wristband indicator, and reproduction of the twelve rounded Table 2 estimates are not verified. The strictness of the comparison between the sufficient and necessary conditions is not separately asserted.

", "headline": "Section 10 and Table 2, the outcome and what the effect estimates characterize when students with no recorded answer are kept and scored zero, corrected claim", "location": "p. 1936 (PDF p. 25); outcome introduced on p. 1934 (PDF p. 23); sample on p. 1935 (PDF p. 24)" }, "id": "P013-T24", "text": { "body": "corrected Section 10 and Table 2, pp. 1934–1936 (PDF pp. 23–25). Recorded-outcome HT estimation has the stated loss decomposition, identification conditions, sufficient condition, exclusion formulas, unequal Hajek factors and conditional numeric scaling. Restricted: the scaling clause assumes item and survey are subsets of S, with |S| = 2050, |item| = 1708 and |survey| = 1899, positive weights on S, and retention of every nonzero recorded score in both sets. It does not assume or establish item ⊆ survey: containment of survey nonrespondents among item nonrespondents is omitted. The reconstruction's actual counts and zero coding, its link to the released orange-wristband indicator, and reproduction of the twelve rounded Table 2 estimates are not verified. The strictness of the comparison between the sufficient and necessary conditions is not separately asserted.", "headline": "Section 10 and Table 2, the outcome and what the effect estimates characterize when students with no recorded answer are kept and scored zero, corrected claim", "location": "p. 1936 (PDF p. 25); outcome introduced on p. 1934 (PDF p. 23); sample on p. 1935 (PDF p. 24)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T24" }, { "declarations": [ { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "The type of the wearing and recording tables already makes them exposure-specific; observed recorded outcomes are evaluated at the received exposure. The exported consistency identity is for g, exactly as target part (a) says.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_report_hypotheses", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4078", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "The equality is pointwise for the whole assignment-indexed record, including exposures and recording flags. The ordinary zero-based world relabeling is taken as exact indexing, not a mathematical discrepancy.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_report_data_same", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4097", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "This is the full-sample N=2 normalization from target part (c), with exact π_i=1/2. The conventional zero-based labels do not alter the four assignment values.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_report_estimator", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4106", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "The final estimand equality concerns recorded g, not wearing y. Variance is the finite randomization variance E[X²]-E[X]², so no measure/integrability convention is involved.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_report_moments", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4117", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "The numerical wearing and lost-mean tables exactly match parts (d) and (e) after the ordinary world-index shift. Combined with R098-11 they imply E[τhat_G]-τ_y=L(0)-L(1) in every world.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_wearing_effects_and_losses", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4132", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': \"AronowAudit/P013/P013_Audit.lean:4406 gives exactly the recorded-estimator biases 0, -1/2, and +1/2 required by T23(d), using the four equally weighted assignments and wearing tables defined at lines 4312 and 4320. Lean's worlds 0, 1, 2 simply relabel the target's worlds 1, 2, 3. The target's own text explicitly adds zero scoring of missing reports and distinguishes recorded outcomes from wearing outcomes, so the absence of Condition 2 for wearing in the biased worlds is part of the specified counterexample rather than a fidelity defect.\"}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_report_biases", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4146", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': \"AronowAudit/P013/P013_Audit.lean:4417 asserts that no single real-valued function of the complete recorded-data array equals the wearing effect in all three worlds. Its input includes exposures, recorded outcomes, and recording indicators on every assignment through recordedData at line 4332, exactly the information set in T23(ii); the design is fixed and common to all worlds. Identical inputs with wearing effects 1/2, 1, and 0 give the target's deterministic nonidentification certificate, and the world relabeling loses no case or conclusion.\"}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_report_nonidentification", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4157", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "conditional_on": [], "credit_reason": "All four clauses of the respondent comparison are exported. Together with R098-11, the constant expectation gap 1/2 is compatible with zero, negative and positive full-sample wearing bias.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.two_respondent_comparison", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4178", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:568e50b838ec45c3c79449db2b817640751d29b3297a00087460c24dd0072d4e", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T23", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T23" }, "html": { "body": "

Section 10 and Table 2, p. 1936 (PDF p. 25), pp. 1934–1935 (PDF pp. 23–24). The specified three-world two-unit example has identical records, different wearing effects, the asserted estimator and loss tables, nonidentification, and the respondent comparison. The refutation concerns wearing interpretation combined with the additional zero-scoring rule.

", "headline": "Section 10 and Table 2, the reading of the effect estimates as effects on wearing for all eligible students, when students with no recorded answer are kept and scored zero: exact two-unit example", "location": "p. 1936 (PDF p. 25); outcome introduced on p. 1934 (PDF p. 23); sample on p. 1935 (PDF p. 24)" }, "id": "P013-T23", "text": { "body": "Section 10 and Table 2, p. 1936 (PDF p. 25), pp. 1934–1935 (PDF pp. 23–24). The specified three-world two-unit example has identical records, different wearing effects, the asserted estimator and loss tables, nonidentification, and the respondent comparison. The refutation concerns wearing interpretation combined with the additional zero-scoring rule.", "headline": "Section 10 and Table 2, the reading of the effect estimates as effects on wearing for all eligible students, when students with no recorded answer are kept and scored zero: exact two-unit example", "location": "p. 1936 (PDF p. 25); outcome introduced on p. 1934 (PDF p. 23); sample on p. 1935 (PDF p. 24)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T23" }, { "declarations": [ { "claim_revision_id": "catalog:dd27adb7a10202ef14e2acc5d65a1e016077f20edd189fa98b29151322cc291f", "conditional_on": [], "credit_reason": "The row establishes the formal wearing-outcome reading under Conditions 1–2, without identifying missing reports as wearing outcomes. The queried network ambiguity is resolved consistently by Section 10 of [arXiv v4](https://arxiv.org/pdf/1305.6156): PDF p. 25 describes the network as undirected and PDF p. 24 specifies 56 schools; publisher access was blocked. Empirical point estimates and Hajek/WLS behavior are not claimed in the target’s formal reading.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.section10_wearing_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Audit.lean#L4225", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:dd27adb7a10202ef14e2acc5d65a1e016077f20edd189fa98b29151322cc291f", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T27", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T27" }, "html": { "body": "

Section 10 and Table 2, pp. 1934–1936 (PDF pp. 23–25), formal reading. For binary wearing outcomes observed consistently, the four exposure contrasts are differences of wearing proportions and the HT estimators are unbiased for them.

", "headline": "Section 10 and Table 2: the outcome and what the effect estimates characterize, as printed", "location": "p. 1936 (PDF p. 25); p. 1934 (PDF p. 23); p. 1935 (PDF p. 24)" }, "id": "P013-T27", "text": { "body": "Section 10 and Table 2, pp. 1934–1936 (PDF pp. 23–25), formal reading. For binary wearing outcomes observed consistently, the four exposure contrasts are differences of wearing proportions and the HT estimators are unbiased for them.", "headline": "Section 10 and Table 2: the outcome and what the effect estimates characterize, as printed", "location": "p. 1936 (PDF p. 25); p. 1934 (PDF p. 23); p. 1935 (PDF p. 24)" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T27" }, { "declarations": [ { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.uniform_school_probability_mean", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L57", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.uniform_school_mean_const", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L64", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.uniform_school_mean_add", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L71", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.uniform_school_mean_sub", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L78", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.uniform_school_mean_mul", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L85", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_block_choice_count", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L90", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_assignment_counts", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L95", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_supports_nonempty", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L109", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_local_marginal_count", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L116", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_local_joint_count", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L127", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_filter_one_block", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L138", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_filter_two_blocks", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L160", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_product_one", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L190", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_product_two", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L203", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_B_marginal", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L218", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_B_joint", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L228", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_C_marginal", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L260", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_C_joint", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L276", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "T26(e), exact numerical differences; the theorem is arithmetic, with design identifications supplied by the other credited rows.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_joint_differences", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L292", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.uniform_school_pair_law", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L298", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_B_pair_distribution", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L329", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_C_pair_distribution", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L349", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Fidelity sweep: ok", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_B_embeds_C", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L362", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "A structural helper; strict support inclusion is stated using the existing explicit support counterexample.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.block_school_hosts_injective", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L378", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_local_mean", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L612", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P013_Later.lean:666 equates every individual marginal and every same-school joint under the two specified school designs, exactly the component in T26(g). The common independent student law is an explicit premise of that claim, and every finite student-assignment law and fixed network is represented by the four hosted codes at P013_Later.lean:632. Allowing additional code arrays strengthens the auxiliary result, while excluding N=0 removes only empty universal conclusions.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_exposure_probabilities_unchanged", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L665", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P013_Later.lean:695 states that the no-exposure label occurs exactly when hosting is false. Its definition at P013_Later.lean:632 assigns label zero to nonhosting and labels one through four to the four hosted cases, matching the printed condition for d000 in T26. This definitional representation preserves every case of the component.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_hosted_no_exposure", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L694", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "conditional_on": [], "credit_reason": "T26(g), only its specified d000 marginals, and T26(h), its different-school d000 joints; no claim to establish all exposure-probability invariance is attributed to this row. A finite population has finitely many binary student assignment vectors (and finitely many hosted-code vectors), so finite Q loses no student-law cases. The code rule encodes the printed exposure definition; it does not assume the probability formulas. Independence is exactly an explicit assumption of T26(g).", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_no_exposure_probabilities", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L708", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:0ad106fe05e8fd9689a6fd45b75495723d6b9db731e9fd4141deda62ca08247b", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T26", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T26" }, "html": { "body": "

corrected Section 10, p. 1934 (PDF p. 23), released scripts 1 and 7. The two specified school laws have the stated counts, strict support inclusion, marginals, joint laws, differences and student exposure preservation and nonhosting formulas.

", "headline": "Section 10, school-level block randomization of 28 of 56 schools: exact hosting probabilities under the stated design and under the school lottery of the field experiment's released script 1, corrected claim", "location": "p. 1934 (PDF p. 23); released script 1, line 58, and script 7, lines 73–74 and 87–90" }, "id": "P013-T26", "text": { "body": "corrected Section 10, p. 1934 (PDF p. 23), released scripts 1 and 7. The two specified school laws have the stated counts, strict support inclusion, marginals, joint laws, differences and student exposure preservation and nonhosting formulas.", "headline": "Section 10, school-level block randomization of 28 of 56 schools: exact hosting probabilities under the stated design and under the school lottery of the field experiment's released script 1, corrected claim", "location": "p. 1934 (PDF p. 23); released script 1, line 58, and script 7, lines 73–74 and 87–90" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T26" }, { "declarations": [ { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "This is an exact witness separating the two target laws. The target explicitly identifies the refuted assertion as a nonprinted identification of designs, and expressly leaves the printed block-randomization sentence intact.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_support_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L393", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "Not separately invoked: the selected result rows supply the claim clauses and already incorporate their supporting finite identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_sum_two", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L464", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "The definition replaces masked observed outcomes by their equal fixed hosting potentials, valid because the witness defines observation at the actual binary exposure. The hosting total and full HT reduction are both exported.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_two_HT_reduction", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L478", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "This is supporting algebra for T25(b), with the correct ordered-pair coefficient eight. Allowing arbitrary q loses no positive-q cases; the q=0 extension is interpreted using Lean’s totalized division.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_two_variance_reduction", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L492", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "The broader q domain is harmless for the target’s q>0 specialization; the q=0 extension has no intended probability-input interpretation. The fair-marginal hypotheses exclude the empty-support division convention.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_two_moments", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L526", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "The row covers the moment and nonconservativeness clauses in T25(c–e); the support witness and potential-outcome properties are covered by the companion rows. The positive true joints satisfy the printed theorem’s assumptions, but q=27/110 differs from the actual B joint, so this is a counterexample to the stated wrong-input guarantee only.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_wrong_joint_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L580", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "This is a constructed population for the counterexample. The observation function is explicitly defined to obey consistency, and its potentials depend only on the hosting exposure; this legitimately supplies Conditions 1–2 rather than assuming an unbiasedness result.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_example_potential_hypotheses", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L760", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "This row covers only the q=p clause of T25(b), not the general variance-unbiasedness theorem. Uniform finite sums exactly represent both target designs; no arbitrary-outcome claim is credited here.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_correct_joint_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L782", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "conditional_on": [], "credit_reason": "The row establishes the exact bias clauses of T25(c,d) for the constructed population. Neither these biases nor the target assert that Table 2 or a printed variance theorem is wrong.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P013.school_wrong_joint_biases", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P013/P013_Later.lean#L796", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:eef54a115987317e3f7df8690e87acd6e1a82c370c6db2f44e23a4f75ba083fc", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P013-T25", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P013-T25" }, "html": { "body": "

Section 10, p. 1934 (PDF p. 23), expression (7), p. 1921 (PDF p. 10). The specified school designs differ in support and give the stated moment formulas, wrong-joint variance biases and correct-joint equality for a 56-school population in which exactly two schools have nonzero hosted potential outcomes. This is the canonical counterexample to identifying the two specified lotteries.

", "headline": "Section 10, school-level block randomization, against the school lottery drawn by the field experiment's released script 1 (complete randomization of 28 of 56 schools): exact counterexample to their being one design, with the effect on the variance estimator of expression (7)", "location": "p. 1934 (PDF p. 23); expression (7) on p. 1921 (PDF p. 10); released script 1, line 58" }, "id": "P013-T25", "text": { "body": "Section 10, p. 1934 (PDF p. 23), expression (7), p. 1921 (PDF p. 10). The specified school designs differ in support and give the stated moment formulas, wrong-joint variance biases and correct-joint equality for a 56-school population in which exactly two schools have nonzero hosted potential outcomes. This is the canonical counterexample to identifying the two specified lotteries.", "headline": "Section 10, school-level block randomization, against the school lottery drawn by the field experiment's released script 1 (complete randomization of 28 of 56 schools): exact counterexample to their being one design, with the effect on the variance estimator of expression (7)", "location": "p. 1934 (PDF p. 23); expression (7) on p. 1921 (PDF p. 10); released script 1, line 58" }, "url": "https://pmaronow.github.io/llm-audit/works/P013/#P013-T25" } ] }, "missing_sources": [], "reproduction": { "agreement": "agrees_in_part", "status": "partial", "targets": [ { "agreement": "not_compared", "location": "p. 1933 (PDF p. 22); simulation specification p. 1932 (PDF p. 21)", "method": "I inspected the simulation code and checked the printed table for internal consistency. I could not reproduce the simulation because its Add Health outcome and network files were unavailable.", "status": "blocked", "target": "Table 1" }, { "agreement": "not_compared", "location": "p. 1944 (PDF p. 33), Appendix B; numerical text p. 1931 (PDF p. 20)", "method": "I inspected the exposure-probability code and the printed descriptions. I could not reproduce Figure 3 or its numerical summaries because the Add Health school network files were unavailable.", "status": "blocked", "target": "Figure 3 and exposure-probability text" }, { "agreement": "agrees_in_part", "location": "p. 1935 (PDF p. 24)", "method": "I adapted the authors' probability code and reconstructed sample membership. A 1% screen with the legacy R sampler gives the printed 2,050 students; the stated positive-probability rule gives 2,196. Original membership is not established.", "status": "recalculated", "target": "Field analysis sample" }, { "agreement": "agrees_in_part", "location": "pp. 1934–1935 (PDF pp. 23–24)", "method": "I adapted the authors' probability code and computed exact probabilities. School counts, eligibility ranges and the exposure mapping agree; the assignment script confirms school blocks, which the probability script omits. The construction of student blocks was not reproduced.", "status": "recalculated", "target": "Field design and exposure probabilities" }, { "agreement": "agrees_in_part", "location": "p. 1936 (PDF p. 25)", "method": "I reconstructed expressions (1), (7)–(11), (14) and (15) on public data, adapting the authors' probability code. All twelve point estimates agree to three decimals on the inferred 2,050-person sample; the twelve standard errors and data-based intervals do not reproduce.", "status": "recalculated", "target": "Table 2" }, { "agreement": "agrees_in_part", "location": "p. 1936 (PDF p. 25)", "method": "I reconstructed the public-data analysis. The no-exposure mean of 0.000 and approximate effect magnitudes agree on the inferred sample; the printed uncertainty ranges do not.", "status": "recalculated", "target": "Field application in-text numbers" } ] }, "summary": "The paper asks how to estimate average causal effects in a randomized experiment when one unit's treatment can affect another's outcome. It gives Horvitz--Thompson estimators with conservative variance estimators and asymptotic results, a simulation and a field application. 21 entries are classified as errors (4 Moderate, 17 Minor) in the article and related software, and 7 are clarity notes; 9 further concerns were not confirmed, 4 remain unresolved and 1 is outside the scope. Under the intended dependence condition, Proposition 6.2 on interval coverage can be repaired by adding a uniform positive lower bound on positive joint exposure probabilities used by its variance estimator (\\fid{P013-E001}); this bound holds for the simulation and application designs under the stated regularity conditions.", "what_was_checked": "I read the article in full, the source of the later R package \\texttt{interference} (version 0.1.0, credited to Stephanie Zonszein, Cyrus Samii and P. M. Aronow) and a third party's replication study. For the application of Section 10, I also read the materials of the field experiment. I read its eleven analysis scripts in full against Section 10 and Table 2. I used the public data release (ICPSR study 37070, version 2), and I read the entries of its codebook and questionnaire on the design, the nominations and the wristband. I checked the following and found them correct, with the qualifications stated.\n\\begin{itemize}\n\\item Section 3: the matrix identities for the exposure probabilities; Proposition 3.1 for replicates drawn independently; the identity $\\mathrm{E}[(R+1)/(X_i+1)]=(1-(1-\\pi_i(d_k))^{R+1})/\\pi_i(d_k)$ used in Appendix A.3; and the printed example of 10,593 replicates.\n\\item Lemma 4.1 and Proposition 4.2, with equations (2) and (4)--(6), by derivation and by exact enumeration of small designs with interference and outcomes of mixed sign.\n\\item Propositions 5.1--5.6 and Appendix A.5: the bias of the variance estimators, the corrections based on Young's inequality, their direction and their cases of equality, and the conservativeness in expectation of equation (11), for signed outcomes and with joint exposure probabilities equal to zero.\n\\item Proposition 6.1 and Appendix A.6: the variance bound and consistency under Conditions 3 and 4, apart from the subscript corrected in \\fid{P013-E012}.\n\\item Appendix A.7: I checked the covariance count after adding self-edges to the dependency graph, and the central limit theorem for the point estimator, for a dependency graph of bounded degree. The coverage statement is the subject of \\fid{P013-E001}, \\fid{P013-E007} and \\fid{P013-E011}.\n\\item Section 7: equations (12) and (13) are unbiased for fixed predictions and agree with each other, and the bound on the bias of the Hajek estimator on p.~1929 follows when the ratio is defined.\n\\item Proposition 8.1, Corollary 8.1 and Appendix A.8.\n\\item Table 1: the root mean squared error equals $\\sqrt{\\text{bias}^2+\\text{SD}^2}$ within rounding in all 15 rows, the coverage entries are possible with 500 replications, and the statements of the text agree with the table.\n\\item The exposure mappings of Sections 9 and 10 partition the assignments as stated.\n\\item Section 10, the design: the script that assigned the schools and the public data show 56 schools in 14 blocks of four, with two host schools in each block. Every school has between 40 and 64 eligible students (2,943 in all). In the host schools, half of the eligible students are assigned to participate (728 of 1,456), within four student blocks per school. The script that simulates the assignments omits the school blocks (\\fid{P013-N016}).\n\\item Section 10, the network and the exposures: the released script builds an undirected graph within each school from the ten nominations of the first-wave survey, with a tie if either student named the other. Its five exposure indicators are the five conditions of the article. The recall period of the nomination question is the subject of \\fid{P013-N015}.\n\\item Section 10, the mean of 0.000 in the no-exposure condition: on the public data, none of the 1,487 eligible students of the control schools is coded as wearing the orange wristband.\n\\item Table 2, the point estimates: all twelve are reconstructed on the public data to three decimals, with $N=2{,}050$, using 10,000 probability draws with seed 123 and the pre-R-3.6 sampler, under four conditions. The eligible students are kept if their five simulated exposure probabilities exceed $0.01$, which is not the rule that the article states. The outcome is the orange-wristband indicator with absent answers counted as zero (\\fid{P013-N020}). The weights are inverse exposure probabilities (\\fid{P013-E013}). The reconstructed WLS fit has one slope on network degree for all exposures.\n\\item Table 2, the intervals: the four HT intervals, the four Hajek intervals and the WLS interval for $d_{011}$ equal the estimate $\\pm1.96$ standard errors within rounding, and the magnitudes quoted in the text agree with the table. The WLS rows for $d_{101}$ and $d_{111}$ are the subject of \\fid{P013-E008}.\n\\item Section 10, the naive comparison of p.~1937: in the reconstructed sample on the public data, the contrast of participants with eligible nonparticipants in program schools is $0.159$, against $0.288$ for participants and students with no exposure.\n\\item In the R package \\texttt{interference}, the point estimators, equations (7), (10) and (11), the correction for zero joint probabilities and the two estimators of the exposure probabilities agree with the formulas of the paper, apart from \\fid{P013-E018}, \\fid{P013-N012} and \\fid{P013-N013}.\n\\end{itemize}\nMy checks had the following limits. The Add Health outcome and network files that the simulation reads were not available. Table 1, Figure 3 and the exposure probabilities quoted in Section 9 were therefore not reproduced. I did not run the simulation again, and I did not check the distributions of Appendix B numerically. None of the eleven scripts of the field experiment computes Table 2, and the script that did so was not available; the authors' own analysis files were not available to me. The scripts could not be run as written, because the input files that they name are not in the release. Their steps were applied to the public data through an adapter. Every recalculated value for Section 10 comes from the public data, which ICPSR's notes say differ from the authors' files in some variables. I cannot establish that these are the values the authors' files would give. On the public data, the rule that the article states for the sample gives 2,196 students and not 2,050, and whether the published analysis used a cut-off is undetermined. The twelve standard errors of Table 2 were not reproduced. With the variance expressions of the article and with exact or simulated joint probabilities, no recalculated standard error rounds to its printed value, and the cause is not identified. The code that produced the published tables was not available, and the package is a later implementation whose link to those tables is not established. The functions of the package for two-stage designs were read for context, without checking them against their separate paper; its generated documentation and most of the codebook of the field experiment were read in part or not at all. The companion paper on the field experiment and most works cited by the article were not consulted; selected cited results on location invariance and dependency graphs were checked.\n\nI stated selected results of the paper in Lean~4 with Mathlib as 27 targets with 255 statements. They cover the results of Sections 3 to 8, the corrected statements, the counterexamples to the printed ones, and the two findings on printed numbers \\fid{P013-N004} and \\fid{P013-E008}. For the application of Section 10, they also cover the mathematical content of \\fid{P013-N016} and \\fid{P013-N020}. The statements for \\fid{P013-N016} give the exact probabilities that one school and that two schools host the program, under the block randomization that the article states and under the lottery that the released script simulates. They include an example of what the joint probabilities of that lottery do to the variance estimator of expression (7). The statements for \\fid{P013-N020} say what the Horvitz--Thompson estimates measure when a student with no recorded answer is scored as not wearing, and by what factor each contrast changes when such students are removed from the sums. They include a two-unit example in which the recorded data do not determine the effect on wearing, and the printed description of the outcome read with Condition 2 for the wearing outcome of every student analysed. Of these statements, 242 are proved and 13 are counterexamples that are proved. The proofs include the counterexamples to Proposition 6.2 and to the convergence claims of Appendix A.7, the corrected Proposition 6.2 and its two bounds on the variance of the variance estimator, and the corrected statements for Appendix A.7. They also refute Proposition 4.1, its rule for the number of replicates and Proposition 6.2 as printed, as in \\fid{P013-E003} and \\fid{P013-E001}. The counts 2,050, 342 and 151 come from the reconstruction on the public data, and two statements take them as given numbers or as hypotheses. The Lean statements do not verify these counts or any other empirical input, and they do not reproduce an entry of Table 2. The absence of further corrections does not establish that the rest of the paper is correct.\n\nThe rejection of repeated Bernoulli vectors occurs in that later package (\\fid{P013-N012}); I have not established that it was used for the published calculations. Appendix A.1's separate without-replacement wording needs the clarification in \\fid{P013-E010}.", "work": { "arxiv_id": "1305.6156", "audit_complete": true, "authors": [ "P. M. Aronow", "Cyrus Samii" ], "category": "Articles - Theory and Methodology", "citation": "P. M. Aronow and Cyrus Samii (2017). Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment. Annals of Applied Statistics 11(4):1912–1947.", "doi": "10.1214/16-aoas1005", "edition": { "pages_in_audited_file": 36, "sha256": "2e01576f8c944e7ec8fdcec991101c7a1fd1f03c2cab60799ddbef3951dbf233", "source_url": "https://doi.org/10.1214/16-aoas1005", "venue": "Annals of Applied Statistics", "year": "2017" }, "id": "P013", "later_materials": { "related_entries": [ "P013-E008", "P013-E013", "P013-N013", "P013-N015", "P013-N016", "P013-N020" ] }, "title": "Estimating Average Causal Effects Under General Interference, with Application to a Social Network Experiment", "type": "Journal article", "url": "https://pmaronow.github.io/llm-audit/works/P013/", "venue": "Annals of Applied Statistics", "year": "2017" } }