{ "claims": [ { "claim": "The interval estimator $[\\hat V_N^L,\\hat V_N^H]$ is consistent for the sharp bounds on the variance of the difference-in-means estimator: $NV_N^H$ and $NV_N^L$ converge to explicit limits involving the supremum and infimum of $\\mathrm{Cov}_h(y_1,y_0)$ over all bivariate distributions $h$ with the limiting marginals, and $(\\hat V_N^H-V_N^H,\\hat V_N^L-V_N^L)=o_P(1/N)$.", "claim_id": "P022-H1", "related_corrections": [ "P022-E001" ], "tied_to": "Proposition 1; Lemma 2 (abstract; introduction p. 850; Section 4 p. 855)" }, { "claim": "Given only the marginal distributions $G_N$ and $F_N$, $[\\sigma_N^L,\\sigma_N^H]$ is the sharp bound on the covariance of the potential outcomes, so $[V_N^L,V_N^H]$ is the smallest interval containing all values of $V_N$ compatible with the marginals.", "claim_id": "P022-H2", "related_corrections": [], "tied_to": "Lemma 1; equations (2), (7) (introduction p. 850; Section 3 p. 853)" }, { "claim": "When $n=N$, the upper bound estimator is never larger than the conventional approximations: $\\hat V^H\\le\\hat V^{b+}$ (Neyman's Cauchy-Schwarz bound) $\\le\\hat V^a$ (the conventional estimator), and $\\hat V^L\\ge\\hat V^{b-}$; the sharp bounds improve on the bounds obtained from second moments alone.", "claim_id": "P022-H3", "related_corrections": [], "tied_to": "Equations (4), (6), (9); the sentence after equation (9) (introduction p. 850; Section 3 p. 854)" }, { "claim": "The estimator gives a unified treatment of $n=N$, $n\\varepsilon)$ and $\\mathrm P(N|\\hat V_N^L-V_N^L|>\\varepsilon)$ are $O(1/N)$.", "claim_id": "P022-H6", "related_corrections": [ "P022-E001", "P022-E005", "P022-E006", "P022-N007" ], "tied_to": "Proposition 2; Remark 3; Lemma 2 (p. 856; Appendix A pp. 863-869)" }, { "claim": "The paper reports that in the Harrison-Michelson experiment ($N=n=1561$, $m=781$) the estimated effect is −0.682 USD and the sharp upper bound variance estimate is 0.186 USD$^2$ against 0.199 (conventional) and 0.196 (Neyman upper bound), a 7% reduction in nominal variance, equivalent to over 100 (respectively over 75) additional subjects.", "claim_id": "P022-H7", "related_corrections": [ "P022-N003" ], "tied_to": "Table 1 (introduction p. 851; Section 6 p. 857)" }, { "claim": "In simulations on the same data, $\\hat V_N^H$ is slightly anti-conservative under the sharp null (bias $-0.010$, coverage 93.7%), and under two heterogeneity scenarios it has smaller bias, narrower intervals and coverage closer to nominal than Neyman's estimators.", "claim_id": "P022-H8", "related_corrections": [ "P022-N005", "P022-N006" ], "tied_to": "Table 2 (introduction p. 851; Section 6.1 pp. 857-859)" }, { "claim": "For Beta-distributed marginals the sharp upper bound can be materially below Neyman's bounds when the marginals differ in shape (ratios down to 0.61 against $V^a$ and 0.79 against $V^{b+}$), and all upper bounds coincide when the marginals are identical.", "claim_id": "P022-H9", "related_corrections": [], "tied_to": "Table 3 (introduction p. 850; Section 6.1 p. 859; Appendix C pp. 869-870)" }, { "claim": "The R function sharp.var printed in Appendix B implements $\\hat V_N^H$ (upper=TRUE) and $\\hat V_N^L$ (upper=FALSE).", "claim_id": "P022-H10", "related_corrections": [ "P022-E004", "P022-E002", "P022-E003" ], "tied_to": "Appendix B; equations (8), (9) (introduction p. 851; Section 3 p. 854; p. 869)" } ], "date_published": "2026-10-25", "findings": [ { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "The opening of Lemma 2. A red box marks the words 'Suppose conditions 1-3 of Proposition 1 hold. Then'. The display below it states that the supremum over y of the distance between G and G N tends to 0, and likewise for F and F N.", "caption": "Lemma 2, p. 860 (PDF p. 11): the hypothesis and the first assertion. The marked line states the hypothesis, conditions 1–3 of Proposition 1.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E001.png" }, { "alt": "Four lines from the proof of Lemma 2. A red box marks the sentence 'By conditions 2 and 3, this is satisfied for all N sufficiently large', which follows the requirement that the distance between G and G N at the points s i minus 1, and between their left limits at s i, be below eta 1 over 2.", "caption": "Proof of Lemma 2, p. 861 (PDF p. 12). The marked sentence obtains the convergence of the left limits $G_N(s_i-)$ from conditions 2 and 3.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E001-b.png" }, { "alt": "Conditions 2, 3 and 4 of Proposition 1. A red box marks condition 3: 'G N of y tends to G of y at any discontinuity point of G, and F N of y tends to F of y at any discontinuity point of F'.", "caption": "Proposition 1, p. 855 (PDF p. 6). The marked condition 3 concerns the values $G_N(y)$ and $F_N(y)$ at the discontinuity points of $G$ and $F$, and not the left limits.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E001-c.png" }, { "alt": "The first three lines of Remark 1. A red box marks the sentence 'Condition 3 is used to establish the functional convergence of (G N, F N) to (G, F)'.", "caption": "Remark 1, p. 855 (PDF p. 6). The marked sentence says what condition 3 is used for.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E001-d.png" }, { "alt": "Line chart with population size N from 4 to 100 on a logarithmic horizontal axis and the largest distribution-function error on the vertical axis, with tick labels from 0 to 3/4. Treatment outcomes (blue circles) and control outcomes (orange squares) both take the values 1/2, 3/5 and 4/7 at N equal to 4, 5 and 7. At N equal to 6, the treatment error is 2/3 and the control error is 1/2. Both series equal 1/2 at N equal to 8, 12, 16, 32 and 100. A dashed line at 1/2 is labelled 'counterexample: at least 1/2 for every N at least 4', and a dotted line at 0 is labelled 'Lemma 2 as printed: tends to 0'.", "caption": "The counterexample to Lemma 2. The largest error of the population distribution functions, $\\sup_y|G(y)-G_N(y)|$ and $\\sup_y|F(y)-F_N(y)|$, is shown at nine population sizes from 4 to 100 (exact values). It is never below $1/2$, where Lemma 2 as printed asserts convergence to 0.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E001-cdf-error.png" } ], "formal_release": null, "formal_statements": [ { "id": "P022-T1", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T1" }, { "id": "P022-T2", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T2" }, { "id": "P022-T3", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T3" }, { "id": "P022-T15", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T15" }, { "id": "P022-T18", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T18" } ], "headline": "Lemma 2 does not hold under conditions 1–3 of Proposition 1: it needs in addition that the left limits $G_N(x-)$ and $F_N(x-)$ converge to $G(x-)$ and $F(x-)$ at the discontinuity points of $G$ and $F$, while Propositions 1 and 2 keep their statements through proofs that do not use the lemma.", "id": "P022-E001", "location": "Lemma 2, p. 860 (PDF p. 11), and its proof, p. 861 (PDF p. 12); condition 3 of Proposition 1 and Remark 1, p. 855 (PDF p. 6); the lemma is used on pp. 856, 862 and 867–869", "pdf_page": 11, "printed_page": "860", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Moderate", "subject": "paper", "target_ids": [ "P022-T1", "P022-T15", "P022-T18", "P022-T2", "P022-T3" ], "text": { "as_printed": "\\emph{Condition 3 of Proposition 1 (p.~855).} $G_N(y)\\to G(y)$ at any discontinuity point of $G$, and $F_N(y)\\to F(y)$ at any discontinuity point of $F$;\n\n\\emph{Remark 1 (p.~855).} Condition 3 is used to establish the functional convergence of $(G_N,F_N)$ to $(G,F)$.\n\n\\emph{Lemma 2 (p.~860).} Suppose \\hl{conditions 1--3 of Proposition 1} hold. Then\n\\[\n\\sup_y|G(y)-G_N(y)|\\to 0 \\quad\\text{and}\\quad \\sup_y|F(y)-F_N(y)|\\to 0 .\n\\]\nIn addition, given $\\eta_1,\\eta_0>0$, there exist two positive integers $K_1(\\eta_1)$ and $K_0(\\eta_0)$ such that\n\\[\n\\begin{aligned}\n\\limsup_N\\Bigl\\{N\\,\\mathrm{P}\\Bigl(\\sup_y|G(y)-\\hat G_N(y)|\\ge\\eta_1\\Bigr)\\Bigr\\} &\\le \\frac{(1-\\theta\\rho)K_1(\\eta_1)}{\\theta\\rho\\,\\eta_1^2},\\\\\n\\limsup_N\\Bigl\\{N\\,\\mathrm{P}\\Bigl(\\sup_y|F(y)-\\hat F_N(y)|\\ge\\eta_0\\Bigr)\\Bigr\\} &\\le \\frac{\\{1-\\theta(1-\\rho)\\}K_0(\\eta_0)}{\\theta(1-\\rho)\\,\\eta_0^2}.\n\\end{aligned}\n\\]\nThe integers are nonincreasing in $\\eta$, and depend also on the limiting distribution $H$ of $(y_1,y_0)$.\n\n\\emph{Proof of Lemma 2 (p.~861).} hence $\\sup_y|G(y)-G_N(y)|<\\eta_1$ if $|G(s_{i-1})-G_N(s_{i-1})|<\\eta_1/2$ and $|G(s_i-)-G_N(s_i-)|<\\eta_1/2$ for all $i$. \\hl{By conditions 2 and 3, this is satisfied for all $N$ sufficiently large.}", "corrected": "\\emph{Lemma 2.} Suppose conditions 1--3 of Proposition 1 hold, that is, $(m/N,n/N)\\to(\\theta\\rho,\\theta)$ for $\\theta\\in(0,1]$ and $\\rho\\in(0,1)$; $H_N$ converges weakly to a limit distribution $H$ with marginals $G(y)=H(y,\\infty)$ and $F(y)=H(\\infty,y)$; and $G_N(y)\\to G(y)$ at any discontinuity point of $G$, and $F_N(y)\\to F(y)$ at any discontinuity point of $F$. Suppose in addition that $\\hl{G_N(y-)\\to G(y-)}$ at any discontinuity point of $G$, and $\\hl{F_N(y-)\\to F(y-)}$ at any discontinuity point of $F$. Then\n\\[\n\\sup_y|G(y)-G_N(y)|\\to 0 \\quad\\text{and}\\quad \\sup_y|F(y)-F_N(y)|\\to 0 .\n\\]\nIn addition, given $\\eta_1,\\eta_0>0$, there exist two positive integers $K_1(\\eta_1)$ and $K_0(\\eta_0)$ such that\n\\[\n\\begin{aligned}\n\\limsup_N\\Bigl\\{N\\,\\mathrm{P}\\Bigl(\\sup_y|G(y)-\\hat G_N(y)|\\ge\\eta_1\\Bigr)\\Bigr\\} &\\le \\frac{(1-\\theta\\rho)K_1(\\eta_1)}{\\theta\\rho\\,\\eta_1^2},\\\\\n\\limsup_N\\Bigl\\{N\\,\\mathrm{P}\\Bigl(\\sup_y|F(y)-\\hat F_N(y)|\\ge\\eta_0\\Bigr)\\Bigr\\} &\\le \\frac{\\{1-\\theta(1-\\rho)\\}K_0(\\eta_0)}{\\theta(1-\\rho)\\,\\eta_0^2}.\n\\end{aligned}\n\\]\nThe integers are nonincreasing in $\\eta$, and $K_1(\\eta)=K_0(\\eta)=2\\lceil 4/\\min(\\eta,1)\\rceil$ is an admissible choice.\n\n\\emph{Remark 1.} Condition 3, \\hl{together with the convergence of the left limits} $G_N(y-)\\to G(y-)$ and $F_N(y-)\\to F(y-)$ at the same points, is used to establish the functional convergence of $(G_N,F_N)$ to $(G,F)$. When the units of $U_N$ are independent and identically distributed samples from a superpopulation, both conditions hold with probability one, and both are satisfied if $G$ and $F$ are continuous.\n\n\\emph{Proposition 1.} The statement is unchanged. The proof outline and stages (i) and (ii) of the proof rest on the uniform convergence of $(\\hat G_N,\\hat F_N)$ to $(G,F)$. They are replaced by the weak convergence of $\\hat G_N$ and $\\hat F_N$ to $G$ and $F$ in probability and, by condition 4, the convergence of their quantile functions in $L^2(0,1)$ in probability. No condition at the discontinuity points is needed.\n\n\\emph{Proposition 2.} The statement is unchanged. The steps of the proof that apply Lemma 2 are replaced by the following bound. Suppose $|y_{1i}|\\le C$ and $|y_{0i}|\\le C$ for all $i$, and let $c>0$ be such that $m\\ge cN$ and $n-m\\ge cN$ for all large $N$, as condition 1 allows. Then for $B\\in\\{H,L\\}$, every $\\varepsilon>0$, and every such $N$ with $N\\ge\\max\\{2/c,\\ 16C^2/(c^2\\varepsilon)\\}$,\n\\[\n\\mathrm{P}\\bigl\\{N|\\hat V_N^B-V_N^B|\\ge\\varepsilon\\bigr\\} \\le \\frac{16\\,(16/c+8)^2\\,C^4}{cN\\varepsilon^2}.\n\\]", "evidence": "The evidence is an exact counterexample to the lemma, together with derivations of the corrected statements.\n\n\\emph{The counterexample.} For $i=1,2,\\ldots$, let $(a_i,b_i)$ run through the cycle $(0,0),(0,1),(1,0),(1,1)$, and define\n\\[\ny_{1i} = a_i - \\frac{1}{i+2}, \\qquad y_{0i} = b_i - \\frac{1}{i+2}.\n\\]\nLet $U_N$ consist of the first $N$ units, for $N\\ge 4$, and set $n=N$ and $m=\\lfloor N/2\\rfloor$. The populations are nested and the outcomes are fixed. Condition 1 holds with $\\theta=1$ and $\\rho=1/2$. Condition 2 holds: $H_N$ converges weakly to the distribution $H$ with mass $1/4$ at each of the points $(0,0)$, $(0,1)$, $(1,0)$ and $(1,1)$. Both marginals of $H$ have mass $1/2$ at $0$ and at $1$, so that $G(x)=F(x)=1/2$ for $0\\le x<1$. Condition 3 holds as well. Since $y_{1i}\\le 0$ exactly when $a_i=0$, I have $G_N(0)\\to 1/2 = G(0)$, and since every outcome is below $1$, I have $G_N(1)=1=G(1)$; the same holds for $F_N$. All outcomes lie in $(-1,1)$, which implies condition 4, the boundedness assumed in Proposition 2 and the moment condition of Corollary 1. The distribution $H$ is not concentrated on a line.\n\nSet $t_N = 1 - 1/\\{2(N+2)\\}$. For $i\\le N$, I have $y_{1i}\\le 1-1/(N+2) < t_N$ and $y_{0i}\\le 1-1/(N+2) < t_N$, and $t_N<1$. Then $G_N(t_N)=F_N(t_N)=1$, and $\\hat G_N(t_N)=\\hat F_N(t_N)=1$ for every assignment, while $G(t_N)=F(t_N)=1/2$. This implies that, for every $N\\ge 4$ and every assignment,\n\\[\n\\sup_y|G(y)-G_N(y)|\\ge \\frac12, \\qquad \\sup_y|G(y)-\\hat G_N(y)|\\ge\\frac12,\n\\]\nand likewise for $F$, $F_N$ and $\\hat F_N$. The first assertion of the lemma fails. For $\\eta_1=\\eta_0=1/4$, the two probabilities in the second assertion equal $1$, and $N$ times each of them tends to infinity, while the right-hand sides equal $16K_1(1/4)$ and $16K_0(1/4)$. There exist no integers $K_1(1/4)$ and $K_0(1/4)$ for which the second assertion holds. Note that the left limits that the proof requires do not converge: $G_N(0-)\\to 1/2$ while $G(0-)=0$, and $G_N(1-)=1$ while $G(1-)=1/2$. The table lists exact values at three population sizes, obtained by enumerating every assignment.\n\\begin{audittable}{rrrrr}{The counterexample at three population sizes, with $\\eta_1=1/4$ and $m=N/2$. The population error is $\\sup_y|G(y)-G_N(y)|$. The smallest sample error is the minimum of $\\sup_y|G(y)-\\hat G_N(y)|$ over all assignments, and the probability is that of the event $\\sup_y|G(y)-\\hat G_N(y)|\\ge 1/4$ over all assignments. Every entry is exact, and the same values hold for $F$, $F_N$ and $\\hat F_N$.}\n$N$ & Assignments & Population error & Smallest sample error & Probability \\\\\n4 & 6 & $1/2$ & $1/2$ & 1 \\\\\n8 & 70 & $1/2$ & $1/2$ & 1 \\\\\n12 & 924 & $1/2$ & $1/2$ & 1 \\\\\n\\end{audittable}\n\n\\emph{The added condition suffices.} I write $P$ for $G$ or $F$, $P_N$ for the corresponding distribution function in the population, and $\\hat P_N$ for the one in the sample, with $k$ sampled units and $\\alpha=\\lim k/N>0$. By weak convergence, I have $P_N(x)\\to P(x)$ and $P_N(x-)\\to P(x-)$ at every continuity point $x$ of $P$. At the discontinuity points, both limits hold by condition 3 and the added condition. Fix $\\eta$ such that $0<\\eta\\le 1$, and set $R=\\lceil 4/\\eta\\rceil$. The points $\\pm\\infty$ and the distinct quantiles of $P$ at the levels $j/R$, for $1\\le j\\le R-1$, form a partition with at most $R$ intervals such that $P(t-)-P(s)\\le 1/R$ for adjacent points $s1$ the event is empty.\n\n\\emph{Proposition 1 without the lemma.} Let $c>0$ be such that $m\\ge cN$ and $n-m\\ge cN$ for all large $N$. For fixed $x$, the value $\\hat G_N(x)$ has mean $G_N(x)$ and variance at most $1/(4cN)$, so that $\\hat G_N(x)-G_N(x)\\to 0$ in probability. By condition 2, $\\hat G_N$ converges weakly to $G$ in probability, and the same holds for $\\hat F_N$ and $F$. For every sample and every $T>0$, I have\n\\[\n\\int x^2\\,\\mathbf{1}\\{|x|>T\\}\\,d\\hat G_N(x) \\le \\frac{N}{m}\\int x^2\\,\\mathbf{1}\\{|x|>T\\}\\,dG_N(x),\n\\]\nand by condition 4 the sampled distributions are uniformly square-integrable. Weak convergence and uniform square-integrability imply that the quantile functions converge in $L^2(0,1)$, in probability for the samples. The means, the second moments and the two integrals of products of quantile functions in (7)--(9) are continuous in the quantile functions with respect to that norm. I conclude that $NV_N^H$ and $NV_N^L$ have the limits stated in Proposition 1, and that $N(\\hat V_N^B-V_N^B)\\to 0$ in probability for $B\\in\\{H,L\\}$. Condition 3 is not used.\n\n\\emph{Proposition 2 without the lemma.} Suppose $|y_{1i}|\\le C$ and $|y_{0i}|\\le C$, and write $P_{1N}=G_N$, $P_{0N}=F_N$, $k_1=m$ and $k_0=n-m$. For $a\\in\\{0,1\\}$, define\n\\[\nD_a=\\int_{-C}^{C}|\\hat P_{aN}(x)-P_{aN}(x)|\\,dx .\n\\]\nBy the Cauchy--Schwarz inequality and the bound on the variance above, I have $\\mathrm{E}\\,D_a^2\\le C^2/(cN)$, so that $\\mathrm{E}(D_1+D_0)^2\\le 4C^2/(cN)$. The quantile functions $q_A$ and $q_P$ of two distributions on $[-C,C]$ with distribution functions $A$ and $P$ satisfy $\\int_0^1|q_A(u)-q_P(u)|\\,du=\\int_{-C}^{C}|A(x)-P(x)|\\,dx$. It follows that the error of the sample mean in arm $a$ is at most $D_a$, the error of the unbiased variance estimator is at most $8CD_a+C^2/(k_a-1)$, and the error of each estimated covariance bound in (8) is at most $2C(D_1+D_0)$. Substituting these bounds into (9), I obtain, for $N\\ge 2/c$ and $B\\in\\{H,L\\}$,\n\\[\nN|\\hat V_N^B-V_N^B| \\le \\Bigl(\\frac{16}{c}+8\\Bigr)C(D_1+D_0)+\\frac{8C^2}{c^2N}.\n\\]\nMarkov's inequality then implies the bound stated in the correction below.", "problem": "Lemma 2 makes two assertions about the distribution functions $G_N$ and $F_N$ of the potential outcomes in the population $U_N$ and their sample analogues $\\hat G_N$ and $\\hat F_N$. The first is that $G_N$ and $F_N$ converge uniformly to their limits $G$ and $F$. The second is that the probability of a uniform error of at least $\\eta$ between a sample distribution function and its limit is at most of order $1/N$. Neither follows from conditions 1--3. Weak convergence (condition 2) implies that $G_N(y)\\to G(y)$ at the points where $G$ is continuous, and condition 3 adds the points where $G$ jumps. Uniform convergence also requires the left limits to converge at the jumps, that is, $G_N(y-)\\to G(y-)$, and no condition supplies this. The proof on p.~861 needs $|G(s_i-)-G_N(s_i-)|<\\eta_1/2$ for large $N$ and obtains it ``by conditions 2 and 3''. At a jump of $G$ this step can fail. If the outcomes of a population approach a jump point of $G$ from below without reaching it, then conditions 1--4 can hold while the uniform error stays at $1/2$ or above for every $N$ and every assignment.\n\nThe lemma underlies Remark 1, the proof outline on p.~856, stages (i) and (ii) in the proof of Proposition 1 on p.~862, and the bounds for the last terms in the proof of Proposition 2 on pp.~867--869. Those steps are not justified by the stated conditions when $G$ or $F$ has a jump. The conclusions of Propositions 1 and 2 are not affected, since both hold under their printed hypotheses by arguments that do not pass through uniform convergence. To repair the lemma, one regularity condition must be added. That condition holds when $G$ and $F$ are continuous, and it holds with probability one when the units are independent and identically distributed draws from a superpopulation.", "what_survives": "Lemma 1 and the sharpness of the interval $[V_N^L,V_N^H]$, the bounds and estimators (7)--(9), and the ordering of $\\hat V_N^H$ and $\\hat V_N^L$ relative to Neyman's bounds do not depend on Lemma 2. Propositions 1 and 2 keep their hypotheses and conclusions. With them stand the consistency of $\\hat V_N^H$ used in Corollary 1 and the argument for asymptotically conservative Wald intervals. This error does not change the computations in the application, the simulations or Appendix C. Three things change: Lemma 2 gains a hypothesis, Remark 1 is reworded, and the proof outline and the steps in the proofs of Propositions 1 and 2 that rest on the lemma are replaced." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-E001", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Equations (11) and (12) with the sentence that introduces them. A red box marks the second equals sign in equation (11), between the term with sigma squared N and the fraction N minus m over 4 (N minus 1) m. In equation (12) the corresponding sign is 'less than or equal to'.", "caption": "Proof of Lemma 2, equations (11) and (12), p. 861 (PDF p. 12). The marked sign in (11) is printed as an equality; the corresponding sign in (12) is an inequality.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E005.png" } ], "formal_release": null, "formal_statements": [ { "id": "P022-T4", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T4" }, { "id": "P022-T5", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T5" } ], "headline": "The last relation in equation (11) is an upper bound, not an equality.", "id": "P022-E005", "location": "Proof of Lemma 2, equation (11), p. 861 (PDF p. 12)", "pdf_page": 12, "printed_page": "861", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P022-T4", "P022-T5" ], "text": { "as_printed": "Noting that $\\sigma_N^2(I\\{y_1\\le y\\})=G_N(y)\\{1-G_N(y)\\}\\le 1/4$ and similarly $\\sigma_N^2(I\\{y_0\\le y\\})\\le 1/4$, we have for all $y$,\n\\[\n\\operatorname{Var}_X\\hat G_N(y)=\\frac{N-m}{(N-1)m}\\,\\sigma_N^2\\bigl(I\\{y_1\\le y\\}\\bigr)\\ \\hl{=}\\ \\frac{N-m}{4(N-1)m}, \\qquad (11)\n\\]", "corrected": "Let $G_N$ be the distribution function of $y_1$ in the population $U_N$, and let $\\hat G_N$ be the distribution function of the observed outcomes of the $m$ units assigned to treatment. For all $y$,\n\\[\n\\operatorname{Var}_X\\hat G_N(y)=\\frac{N-m}{(N-1)m}\\,G_N(y)\\{1-G_N(y)\\}\\ \\hl{\\le}\\ \\frac{N-m}{4(N-1)m}. \\qquad (11)\n\\]", "evidence": "I used an exact counterexample. Let $N=4$ and $m=2$, and take $y$ below every potential outcome under treatment. Then $\\hat G_N(y)=0$ in each of the six assignments, so that $\\operatorname{Var}_X\\hat G_N(y)=0$, while the right-hand side of (11) is $(4-2)/(4\\cdot 3\\cdot 2)=1/12$.", "problem": "The variance $\\sigma_N^2(I\\{y_1\\le y\\})=G_N(y)\\{1-G_N(y)\\}$ is at most $1/4$, and it equals $1/4$ only when $G_N(y)=1/2$. The last relation in (11) is therefore an inequality. The sentence before the display and the companion display (12) state the bound correctly.", "what_survives": "The proof substitutes (11) into (10), and that step uses only the upper bound. The bound that follows is unchanged. The separate problem with Lemma 2 is \\fid{P022-E001}." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-E005", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "complete_derivation", "exhibits": [ { "alt": "Three lines of text and the first line of a display from the proof of Proposition 2. Red boxes mark '1 minus epsilon 1' twice: in the sentence 'To bound the probability that the last term exceeds 1 minus epsilon 1', and at the right end of the display, where the probability concerns an integral being less than 1 minus epsilon 1.", "caption": "Proof of Proposition 2, p. 867 (PDF p. 18). The marked tolerance $1-\\varepsilon_1$ is assigned to the last term, in the sentence and in the first line of the display that follows.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E006.png" }, { "alt": "The sentence 'The bounds (17)-(23) imply that' and the display below it. A red box marks the symbol V with subscript N, without a superscript, inside the absolute value 'V hat H N minus V N'.", "caption": "Proof of Proposition 2, display after equation (23), p. 868 (PDF p. 19). The marked symbol is printed $V_N$, without the superscript $H$.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E006-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P022-T6", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T6" }, { "id": "P022-T7", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T7" } ], "headline": "In the proof of the first bound of Proposition 2, the last term must receive a positive share of the tolerance $\\varepsilon_1$, such as $\\varepsilon_1/2$, in place of $1-\\varepsilon_1$, and the concluding display concerns $V_N^H$, not $V_N$.", "id": "P022-E006", "location": "Proof of Proposition 2, bound for the last term, p. 867 (PDF p. 18), through equation (23) and the display after it, p. 868 (PDF p. 19)", "pdf_page": 18, "printed_page": "867", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P022-T6", "P022-T7" ], "text": { "as_printed": "\\emph{p.~865.} Let $\\nu_1,\\ldots,\\nu_8>0$ be a tuple whose sum is $\\hl{\\varepsilon_1}$.\n\n\\emph{p.~867.} To bound the probability that the last term exceeds $\\hl{1-\\varepsilon_1}$, first note that $|u-u'|<\\eta$ and $|v-v'|<\\eta$ implies $|\\min(u,v)-\\min(u',v')|<\\eta$. This gives the third inequality below:\n\\[\n\\begin{aligned}\n&\\mathrm{P}\\Bigl[\\frac{2N}{N-1}\\Bigl|\\int_{[-C,C]^2}\\bigl\\{\\hat H_N^H(y_1,y_0)-H_N^H(y_1,y_0)\\bigr\\}\\,dy_1\\,dy_0\\Bigr|<\\hl{1-\\varepsilon_1}\\Bigr]\\\\\n&\\qquad\\ge \\mathrm{P}\\Bigl\\{\\sup_{y_1,y_0}\\bigl|\\hat H_N^H(y_1,y_0)-H_N^H(y_1,y_0)\\bigr|<\\frac{(N-1)(\\hl{1-\\varepsilon_1})}{8NC^2}\\Bigr\\}\n\\end{aligned}\n\\]\n\n\\emph{p.~868, equation (23) and the text after it.}\n\\[\n\\limsup_N N\\,\\mathrm{P}\\Bigl[\\frac{2N}{N-1}\\Bigl|\\int_{[-C,C]^2}\\bigl\\{\\hat H_N^H(y_1,y_0)-H_N^H(y_1,y_0)\\bigr\\}\\,dy_1\\,dy_0\\Bigr|\\ge\\hl{1-\\varepsilon_1}\\Bigr]\\le C^4\\kappa_1(\\varepsilon_1).\n\\]\nThe bounds (17)--(23) imply that\n\\[\n\\limsup_N N\\,\\mathrm{P}\\bigl(N|\\hat V_N^H-\\hl{V_N}|\\ge\\varepsilon\\bigr)\\le C^4\\Bigl(\\sum_{i=1}^8\\frac{c_i}{\\nu_i^2}+\\kappa_1(\\varepsilon_1)\\Bigr).\n\\]\nBy minimizing the right-hand side over $\\nu_1,\\ldots,\\nu_8>0$ subject to the constraint $\\nu_1+\\cdots+\\nu_8=\\hl{\\varepsilon_1}$, the sum in the parenthesis can be absorbed into $\\kappa_1(\\varepsilon_1)$, yielding the desired convergence rate for $N\\hat V_N^H$.", "corrected": "\\emph{p.~865, for the bound on $\\hat V_N^H$.} Let $\\nu_1,\\ldots,\\nu_8>0$ be a tuple whose sum is $\\hl{\\varepsilon_1/2}$.\n\n\\emph{pp.~867--868.} To bound the probability that the last term exceeds $\\hl{\\varepsilon_1/2}$, replace $1-\\varepsilon_1$ by $\\varepsilon_1/2$ in every line of the two displays that follow, with $0<\\xi<\\varepsilon_1/(16C^2)$ \\hl{for $C>0$ (the case $C=0$ is trivial)}. Equation (23) becomes\n\\[\n\\limsup_N N\\,\\mathrm{P}\\Bigl[\\frac{2N}{N-1}\\Bigl|\\int_{[-C,C]^2}\\bigl\\{\\hat H_N^H(y_1,y_0)-H_N^H(y_1,y_0)\\bigr\\}\\,dy_1\\,dy_0\\Bigr|\\ge\\hl{\\varepsilon_1/2}\\Bigr]\\le C^4\\kappa_1(\\varepsilon_1).\n\\]\nThe bounds (17)--(23) imply that\n\\[\n\\limsup_N N\\,\\mathrm{P}\\bigl(N|\\hat V_N^H-\\hl{V_N^H}|\\ge\\varepsilon\\bigr)\\le C^4\\Bigl(\\sum_{i=1}^8\\frac{c_i}{\\nu_i^2}+\\kappa_1(\\varepsilon_1)\\Bigr).\n\\]\nBy minimizing the right-hand side over $\\nu_1,\\ldots,\\nu_8>0$ subject to the constraint $\\nu_1+\\cdots+\\nu_8=\\hl{\\varepsilon_1/2}$, the sum in the parenthesis can be absorbed into $\\kappa_1(\\varepsilon_1)$.\n\nIn the argument for $\\hat V_N^L$ the tuple keeps the sum $\\varepsilon_1$, as printed, and the last term keeps the tolerance $\\varepsilon_2+\\varepsilon_3$.", "evidence": "The evidence is a derivation with two exact examples. For the tolerance, take $\\varepsilon=3$ and $\\varepsilon_1=\\varepsilon_2=\\varepsilon_3=1$. The event in (23) is then that a nonnegative quantity is at least $0$. It has probability $1$, and the left-hand side of (23) is infinite. For the target, let $N=n=4r$ and $m=N/2$, with $r$ units at each of the points $(0,0)$, $(0,1)$, $(1,0)$ and $(1,1)$. Both marginal variances are $1/4$, the covariance is $0$ and its sharp upper bound is $1/4$, so that $NV_N=N/\\{2(N-1)\\}$ and $NV_N^H=N/(N-1)$. Since $N\\hat V_N^H\\to 1$ in probability, $N|\\hat V_N^H-V_N|$ tends to $1/2$ and not to $0$.\n\nThe repair rests on the inequality that the proof already uses. Since the square $[-C,C]^2$ has area $4C^2$, I have\n\\[\n\\frac{2N}{N-1}\\Bigl|\\int_{[-C,C]^2}\\bigl\\{\\hat H_N^H(y_1,y_0)-H_N^H(y_1,y_0)\\bigr\\}\\,dy_1\\,dy_0\\Bigr|\\le\\frac{8NC^2}{N-1}\\sup_{y_1,y_0}\\bigl|\\hat H_N^H(y_1,y_0)-H_N^H(y_1,y_0)\\bigr| .\n\\]\nA tolerance $x>0$ for the last term therefore corresponds to the threshold $(N-1)x/(8NC^2)$ for the supremum. With $x=\\varepsilon_1/2$ and $\\nu_1+\\cdots+\\nu_8=\\varepsilon_1/2$, the union bound controls the event $\\{N|\\hat V_N^H-V_N^H|\\ge\\varepsilon_1\\}$, which contains the event at $\\varepsilon$ because $\\varepsilon_1<\\varepsilon$.", "problem": "Proposition 2 bounds $\\mathrm{P}\\{N|\\hat V_N^H-V_N^H|\\ge\\varepsilon\\}$ for an arbitrary $\\varepsilon>0$, with $\\varepsilon_1+\\varepsilon_2+\\varepsilon_3=\\varepsilon$. The proof splits $N(\\hat V_N^H-V_N^H)$ into seven terms. The first six receive the tolerances $\\nu_1,\\ldots,\\nu_8$, whose sum is $\\varepsilon_1$, and the last receives $1-\\varepsilon_1$. The tolerances add to $1$ and not to $\\varepsilon$. As printed, the argument bounds the event $\\{N|\\hat V_N^H-V_N^H|\\ge 1\\}$. When $\\varepsilon_1\\ge 1$ the tolerance $1-\\varepsilon_1$ is not positive, and (23) cannot hold with a finite $\\kappa_1(\\varepsilon_1)$. The display after (23) has a second slip: it names the true variance $V_N$ where the sharp upper bound $V_N^H$ is meant. Read literally, it says that $N\\hat V_N^H$ concentrates around $NV_N$, which is false when the bound is not attained.", "what_survives": "The statement of Proposition 2 and its $O(1/N)$ rate are unaffected. The decomposition into seven terms and the bounds (17)--(22) stand, and the argument for $\\hat V_N^L$ keeps its own tolerance $\\varepsilon_2+\\varepsilon_3$. The use of Lemma 2 in the same passage is a separate matter, treated in \\fid{P022-E001}, where the rate is obtained from a bound that does not use the lemma." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-E006", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "recalculation_from_source", "exhibits": [ { "alt": "The first eight lines of the R function sharp.var. A red box marks the two lines 'FPvar <- function(x,N) (N-1)/(N*(length(x)-1))' and '* sum((x - mean(x))^2)'.", "caption": "Appendix B, R code, p. 869 (PDF p. 20). The two marked lines define FPvar; the second begins with the multiplication sign.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E004.png" } ], "formal_release": null, "formal_statements": [], "headline": "The multiplication sign that opens the second line of the definition of FPvar belongs at the end of the first line; as printed, the code does not parse.", "id": "P022-E004", "location": "Appendix B, R code of sharp.var, fourth and fifth printed lines, p. 869 (PDF p. 20)", "pdf_page": 20, "printed_page": "869", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "\\emph{Appendix B, p.~869, fourth and fifth lines of the function.}\n\n$\\texttt{FPvar <- function(x,N) (N-1)/(N*(length(x)-1))}$\n\n$\\texttt{~~}\\hl{\\texttt{*}}\\texttt{~sum((x - mean(x))\\textasciicircum 2)}$", "corrected": "\\emph{Fourth and fifth lines of the function.}\n\n$\\texttt{FPvar <- function(x,N) (N-1)/(N*(length(x)-1))~}\\hl{\\texttt{*}}$\n\n$\\texttt{~~sum((x - mean(x))\\textasciicircum 2)}$", "evidence": "I checked the code by running it. The block copied with its printed line breaks fails to parse at the place stated. With the operator moved to the end of the preceding line and no other change, the function parses and runs.", "problem": "R ends a statement at a line break when the expression read so far is complete. The first of the two lines is a complete function definition. The second then begins a new statement with a binary operator, and the parser stops with the message \\texttt{unexpected '*'} at line 5, column 5 of the function.", "what_survives": "The formula intended for \\texttt{FPvar} and the estimator are unchanged by the syntax repair. The reported numbers were not recomputed. Two further defects of the same function are \\fid{P022-E002} and \\fid{P022-E003}." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-E004", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "The final portion of the R function sharp.var. A red box marks the three lines 'p_i <- unique(sort(c(seq(0,n-m,1)/(n-m),seq(0,m,1)/m))) -', '.Machine$double.eps^.5' and 'p_i[1] <- .Machine$double.eps^.5'. A second red box marks '(p_i-p_i_minus)' inside the return expression.", "caption": "Appendix B, R code, p. 869 (PDF p. 20). The upper mark is the three lines that shift the partition points by $\\sqrt{\\texttt{.Machine\\$double.eps}}=2^{-26}$; the lower mark is the weights formed from the shifted points.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E002.png" }, { "alt": "Equation (8): the estimated upper and lower covariance bounds as sums over i from 1 to P. Red boxes mark the weight '(p i minus p i minus 1)' in each of the two sums.", "caption": "Equation (8), p. 854 (PDF p. 5). The marked weights are the widths $p_i-p_{i-1}$ of the cells of the partition, which sum to 1.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E002-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P022-T8", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T8" }, { "id": "P022-T9", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T9" } ], "headline": "The printed function shifts the partition points by $2^{-26}$ and uses the shifted points as integration weights, which sum to $1-2^{-25}$, and its output is not the estimate of equations (8)–(9); the widths $p_i-p_{i-1}$ must be kept, with the quantiles selected at the midpoints of the cells.", "id": "P022-E002", "location": "Appendix B, R code of sharp.var, p. 869 (PDF p. 20): the lines that set p_i and the weights in the return expression; equation (8), p. 854 (PDF p. 5)", "pdf_page": 20, "printed_page": "869", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P022-T8", "P022-T9" ], "text": { "as_printed": "\\emph{Appendix B, p.~869, ninth to seventeenth lines of the function.}\n\n$\\texttt{p\\_i <- unique(sort(c(seq(0,n-m,1)/(n-m),seq(0,m,1)/m)))~}\\hl{\\texttt{-}}$\n\n$\\hl{\\texttt{.Machine\\$double.eps\\textasciicircum.5}}$\n\n$\\hl{\\texttt{p\\_i[1] <- .Machine\\$double.eps\\textasciicircum.5}}$\n\n$\\texttt{yti <- yt[ceiling(p\\_i*m)]}$\n\n$\\texttt{yci <- yc[ceiling(p\\_i*(n-m))]}$\n\n$\\texttt{p\\_i\\_minus <- c(NA,p\\_i[1: (length(p\\_i)-1)])}$\n\n$\\texttt{return(((N-m)/m * FPvar(yt,N) + (N-(n-m))/(n-m) * FPvar(yc,N)}$\n\n$\\texttt{~~+ 2*sum((}\\hl{\\texttt{(p\\_i-p\\_i\\_minus)}}\\texttt{*yti*yci)[2:length(p\\_i)])}$\n\n$\\texttt{~~- 2*mean(yt)*mean(yc))/(N-1))}$\n\n\\emph{Equation (8), p.~854.}\n\\[\n\\begin{aligned}\n\\hat\\sigma_N^H(y_1,y_0)&=\\sum_{i=1}^P(p_i-p_{i-1})\\,y_{1[i]}y_{0[i]}-\\hat\\mu_N(y_1)\\hat\\mu_N(y_0),\\\\\n\\hat\\sigma_N^L(y_1,y_0)&=\\sum_{i=1}^P(p_i-p_{i-1})\\,y_{1[i]}y_{0[P+1-i]}-\\hat\\mu_N(y_1)\\hat\\mu_N(y_0),\n\\end{aligned}\n\\]", "corrected": "Keep the partition $0=p_0= 2, k >= 2, N >= m+k, all(is.finite(c(yt,yc))))}$\n\n$\\texttt{~~FPvar <- function(x,N) (N-1)/(N*(length(x)-1)) *}$\n\n$\\texttt{~~~~sum((x - mean(x))\\textasciicircum 2)}$\n\n$\\texttt{~~vt <- FPvar(yt,N)}$\n\n$\\texttt{~~vc <- FPvar(yc,N)}$\n\n$\\texttt{~~yt <- sort(yt) - mean(yt)}$\n\n$\\texttt{~~yc <- sort(yc,decreasing=!upper) - mean(yc)}$\n\n$\\texttt{~~p <- unique(sort(c(seq(0,k,1)/k,seq(0,m,1)/m)))}$\n\n$\\texttt{~~lo <- p[-length(p)]}$\n\n$\\texttt{~~hi <- p[-1]}$\n\n$\\texttt{~~mid <- (lo+hi)/2}$\n\n$\\texttt{~~cov <- sum((hi-lo)*yt[ceiling(mid*m)]*yc[ceiling(mid*k)])}$\n\n$\\texttt{~~((N-m)/m * vt + (N-k)/k * vc + 2*cov)/(N-1)}$\n\n$\\texttt{\\}}$", "evidence": "I checked an exact counterexample in rational arithmetic and reproduced it in R. Let $k=n-m$, and suppose every cell of the partition is wider than $2e$, so that the shifted points select the same order statistics as (8). Let $a_1$ be the smallest outcome under treatment, and let $b_*$ be the smallest outcome under control for the upper bound and the largest for the lower bound. Since only the first weight changes, the output of the printed function differs from the estimate (9) by exactly\n\\[\n-\\frac{4e\\,a_1b_*}{N-1}, \\qquad e=2^{-26}.\n\\]\nWith all outcomes equal to $1$, $N=n=4$ and $m=2$, the randomization variance and both estimates (9) are $0$, and the printed function returns $-4e/3=-1/50331648$ for both bounds; R returns $-1.9868\\times10^{-8}$. The table lists this case and three others.\n\\begin{audittable}{lrlrr}{The estimate of equation (9) and the output of the printed function, in exact arithmetic with $e=2^{-26}$. The line break in the definition of \\texttt{FPvar} is joined.}\nOutcomes in the two arms & $N$ & Bound & Equation (9) & Printed function \\\\\n$(1,1)$ in both & 4 & upper & $0$ & $-1/50331648\\approx-1.99\\times10^{-8}$ \\\\\n$(1,1)$ in both & 4 & lower & $0$ & $-1/50331648\\approx-1.99\\times10^{-8}$ \\\\\n$(0,1)$ in both & 4 & upper & $5/12$ & $5/12$ \\\\\n$(0,1)$ in both & 4 & lower & $1/12$ & $1/12$ \\\\\n$(10^6,10^6+1)$ in both & 4 & upper & $5/12$ & $-244135505/12288\\approx-19867.80$ \\\\\n$(10^6,10^6+1)$ in both & 4 & lower & $1/12$ & $-15624950089/786432\\approx-19868.15$ \\\\\ntreatment $(-2,1,4)$, control $(1,3)$ & 8 & upper & $179/56$ & $179/56+1/58720256$ \\\\\ntreatment $(-2,1,4)$, control $(1,3)$ & 8 & lower & $115/56$ & $115/56+3/58720256$ \\\\\n\\end{audittable}\nThe third and fourth rows agree because $a_1=0$. The fifth and sixth rows are the third and fourth with $10^6$ added to every outcome, which leaves (9) unchanged. For the application, the cells of the partition have width at least $1/(781\\cdot780)>2e$, and by the displayed formula the discrepancy is at most $4e\\cdot150^2/1560=375/436207616\\approx8.6\\times10^{-7}$ USD$^2$. Since the data are not available, no entry of Table 1 or Table 2 was recomputed, and none is shown to change.", "problem": "Equation (8) weights each product of order statistics by the width $p_i-p_{i-1}$ of a cell of the partition $P_{m,n}$ of $[0,1]$, and the widths sum to $1$. The function subtracts $e=\\sqrt{\\texttt{.Machine\\$double.eps}}=2^{-26}$ from every partition point, resets the first point to $e$, and uses the shifted points both to select the order statistics and as weights. When every cell is wider than $2e$, the shifted points select the order statistics of (8). The first weight, however, becomes $p_1-p_0-2e$, and the weights sum to $1-2e=1-2^{-25}$. A fraction $2e$ of the first product is lost from the sum. The output is then not invariant to adding a constant to every outcome, as the estimate (9) is, and it can be negative. The discrepancy is about $6\\times10^{-8}$ times the product of two extreme outcomes, divided by $N-1$. For the application, with $N=1561$ and outcomes between \\$0 and \\$150, it is at most $8.6\\times10^{-7}$ USD$^2$, against a reported estimate of 0.186. It vanishes for the upper bound when the smallest outcome in either arm is zero.", "what_survives": "The definitions (7)--(9), the sharpness of the bounds and the asymptotic results concern the estimators as defined in the text and are unaffected. Table 3 was computed by numerical integration of Beta quantile functions and does not use this function. No number in Table 1 or Table 2 is shown to be wrong." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-E002", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "The paragraph that introduces the R function sharp.var and the first eight lines of its code. A red box marks the words 'N (the total number of units in the population)'. A second red box marks the expression '(N-1)/(N*(length(x)-1))' in the line that defines FPvar.", "caption": "Appendix B, p. 869 (PDF p. 20). The upper mark is the description of the argument N; the lower mark is the expression in FPvar that evaluates Inf/Inf when N=Inf.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-E003.png" } ], "formal_release": null, "formal_statements": [ { "id": "P022-T10", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T10" } ], "headline": "Called with N=Inf, the printed function returns NaN; the infinite-population estimate $s_T^2/m+s_C^2/(n-m)$ requires a branch of its own.", "id": "P022-E003", "location": "Appendix B, description of the argument N and R code of sharp.var, p. 869 (PDF p. 20); the statements about $N=\\infty$ on pp. 850 and 852 are correct", "pdf_page": 20, "printed_page": "869", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P022-T10" ], "text": { "as_printed": "\\emph{Appendix B, p.~869.} The other inputs are \\texttt{yt} (the observed outcomes under treatment), \\texttt{yc} (the observed outcomes under control) and \\hl{\\texttt{N} (the total number of units in the population)}.\n\n$\\texttt{FPvar <- function(x,N)~}\\hl{\\texttt{(N-1)/(N*(length(x)-1))}}$\n\n\\emph{Introduction, p.~850.} Our estimator is also applicable to all possible cases of $N$ and $n$ ($n=N<\\infty$, $n 0) return(stats::var(yt)/m + stats::var(yc)/(n-m))}}$\n\nWith this line the function returns $s_T^2/m+s_C^2/(n-m)$ for \\texttt{N=Inf} in both modes, and every call with finite \\texttt{N} is unchanged. The arm sizes $m\\ge 2$ and $n-m\\ge 2$ assumed on p.~851 make both sample variances well defined. The description may then read: \\texttt{N} (the total number of units in the population\\hl{, or \\texttt{Inf} for an infinite population}).", "evidence": "I checked an exact example and derived the limit. For treatment and control outcomes $(0,1)$ and $(0,1)$, the printed function, with the line break in \\texttt{FPvar} joined, returns \\texttt{NaN} for \\texttt{N=Inf} with \\texttt{upper=TRUE} and with \\texttt{upper=FALSE}. For finite $N$ it returns $(N-2)/(2N)+1/\\{2(N-1)\\}$ for the upper bound and $(N-2)/(2N)-1/\\{2(N-1)\\}$ for the lower bound, that is, $5/12$ and $1/12$ at $N=4$ and $41/90$ and $31/90$ at $N=10$, and both expressions tend to $1/2$. In general, let $s_T^2$ and $s_C^2$ be the sample variances with denominators $m-1$ and $k-1$, where $k=n-m$. Since $\\texttt{FPvar(yt,N)}=(N-1)s_T^2/N$, the output for finite $N$ is\n\\[\n\\Bigl(\\frac1m-\\frac1N\\Bigr)s_T^2+\\Bigl(\\frac1k-\\frac1N\\Bigr)s_C^2+\\frac{2\\hat c}{N-1},\n\\]\nwhere $\\hat c$ is the covariance term, which does not depend on $N$. Letting $N\\to\\infty$, I obtain $s_T^2/m+s_C^2/k$ for both bounds.", "problem": "The paper says that the estimator covers an infinite population, and that as $N\\to\\infty$ with $m$ and $n$ fixed it becomes the usual variance estimate for a difference of independent means. Both statements are correct. Appendix B presents \\texttt{sharp.var} as the implementation, with \\texttt{N} the size of the population, and the value of an infinite \\texttt{N} in R is \\texttt{Inf}. With \\texttt{N=Inf}, the expression in \\texttt{FPvar} evaluates \\texttt{Inf/Inf}, which is \\texttt{NaN}, and the return expression propagates \\texttt{NaN}. The function returns \\texttt{NaN} for both bounds. The paper documents no call with \\texttt{N=Inf}. This is a gap in the code and not a false statement.", "what_survives": "The limit stated on pp.~850 and 852, the formulas for finite $N$ and the asymptotic results are unaffected. The application has $N=n=1561$, and the simulations use the same finite population. No reported number involves a call with \\texttt{N=Inf}." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-E003", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "complete_derivation", "exhibits": [ { "alt": "Five lines of text from the last paragraph of Section 7. A red box marks the clause 'total variance is greater than the conditional variance (2)'.", "caption": "Section 7, last paragraph, p. 859 (PDF p. 10). The marked clause compares the total variance with the conditional variance (2).", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-A001.png" } ], "formal_release": null, "formal_statements": [], "headline": "With stochastic potential outcomes, the total variance is at least the expected conditional variance (2), and it is strictly greater only when the population average effect varies.", "id": "P022-A001", "location": "Section 7, last paragraph, p. 859 (PDF p. 10), sentence beginning “When the potential outcomes are stochastic”", "pdf_page": 10, "printed_page": "859", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [], "text": { "as_printed": "When the potential outcomes are stochastic, \\hl{the total variance is greater than the conditional variance (2)} because of the additional stochasticity.", "corrected": "When the potential outcomes are stochastic \\hl{and independent of the assignment, and the relevant variances exist, the total variance equals the expected conditional variance (2) plus the variance of the population average effect; it is at least the expected conditional variance, and it is strictly greater when the population average effect varies}.", "intended_meaning": "The sentence appeals to the law of total variance. Suppose the potential outcomes $Y$ are random and independent of the assignment, and write $V_N(Y)$ and $\\tau_N(Y)$ for the conditional variance (2) and the population average effect given $Y$. Since $\\mathrm{E}_X(\\hat\\tau_N\\mid Y)=\\tau_N(Y)$, I have\n\\[\n\\operatorname{Var}(\\hat\\tau_N) = \\mathrm{E}_Y\\{V_N(Y)\\} + \\operatorname{Var}_Y\\{\\tau_N(Y)\\}\n\\]\nwhenever these variances exist. The total variance is at least the expected conditional variance, and the inequality is strict exactly when $\\operatorname{Var}_Y\\{\\tau_N(Y)\\}>0$. This is the comparison the sentence intends, and it is correct. Read literally, the sentence omits the expectation and the case of equality. For example, let $N=n=4$, $m=2$ and $y_1=y_0=(Z,0,0,0)$, where $Z$ takes the values $0$ and $1$ with probability $1/2$ each. Then $\\tau_N=0$ and $V_N=Z^2/4$, the total variance equals $\\mathrm{E}(V_N)=1/8$, and it is below the conditional variance $1/4$ when $Z=1$. Nothing in the paper depends on the sentence." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-A001", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "historical_lean_attribution", "exhibits": [ { "alt": "Nine lines of text from Section 6. A red box marks the figure 7 percent in the sentence which says that the upper bound estimator yields a 7 percent reduction in the nominal variance. The sentences that follow say that over 100 and over 75 additional subjects would be needed.", "caption": "Section 6, second paragraph, p. 857 (PDF p. 8). The marked figure is the stated reduction in the nominal variance.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-N003.png" }, { "alt": "Table 1 with five rows. The variance estimates are 0.199, 0.196, 0.003, 0.186 and 0.098. Red boxes mark the confidence intervals (minus 1.555, 0.192) in the row Conventional and (minus 1.528, 0.165) in the row Sharp upper bound.", "caption": "Table 1, p. 857 (PDF p. 8). The marked cells are the 95% confidence intervals for the conventional estimator and for the sharp upper bound. They determine the ratio of the two variances more closely than the rounded variances do.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-N003-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P022-T11", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T11" } ], "headline": "Table 1 puts the reduction in the nominal variance from $\\hat V_N^a$ to $\\hat V_N^H$ at about 6%, not 7%.", "historical_evidence_type": "lean_proof", "id": "P022-N003", "location": "Section 6, second paragraph, p. 857 (PDF p. 8), sentence beginning “Compared to the conventional variance estimator”; Table 1, p. 857", "pdf_page": 8, "printed_page": "857", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P022-T11" ], "text": { "as_printed": "Compared to the conventional variance estimator $\\hat V_N^a$, we find that our upper bound estimator yields a \\hl{7\\%} reduction in the nominal variance.", "corrected": "Compared to the conventional variance estimator $\\hat V_N^a$, we find that our upper bound estimator yields a \\hl{6\\%} reduction in the nominal variance.", "evidence": "The evidence is a recalculation from Table 1 in exact rational arithmetic. Let $a$ and $h$ be the half-widths of the intervals for $\\hat V_N^a$ and $\\hat V_N^H$. From the printed endpoints, I have $a = 0.8735$ and $h = 0.8465$, each to within $0.0005$, so that $a\\le 0.8740$ and $h\\ge 0.8460$. Since $\\hat V_N^H/\\hat V_N^a = (h/a)^2$, I obtain\n\\[\n\\frac{\\hat V_N^H}{\\hat V_N^a} \\ge \\Bigl(\\frac{0.8460}{0.8740}\\Bigr)^2 = \\frac{178929}{190969},\n\\qquad\nD \\le \\frac{1204000}{190969} \\approx 6.305 < 6.5 .\n\\]\nUsing the variance column, I have $\\hat V_N^H/\\hat V_N^a \\le 0.1865/0.1985 = 373/397$, which gives $D\\ge 2400/397\\approx 6.045$. The inequality $D<6.5$, under the hypotheses $a>0$, $a\\le 0.874$ and $h\\ge 0.846$, also has a Lean proof. Since the data of the application are not available, the unrounded value of $D$ cannot be recovered.", "problem": "The reduction is $D = 100\\,(1-\\hat V_N^H/\\hat V_N^a)$ percent. From the variances printed in Table 1, I find $1-0.186/0.199 \\approx 6.53\\%$, which rounds to 7\\%. However, each of them is rounded to three decimals, and their rounding can change the whole-percent summary. The 95\\% confidence intervals in the same table determine the ratio more closely. Both are Wald intervals with the same critical value. The ratio of the two variances is then the square of the ratio of the two half-widths. Allowing for the rounding of every endpoint, the intervals place $D$ between 5.867 and 6.305. With the variance column as well, $D$ lies between 6.045 and 6.305, and every value in that range rounds to 6\\%.", "what_survives": "Table 1 is unchanged, and the two statements that follow the percentage hold. By the same bounds and nominal inverse-sample-size scaling, matching $\\hat V_N^H$ would take between 100.4 and 105.1 additional subjects with $\\hat V_N^a$, and between 75.3 and 78.5 with $\\hat V_N^{b+}$, in agreement with ``over 100'' and ``over 75''. No formula, proof or other number in the paper uses the percentage." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-N003", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "historical_lean_attribution", "exhibits": [ { "alt": "Seven lines of text from Section 6.1. A red box marks the parenthesis 'minus 0.007 USD squared' after the words 'nonpositive bias'. Later in the paragraph the sharp estimator is said to have bias minus 0.010 USD squared.", "caption": "Section 6.1, paragraph on the Sharp Null, p. 858 (PDF p. 9). The marked parenthesis gives the bias of Neyman’s upper bound estimator.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-N005.png" }, { "alt": "Table 2 with three blocks of three rows. In the Sharp Null block, red boxes mark 'True Var.: 0.199' and, in the row Neyman upper bound, the mean variance estimate 0.193 and the mean 95 percent CI width 1.724. The coverage in that row is 94.1 percent.", "caption": "Table 2, p. 858 (PDF p. 9). The marked entries are the true variance under the Sharp Null and, in the row for Neyman’s upper bound, the mean variance estimate and the mean width of the 95% confidence intervals.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-N005-b.png" } ], "formal_release": null, "formal_statements": [ { "id": "P022-T12", "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T12" } ], "headline": "The bias of $-0.007$ USD$^2$ quoted for $\\hat V_N^{b+}$ under the Sharp Null cannot be reconciled with Table 2; the statement that the bias is nonpositive stands without the figure.", "historical_evidence_type": "lean_proof", "id": "P022-N005", "location": "Section 6.1, paragraph on the Sharp Null, p. 858 (PDF p. 9); Table 2, Sharp Null block, row “Neyman upper bound”, p. 858", "pdf_page": 9, "printed_page": "858", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [ "P022-T12" ], "text": { "as_printed": "Due to the nonlinearity of the square root function, the Cauchy--Schwarz inequality implies that $\\hat V_N^{b+}$ has nonpositive bias ($\\hl{-0.007}$ USD$^2$).\n\n\\emph{Table 2, Sharp Null (True Var.: 0.199), row ``Neyman upper bound ($\\hat V_N^{b+}$)''.} Mean var.\\ estimate 0.193; Mean 95\\% CI width 1.724; Coverage for $\\tau_N$ 94.1\\%.", "corrected": "Due to the nonlinearity of the square root function, the Cauchy--Schwarz inequality implies that $\\hat V_N^{b+}$ has nonpositive bias \\del{($-0.007$ USD$^2$)}.\n\nThe difference of the two means printed in Table 2 is $0.193-0.199=-0.006$ USD$^2$. A figure to three decimals should be taken from the simulation output.", "evidence": "The evidence is an exact calculation from the printed numbers. Let $v_j$ be the value of $\\hat V_N^{b+}$ in assignment $j$, let $w_j = c\\sqrt{v_j}$ be the width of its interval, where $c = 2z_{0.975}$, and let $\\bar v$ and $\\bar w$ be the means of $v_j$ and $w_j$ over the simulated assignments. By the Cauchy--Schwarz inequality, I have $\\bar w^2\\le c^2\\bar v$. Let $V$ be the true variance. Since each printed number is rounded to three decimals, I have $V\\le 0.1995$, $\\bar w\\ge 1.7235$ and $\\bar v - V\\le -0.0065$, so that $\\bar v\\le 0.193$. Setting $c = 3.92$, I obtain\n\\[\n\\bar v \\ge \\frac{1.7235^2}{3.92^2} = \\frac{11881809}{61465600} \\approx 0.193308 > 0.193,\n\\qquad\n\\bar v - V \\ge -0.006192 .\n\\]\nThe contradiction does not depend on the value $1.96$. The row of Table 1 for $\\hat V_N^{b+}$, with variance 0.196 and interval $(-1.548, 0.185)$, gives $c^2\\le 1.734^2/0.1955 = 44217/2875$, which implies that $\\bar v\\ge 30364623/157216000\\approx 0.193140 > 0.193$. The inequality $\\bar w^2\\le c^2\\bar v$ and the contradiction obtained from the bound on $c^2$ have a Lean proof.\n\nWith $z=1.96$, the table summaries bound the simulated bias between $-0.006192$ and $-0.005$, so it rounds to $-0.006$ or to $-0.005$; they do not determine the corrected figure. If the figure $-0.007$ is an exact bias computed separately from the simulations of Table 2, the contradiction is not strict. In that case 25 million independent uniform assignments would produce the discrepancy with probability below $10^{-10}$.", "problem": "The bias of a variance estimator is its expectation less the true variance, and the paragraph discusses the 25 million simulated assignments that Table 2 summarizes. For those assignments the table reports a true variance of 0.199, a mean variance estimate of 0.193 for $\\hat V_N^{b+}$, and a mean width of 1.724 for its 95\\% intervals. A bias that rounds to $-0.007$ and a true variance that rounds to 0.199 require a mean variance estimate of at most 0.193. The mean width requires more. Each interval is a Wald interval of full width $2z\\sqrt{v}$, where $v$ is the variance estimate, and the square of a mean of square roots is at most the mean. With $z=1.96$, a mean width of at least 1.7235 forces a mean variance estimate of at least 0.193308. The four printed numbers cannot all be rounded summaries of the same simulations with the stated intervals. At least one of them is off in the third decimal, or the intervals were not constructed as stated. The simulation output is needed to say which.", "what_survives": "The statement that $\\hat V_N^{b+}$ has nonpositive bias under the Sharp Null is correct, and $\\hat V_N^a$ is unbiased there. The ordering of the three estimators, the coverage figures and the bias of $-0.010$ USD$^2$ quoted for $\\hat V_N^H$ are not contradicted by this calculation. None of them was recomputed, since the data are not available. No result of the paper uses the figure." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-N005", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "page_check", "exhibits": [ { "alt": "Text from Section 6.1. A red box marks the symbol tau with subscript N, printed without a hat, in the phrase 'the sampling distribution of tau N is not perfectly normal'. The variance estimators in the neighbouring lines carry hats.", "caption": "Section 6.1, paragraph on the Sharp Null, p. 858 (PDF p. 9). The marked symbol is printed without a hat.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-N006.png" } ], "formal_release": null, "formal_statements": [], "headline": "The sampling distribution in the parenthesis is that of the estimator $\\hat\\tau_N$; the hat is missing.", "id": "P022-N006", "location": "Section 6.1, paragraph on the Sharp Null, p. 858 (PDF p. 9), parenthesis after “95.2% and 94.1%, respectively”", "pdf_page": 9, "printed_page": "858", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": "Minor", "subject": "paper", "target_ids": [], "text": { "as_printed": "(the former is not exactly 95\\% because the sampling distribution of $\\hl{\\tau_N}$ is not perfectly normal)", "corrected": "(the former is not exactly 95\\% because the sampling distribution of $\\hl{\\hat\\tau_N}$ is not perfectly normal)", "evidence": "I checked the page at high magnification. The $\\tau$ carries no hat, while the variance estimators in the neighbouring lines do.", "problem": "The quantity $\\tau_N$ is the average treatment effect in the population. It is fixed under the design and has no sampling distribution. The parenthesis refers to the difference-in-means estimator $\\hat\\tau_N$ defined in (1).", "what_survives": "The coverage figures and the explanation offered for them are unaffected." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-N006", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" }, { "before_review": null, "class": "clarity_note", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "complete_derivation", "exhibits": [ { "alt": "The bound for the first term in the proof of Proposition 2: a four-line display followed by four lines of text. A red box in the first line of the display marks the fraction N (m minus 1) over m (N minus 1). A second red box in the text marks the symbol beta with subscript N.", "caption": "Proof of Proposition 2, bound for the first term, p. 865 (PDF p. 16). The lower mark is the first use of $\\beta_N$, and the upper mark is the coefficient in the display that it stands for. The symbol recurs once in the next line.", "url": "https://pmaronow.github.io/llm-audit/works/P022/exhibits/P022-N007.png" } ], "formal_release": null, "formal_statements": [], "headline": "The symbol $\\beta_N$ stands for the coefficient $N(m-1)/\\{m(N-1)\\}$ in the display above it; the paper does not define it.", "id": "P022-N007", "location": "Proof of Proposition 2, bound for the first term, p. 865 (PDF p. 16), sentence beginning “where the first inequality follows from”", "pdf_page": 16, "printed_page": "865", "reaches": [], "related_entries": [], "revisions": [], "same_defect_as": [], "severity": null, "subject": "paper", "target_ids": [], "text": { "as_printed": "where the first inequality follows from $|\\mathrm{E}_{\\hat G_N}(y_1^2)-\\hl{\\beta_N}\\mathrm{E}_{G_N}(y_1^2)|$ ${}\\le|\\mathrm{E}_{\\hat G_N}(y_1^2)-\\mathrm{E}_{G_N}(y_1^2)|$ ${}+|(1-\\hl{\\beta_N})|\\mathrm{E}_{G_N}(y_1^2)$, and the second inequality from Chebyshev's inequality and the fact that $\\mathrm{E}_X\\mathrm{E}_{\\hat G_N}(y_1^p)=\\mathrm{E}_{G_N}(y^p)$.", "corrected": "where $\\hl{\\beta_N=N(m-1)/\\{m(N-1)\\}}$, the first inequality follows from $|\\mathrm{E}_{\\hat G_N}(y_1^2)-\\beta_N\\mathrm{E}_{G_N}(y_1^2)|\\le|\\mathrm{E}_{\\hat G_N}(y_1^2)-\\mathrm{E}_{G_N}(y_1^2)|+|(1-\\beta_N)|\\mathrm{E}_{G_N}(y_1^2)$, and the second inequality from Chebyshev's inequality and the fact that $\\mathrm{E}_X\\mathrm{E}_{\\hat G_N}(y_1^p)=\\mathrm{E}_{G_N}(y^p)$.", "intended_meaning": "The symbol $\\beta_N$ occurs only in this sentence. It abbreviates the coefficient of $\\mathrm{E}_{G_N}(y_1^2)$ in the first line of the display above, that is, $\\beta_N=N(m-1)/\\{m(N-1)\\}$. With this value, $1-\\beta_N=(N-m)/\\{m(N-1)\\}$, which accounts for the term $(N-m)C^2/\\{m(N-1)\\}$ in the second line of the display, and the first inequality holds. In the last identity, $y$ is the generic argument of the distribution $G_N$, so that $\\mathrm{E}_{G_N}(y^p)$ and $\\mathrm{E}_{G_N}(y_1^p)$ denote the same moment and the identity is correct as printed." }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-N007", "work": { "arxiv_id": "1405.6555", "doi": "10.1214/13-aos1200", "title": "Sharp bounds on the variance in randomized experiments", "url": "https://pmaronow.github.io/llm-audit/works/P022/" }, "work_id": "P022" } ], "lean": { "applicable": true, "guide": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/docs/works/P022.md", "repository": "https://github.com/pmaronow/llm-audit-lean", "targets": [ { "declarations": [ { "claim_revision_id": "catalog:39801869ed7c81f85af6351ae84cabd7dbb5fa4020cb1fddee2ee52074aee723", "conditional_on": [], "credit_reason": "Auxiliary for the two half-width bins in part (a). Its extra generality is incidental to a valid algebraic rounding fact.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.half_width_rounding", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L23", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:39801869ed7c81f85af6351ae84cabd7dbb5fa4020cb1fddee2ee52074aee723", "conditional_on": [], "credit_reason": "The half-width hypotheses are the very bounds obtained from the endpoint bins; this row is an auxiliary ratio inequality, not an independent rounding model.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.table1_ratio_lower_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L33", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:39801869ed7c81f85af6351ae84cabd7dbb5fa4020cb1fddee2ee52074aee723", "conditional_on": [], "credit_reason": "Exactly the combined-bins assertion in part (b). Part (a) does not need the variance bins and is supported separately by the half-width and ratio lemmas; the row is not being treated as a proof under endpoint bins alone.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.table1_corrected_reduction", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L53", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:39801869ed7c81f85af6351ae84cabd7dbb5fa4020cb1fddee2ee52074aee723", "conditional_on": [], "credit_reason": "Exactly part (c); it does not assert that 1300/199 is the unrounded empirical percentage.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.displayed_variance_reduction", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L91", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:39801869ed7c81f85af6351ae84cabd7dbb5fa4020cb1fddee2ee52074aee723", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T11", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T11" }, "html": { "body": "

p. 857 (PDF p. 8). Closed rounding bins and a common positive Wald multiplier imply the endpoint-only upper bound, the combined six-percent bounds, and the exact displayed-input percentage.

", "headline": "Section 6, 'a 7% reduction in the nominal variance' (with Table 1), corrected claim", "location": "p. 857 (PDF p. 8)" }, "id": "P022-T11", "text": { "body": "p. 857 (PDF p. 8). Closed rounding bins and a common positive Wald multiplier imply the endpoint-only upper bound, the combined six-percent bounds, and the exact displayed-input percentage.", "headline": "Section 6, 'a 7% reduction in the nominal variance' (with Table 1), corrected claim", "location": "p. 857 (PDF p. 8)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T11" }, { "declarations": [ { "claim_revision_id": "catalog:5729aa891728893d8feb6c66361256e0ecbc375f1b611ff02627bd68645ea508", "conditional_on": [], "credit_reason": "The finite Wald-width row directly supplies part (b); its square-root helper is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.finite_sqrt_mean_squared_le_mean", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L97", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:5729aa891728893d8feb6c66361256e0ecbc375f1b611ff02627bd68645ea508", "conditional_on": [], "credit_reason": "Exactly the finite realized-collection inequality in part (b), including negative or zero real multipliers. No population-expectation substitution is made.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.finite_wald_mean_width_squared_le", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L111", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:5729aa891728893d8feb6c66361256e0ecbc375f1b611ff02627bd68645ea508", "conditional_on": [], "credit_reason": "Exactly the algebraic multiplier bound in (c); the theorem assumes rather than silently estimates the width-variance identity.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.table1_multiplier_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L121", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:5729aa891728893d8feb6c66361256e0ecbc375f1b611ff02627bd68645ea508", "conditional_on": [], "credit_reason": "The incompatibility is deterministic only under the target’s explicit same-realized-collection bias interpretation. The Lean premises match that corrected nonexistence claim exactly.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reported_numbers_incompatible", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L135", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:5729aa891728893d8feb6c66361256e0ecbc375f1b611ff02627bd68645ea508", "conditional_on": [], "credit_reason": "Covers the lower-bound component of (e); the separate upper cap there follows directly from its printed bounds on true variance and bias.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.fixed_multiplier_mean_lower", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Numerical.lean#L155", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:5729aa891728893d8feb6c66361256e0ecbc375f1b611ff02627bd68645ea508", "conditional_on": [], "credit_reason": "The conclusion is nonpositive bias, including equality. That is exactly the retained corrected claim, which asserts neither strict negativity nor the deleted numerical bias.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.sharp_null_neyman_bias_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GP022B1.lean#L294", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:5729aa891728893d8feb6c66361256e0ecbc375f1b611ff02627bd68645ea508", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T12", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T12" }, "html": { "body": "

p. 858 (PDF p. 9); Table 1 on p. 857 (PDF p. 8). Neyman’s upper estimator has nonpositive sharp-null bias, and the finite Wald-width and rounded-number calculations establish the corrected incompatibility statement.

", "headline": "Section 6.1, Sharp Null: the bias '(−0.007-0.007−0.007 USD2^22)' of V^Nb+\\hat V_N^{b+}V^Nb+​ (with Table 2), corrected claim", "location": "p. 858 (PDF p. 9); Table 1 on p. 857 (PDF p. 8)" }, "id": "P022-T12", "text": { "body": "p. 858 (PDF p. 9); Table 1 on p. 857 (PDF p. 8). Neyman’s upper estimator has nonpositive sharp-null bias, and the finite Wald-width and rounded-number calculations establish the corrected incompatibility statement.", "headline": "Section 6.1, Sharp Null: the bias '($-0.007$ USD$^2$)' of $\\hat V_N^{b+}$ (with Table 2), corrected claim", "location": "p. 858 (PDF p. 9); Table 1 on p. 857 (PDF p. 8)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T12" }, { "declarations": [ { "claim_revision_id": "catalog:3e55640247e48b6172a17e705cac14663744ba30878849a31641141d5048067e", "conditional_on": [], "credit_reason": "The explicit corrected-lemma row also fixes the universal positive nonincreasing integer function.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.TailA.lemma2_corrected_final", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/TailA/Lemma2.lean#L8", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:3e55640247e48b6172a17e705cac14663744ba30878849a31641141d5048067e", "conditional_on": [], "credit_reason": "The explicit corrected-lemma row supplies the target; this scalar bracketing helper is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.PaperDomains.cdf_partition_bracket", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_PaperDomains.lean#L7", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:3e55640247e48b6172a17e705cac14663744ba30878849a31641141d5048067e", "conditional_on": [], "credit_reason": "Its positivity and monotonicity conclusions are already in the explicit corrected-lemma row.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.corrected_partition_integer_properties", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L873", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3e55640247e48b6172a17e705cac14663744ba30878849a31641141d5048067e", "conditional_on": [], "credit_reason": "The same universal eta can be instantiated separately for each arm, so both eta_1 and eta_0 quantifiers are preserved. The explicit K is independent of H exactly as the corrected target allows.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.lemma2_corrected_explicit_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GLP022B3.lean#L624", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:3e55640247e48b6172a17e705cac14663744ba30878849a31641141d5048067e", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T2", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T2" }, "html": { "body": "

p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12). Convergence of left limits at atoms gives uniform CDF convergence and both scaled tail bounds with the explicit universal integer function.

", "headline": "Lemma 2, corrected", "location": "p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12)" }, "id": "P022-T2", "text": { "body": "p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12). Convergence of left limits at atoms gives uniform CDF convergence and both scaled tail bounds with the explicit universal integer function.", "headline": "Lemma 2, corrected", "location": "p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T2" }, { "declarations": [ { "claim_revision_id": "catalog:b54b3b19aff094d46b2c68b7e823041569cb568b65473d3d10405c8797396fec", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P022/Wave1/PrintedLemma2.lean:9 negates the universal Conditions123-to-Lemma2Conclusions contract, and P022/P022_Variance.lean:243 does allow K1 and K0 to depend on the experiment and allocation parameters. This weakens the positive assertion relative to the printed dependence on eta and H, so refuting it is stronger and directly refutes the printed lemma as well. Independently of that quantifier difference, the reciprocal witness at P022/P022_Variance.lean:270 satisfies the printed conditions, while P022/P022_Variance.lean:850 states that its population CDF distances equal one for every row, contradicting the uniform-convergence conclusion. The constants cannot affect that failure, so their relaxed dependence creates no material fidelity problem for this refutation.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.lemma2_as_printed_refuted", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/PrintedLemma2.lean#L9", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:b54b3b19aff094d46b2c68b7e823041569cb568b65473d3d10405c8797396fec", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T18", "decision": "refuted", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T18" }, "html": { "body": "

The printed claim is false: P022-T18; p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12). There exists a sequence satisfying conditions 1–3 for which the joint conclusion of printed Lemma 2 fails.

", "headline": "Lemma 2", "location": "p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12)" }, "id": "P022-T18", "text": { "body": "The printed claim is false: P022-T18; p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12). There exists a sequence satisfying conditions 1–3 for which the joint conclusion of printed Lemma 2 fails.", "headline": "Lemma 2", "location": "p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T18" }, { "declarations": [ { "claim_revision_id": "catalog:966b0ead0327301e97943c231cdd89825ea9c63ddf93ef74a67bb25d993f2bf4", "conditional_on": [], "credit_reason": "All sample-size and tolerance conditions match finite parts (a)-(c). Explicit C>=0 is already implied by the bounds on a nonempty population. The empirical-quantile endpoint convention affects only a null set in the integrals.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.bounded_direct_finite_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GLP022B3.lean#L703", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:966b0ead0327301e97943c231cdd89825ea9c63ddf93ef74a67bb25d993f2bf4", "conditional_on": [], "credit_reason": "This row covers the asymptotic component, with the finite component covered by R143-19. Metadata inconsistency: P022-T3 has asymptotic=false although its statement expressly includes this asymptotic consequence. ENNReal preserves infinite limsups, and its finite right side is nonnegative.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.TailA.bounded_direct_asymptotic_final", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/TailA/BoundedAsymptotic.lean#L10", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:966b0ead0327301e97943c231cdd89825ea9c63ddf93ef74a67bb25d993f2bf4", "conditional_on": [], "credit_reason": "The finite and asymptotic bounded-outcome rows already supply the requested probability bounds.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.finite_squared_markov", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L953", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:966b0ead0327301e97943c231cdd89825ea9c63ddf93ef74a67bb25d993f2bf4", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T3", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T3" }, "html": { "body": "

pp. 867–869 (PDF pp. 18–20); statement of Proposition 2 on p. 863 (PDF p. 14). Bounded outcomes give the stated finite second moments, both deterministic endpoint errors, finite tail bounds, and the allocation-only asymptotic bound.

", "headline": "Proof of Proposition 2, the steps that apply Lemma 2, corrected (direct bound for bounded outcomes)", "location": "pp. 867–869 (PDF pp. 18–20); statement of Proposition 2 on p. 863 (PDF p. 14)" }, "id": "P022-T3", "text": { "body": "pp. 867–869 (PDF pp. 18–20); statement of Proposition 2 on p. 863 (PDF p. 14). Bounded outcomes give the stated finite second moments, both deterministic endpoint errors, finite tail bounds, and the allocation-only asymptotic bound.", "headline": "Proof of Proposition 2, the steps that apply Lemma 2, corrected (direct bound for bounded outcomes)", "location": "pp. 867–869 (PDF pp. 18–20); statement of Proposition 2 on p. 863 (PDF p. 14)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T3" }, { "declarations": [ { "claim_revision_id": "catalog:3e667e6c8d737f81289d0b86094a1cb05a2fdb35cb7890d254af78a47d3a986f", "conditional_on": [], "credit_reason": "This is the corrected target: the last relation is an inequality. No equality from printed (11) is credited.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.cdf_variance_corrected_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GP022B2.lean#L117", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3e667e6c8d737f81289d0b86094a1cb05a2fdb35cb7890d254af78a47d3a986f", "conditional_on": [], "credit_reason": "The corrected CDF-variance row already supplies both quarter-factor upper bounds.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.bernoulli_variance_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L695", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3e667e6c8d737f81289d0b86094a1cb05a2fdb35cb7890d254af78a47d3a986f", "conditional_on": [], "credit_reason": "The indicator-variance row supplies the relevant population-variance identity directly.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.finite_variance_second_moment", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L895", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3e667e6c8d737f81289d0b86094a1cb05a2fdb35cb7890d254af78a47d3a986f", "conditional_on": [], "credit_reason": "The statement covers precisely the <= threshold convention and extends it to smaller nonempty populations.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.indicator_population_variance", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L912", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3e667e6c8d737f81289d0b86094a1cb05a2fdb35cb7890d254af78a47d3a986f", "conditional_on": [], "credit_reason": "Both unbiasedness identities are already in the corrected CDF-variance row.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.sampled_cdfs_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1225", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:3e667e6c8d737f81289d0b86094a1cb05a2fdb35cb7890d254af78a47d3a986f", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T4", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T4" }, "html": { "body": "

p. 861 (PDF p. 12). Both sampled CDFs are unbiased, have the corrected Bernoulli-factor variances, and satisfy the quarter-factor upper bounds.

", "headline": "Equation (11) in the proof of Lemma 2, corrected (with equation (12))", "location": "p. 861 (PDF p. 12)" }, "id": "P022-T4", "text": { "body": "p. 861 (PDF p. 12). Both sampled CDFs are unbiased, have the corrected Bernoulli-factor variances, and satisfy the quarter-factor upper bounds.", "headline": "Equation (11) in the proof of Lemma 2, corrected (with equation (12))", "location": "p. 861 (PDF p. 12)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T4" }, { "declarations": [ { "claim_revision_id": "catalog:a811f875f9a7fdc75bea0e41249cf647de52048961ce44de212c60ccf419587f", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.finiteMean_const", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L287", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:a811f875f9a7fdc75bea0e41249cf647de52048961ce44de212c60ccf419587f", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.design_constant", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L294", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:a811f875f9a7fdc75bea0e41249cf647de52048961ce44de212c60ccf419587f", "conditional_on": [], "credit_reason": "The witness refutes the universal printed equality. The additional general equality criterion in T5 follows from the corrected variance formula and R144-02; this row describes only the concrete witness.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.equation11_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L305", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:a811f875f9a7fdc75bea0e41249cf647de52048961ce44de212c60ccf419587f", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T5", "decision": "refuted", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T5" }, "html": { "body": "

The printed claim is false: P022-T5; p. 861 (PDF p. 12). The four-unit constant-one example refutes equation (11), and equality in its quarter-factor bound holds exactly when the population CDF is one half.

", "headline": "Equation (11) in the proof of Lemma 2, as printed", "location": "p. 861 (PDF p. 12)" }, "id": "P022-T5", "text": { "body": "The printed claim is false: P022-T5; p. 861 (PDF p. 12). The four-unit constant-one example refutes equation (11), and equality in its quarter-factor bound holds exactly when the population CDF is one half.", "headline": "Equation (11) in the proof of Lemma 2, as printed", "location": "p. 861 (PDF p. 12)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T5" }, { "declarations": [ { "claim_revision_id": "catalog:3198e862b40b7e4523e92d63ffdfa535b024ba6aff197837f8ad3f9a8a46f0a8", "conditional_on": [], "credit_reason": "Matches the corrected exact-arithmetic target, including both covariance endpoints and overlap masses. No assertion about floating-point execution is part of this target.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.appendixB_corrected_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GLP022B4.lean#L1001", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3198e862b40b7e4523e92d63ffdfa535b024ba6aff197837f8ad3f9a8a46f0a8", "conditional_on": [], "credit_reason": "The corrected Appendix B row already supplies the centered covariance and variance substitution identities.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.centered_quadrature", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L939", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false } ], "formalization": { "claim_revision_id": "catalog:3198e862b40b7e4523e92d63ffdfa535b024ba6aff197837f8ad3f9a8a46f0a8", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T8", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T8" }, "html": { "body": "

p. 869 (PDF p. 20); equations (8)–(9) on p. 854 (PDF p. 5). Using the original partition widths and midpoint quantile selections returns the sharp covariance and variance estimates, with the stated rank and overlap-weight identities.

", "headline": "Appendix B, function sharp.var: partition points and integration weights, corrected", "location": "p. 869 (PDF p. 20); equations (8)–(9) on p. 854 (PDF p. 5)" }, "id": "P022-T8", "text": { "body": "p. 869 (PDF p. 20); equations (8)–(9) on p. 854 (PDF p. 5). Using the original partition widths and midpoint quantile selections returns the sharp covariance and variance estimates, with the stated rank and overlap-weight identities.", "headline": "Appendix B, function sharp.var: partition points and integration weights, corrected", "location": "p. 869 (PDF p. 20); equations (8)–(9) on p. 854 (PDF p. 5)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T8" }, { "declarations": [ { "claim_revision_id": "catalog:f3737142e461daaf4360f03c0a5815387123f4e1c6c8eb13c9c0128823b9582f", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.partition_two", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L374", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f3737142e461daaf4360f03c0a5815387123f4e1c6c8eb13c9c0128823b9582f", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.quantile_two_ones", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L391", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f3737142e461daaf4360f03c0a5815387123f4e1c6c8eb13c9c0128823b9582f", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.sharp_covariance_two_ones", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L403", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f3737142e461daaf4360f03c0a5815387123f4e1c6c8eb13c9c0128823b9582f", "conditional_on": [], "credit_reason": "This is a witness for failure of the printed routine, not a proof of its positive correctness claim. All printed witness hypotheses hold, including the permitted constant outcomes and census design.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.appendixB_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L426", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:f3737142e461daaf4360f03c0a5815387123f4e1c6c8eb13c9c0128823b9582f", "conditional_on": [], "credit_reason": "Auxiliary general-discrepancy component; both discrepancy equalities are exported. The real-arithmetic model and width restriction are already explicit in the target, so neither is an extra Lean assumption.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.appendixB_general_discrepancy_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GP022B3.lean#L354", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:f3737142e461daaf4360f03c0a5815387123f4e1c6c8eb13c9c0128823b9582f", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T9", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T9" }, "html": { "body": "

The printed claim is false: P022-T9; Appendix B, p. 869 (PDF p. 20), (8)–(9) on p. 854. In real arithmetic the all-ones four-unit example has constant observed samples, zero effect, variance and sharp estimates, but the printed routine returns the negative value -1/50331648. Its shifted weights sum to 1-2e. Under the stated cell-width condition both endpoint discrepancies and the exact translation defect hold, while the mathematical sharp estimates are translation invariant. The usual measure-theoretic conditions are assumed.

", "headline": "Appendix B, function sharp.var, as printed", "location": "p. 869 (PDF p. 20); equations (8)–(9) on p. 854 (PDF p. 5)" }, "id": "P022-T9", "text": { "body": "The printed claim is false: P022-T9; Appendix B, p. 869 (PDF p. 20), (8)–(9) on p. 854. In real arithmetic the all-ones four-unit example has constant observed samples, zero effect, variance and sharp estimates, but the printed routine returns the negative value -1/50331648. Its shifted weights sum to 1-2e. Under the stated cell-width condition both endpoint discrepancies and the exact translation defect hold, while the mathematical sharp estimates are translation invariant. The usual measure-theoretic conditions are assumed.", "headline": "Appendix B, function sharp.var, as printed", "location": "p. 869 (PDF p. 20); equations (8)–(9) on p. 854 (PDF p. 5)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T9" }, { "declarations": [ { "claim_revision_id": "catalog:d0ca476b9a458a52fe0b9e8d05d632d5d23cd2a3825a21f28da203fa0adfd93a", "conditional_on": [], "credit_reason": "This is the mathematical fixed-sample identity in part (a), with no claim to execute an R branch at an infinite argument.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.fixed_sample_identity", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L456", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d0ca476b9a458a52fe0b9e8d05d632d5d23cd2a3825a21f28da203fa0adfd93a", "conditional_on": [], "credit_reason": "This proves the mathematical independent-means estimator limit for both modes; finite exceptional natural population sizes are irrelevant.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.fixed_sample_infinite_limit", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L475", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d0ca476b9a458a52fe0b9e8d05d632d5d23cd2a3825a21f28da203fa0adfd93a", "conditional_on": [], "credit_reason": "This is algebra for varianceExpression; the restricted theorem does not assert its exact identity for actual variance.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.independent_means_remainder", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L506", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:d0ca476b9a458a52fe0b9e8d05d632d5d23cd2a3825a21f28da203fa0adfd93a", "conditional_on": [], "credit_reason": "Eventually constant arm sizes correctly avoid the artificial all-row restriction caused by starting the sequence at N=4. The corrected target itself supplies the bounded-variance condition; no positive limiting sampling fraction is assumed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.fixed_arm_variance_limit_eventually", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/FixedArm.lean#L13", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:d0ca476b9a458a52fe0b9e8d05d632d5d23cd2a3825a21f28da203fa0adfd93a", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T10", "decision": "restricted", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T10" }, "html": { "body": "

p. 850 (PDF p. 1); p. 852 (PDF p. 3); Appendix B, p. 869 (PDF p. 20). Fixed observed samples give the independent-means estimator limit, and bounded marginal variances with eventual fixed arm sizes give the corresponding actual-variance approximation.

", "headline": "The infinite-population case (N=∞N = \\inftyN=∞) of the estimator: Introduction and Section 2, and Appendix B called with N=Inf, corrected claim", "location": "p. 850 (PDF p. 1); p. 852 (PDF p. 3); Appendix B, p. 869 (PDF p. 20)" }, "id": "P022-T10", "text": { "body": "p. 850 (PDF p. 1); p. 852 (PDF p. 3); Appendix B, p. 869 (PDF p. 20). Fixed observed samples give the independent-means estimator limit, and bounded marginal variances with eventual fixed arm sizes give the corresponding actual-variance approximation.", "headline": "The infinite-population case ($N = \\infty$) of the estimator: Introduction and Section 2, and Appendix B called with N=Inf, corrected claim", "location": "p. 850 (PDF p. 1); p. 852 (PDF p. 3); Appendix B, p. 869 (PDF p. 20)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T10" }, { "declarations": [ { "claim_revision_id": "catalog:fe50ce950d42e2338d2879e6af0f64be6ff6a936d545d455eeea3126f8dc1858", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P022/Wave1/SharpBounds.lean:24 includes unbiasedness, the actual assignment-variance identity, bounds for every population with the given marginal multisets, endpoint attainment, and covariance bounds and quantile-law attainment for every compatible Borel probability coupling. Design at P022/P022_Variance.lean:13 imposes precisely the standing population and arm-size conditions, and uniform permutations give each unordered assignment the same multiplicity. Equality of all bounded continuous test integrals specifies the printed equality in distribution, while the quantile endpoint totalization changes no uniform law or integral. None of the noted representations narrows Lemma 1 or the sharp variance interval claim.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.sharp_bounds_variance_and_all_couplings", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/SharpBounds.lean#L24", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fe50ce950d42e2338d2879e6af0f64be6ff6a936d545d455eeea3126f8dc1858", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.finite_covariance_cauchy", "manifest_use": "not_used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L925", "status": "proved", "support_scope": "historical declaration", "used_in_solution": false }, { "claim_revision_id": "catalog:fe50ce950d42e2338d2879e6af0f64be6ff6a936d545d455eeea3126f8dc1858", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.permutation_eval_mean", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1172", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fe50ce950d42e2338d2879e6af0f64be6ff6a936d545d455eeea3126f8dc1858", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.observed_means_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1198", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:fe50ce950d42e2338d2879e6af0f64be6ff6a936d545d455eeea3126f8dc1858", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.difference_in_means_unbiased", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1216", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:fe50ce950d42e2338d2879e6af0f64be6ff6a936d545d455eeea3126f8dc1858", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T13", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T13" }, "html": { "body": "

p. 853 (PDF p. 4); equation (2) on p. 851 (PDF p. 2); proof of Lemma 1 on pp. 859–860 (PDF pp. 10–11). Difference in means is unbiased with the printed variance, and the covariance and variance endpoints bound and attain all compatible finite rearrangements and probability couplings.

", "headline": "Lemma 1 (Hoeffding) and the sentence 'Lemma 1 implies that [VNL,VNH][V_N^L, V_N^H][VNL​,VNH​] is the sharpest interval bound for VNV_NVN​'; with equation (2)", "location": "p. 853 (PDF p. 4); equation (2) on p. 851 (PDF p. 2); proof of Lemma 1 on pp. 859–860 (PDF pp. 10–11)" }, "id": "P022-T13", "text": { "body": "p. 853 (PDF p. 4); equation (2) on p. 851 (PDF p. 2); proof of Lemma 1 on pp. 859–860 (PDF pp. 10–11). Difference in means is unbiased with the printed variance, and the covariance and variance endpoints bound and attain all compatible finite rearrangements and probability couplings.", "headline": "Lemma 1 (Hoeffding) and the sentence 'Lemma 1 implies that $[V_N^L, V_N^H]$ is the sharpest interval bound for $V_N$'; with equation (2)", "location": "p. 853 (PDF p. 4); equation (2) on p. 851 (PDF p. 2); proof of Lemma 1 on pp. 859–860 (PDF pp. 10–11)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T13" }, { "declarations": [ { "claim_revision_id": "catalog:efb2915a55a461eb2194d0a3ef2d32ef8c9836f22b7a55aa8bfaf52642252628", "conditional_on": [], "credit_reason": "The quantified pointwise ordering is fully established. The introductory phrase often smaller carries no frequency quantifier in the target formal reading.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.estimated_bound_ordering_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GP022B1.lean#L657", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:efb2915a55a461eb2194d0a3ef2d32ef8c9836f22b7a55aa8bfaf52642252628", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.sqrt_product_le_arithmetic", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1245", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:efb2915a55a461eb2194d0a3ef2d32ef8c9836f22b7a55aa8bfaf52642252628", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T14", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T14" }, "html": { "body": "

p. 854 (PDF p. 5); (4) and (6) on pp. 852–853 (PDF pp. 3–4); Introduction, p. 850 (PDF p. 1). When the full population is sampled, every assignment obeys the five-estimator ordering from Neyman’s lower estimate through the conventional upper estimate.

", "headline": "Section 3, sentence after equation (9): V^NH\\hat V_N^HV^NH​ is never greater than V^Nb+\\hat V_N^{b+}V^Nb+​, and V^NL\\hat V_N^LV^NL​ is never smaller than V^Nb−\\hat V_N^{b-}V^Nb−​", "location": "p. 854 (PDF p. 5); (4) and (6) on pp. 852–853 (PDF pp. 3–4); Introduction, p. 850 (PDF p. 1)" }, "id": "P022-T14", "text": { "body": "p. 854 (PDF p. 5); (4) and (6) on pp. 852–853 (PDF pp. 3–4); Introduction, p. 850 (PDF p. 1). When the full population is sampled, every assignment obeys the five-estimator ordering from Neyman’s lower estimate through the conventional upper estimate.", "headline": "Section 3, sentence after equation (9): $\\hat V_N^H$ is never greater than $\\hat V_N^{b+}$, and $\\hat V_N^L$ is never smaller than $\\hat V_N^{b-}$", "location": "p. 854 (PDF p. 5); (4) and (6) on pp. 852–853 (PDF pp. 3–4); Introduction, p. 850 (PDF p. 1)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T14" }, { "declarations": [ { "claim_revision_id": "catalog:b021afaa9ac3e5f516bd3310eeb06aff8bfb26000d44430ce5c6a98f4ee1b9d2", "conditional_on": [], "credit_reason": "All moment and consistency claims are conclusions of the inherited theorem type, not assumptions inside a structure. Coordinate second moments bound every compatible covariance range, so sSup/sInf and Bochner integrals have their intended meanings. No additional left-CDF convergence at atoms is assumed.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.TailB_proposition1", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/TailBBoundConsistency.lean#L523", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:b021afaa9ac3e5f516bd3310eeb06aff8bfb26000d44430ce5c6a98f4ee1b9d2", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T15", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T15" }, "html": { "body": "

p. 855 (PDF p. 6); proof on pp. 862–863 (PDF pp. 13–14). Conditions 1–4 give second moments for all compatible laws, both sharp population limits, and joint bound-estimator consistency at scale one over population size.

", "headline": "Proposition 1", "location": "p. 855 (PDF p. 6); proof on pp. 862–863 (PDF pp. 13–14)" }, "id": "P022-T15", "text": { "body": "p. 855 (PDF p. 6); proof on pp. 862–863 (PDF pp. 13–14). Conditions 1–4 give second moments for all compatible laws, both sharp population limits, and joint bound-estimator consistency at scale one over population size.", "headline": "Proposition 1", "location": "p. 855 (PDF p. 6); proof on pp. 862–863 (PDF pp. 13–14)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T15" }, { "declarations": [ { "claim_revision_id": "catalog:d801fda1809384cf5ff3721eaf40b05567e7f02cc1c696a177cee7495b36c36f", "conditional_on": [], "credit_reason": "{'source': 'adjudication', 'decision': 'rejected', 'what': 'P022/Wave1/Gap/GP022B1b.lean:1001 supplies both endpoint bounds for every positive tolerance allocation under Conditions123 and uniformly bounded outcomes. Conditions123 at P022/P022_Variance.lean:194 contains the printed conditions 1–3 without an extra left-limit or uniform-CDF assumption, and scaledProbabilityLimsup at :233 is the extended-real limsup of N times the actual assignment probability. Finite nonnegative κ functions are chosen before the sequence and tolerance allocation with only the parameter dependencies permitted by the target. The lower bound with outside coefficient one strengthens the allowed finite-constant bound, and N = q + 4 merely indexes all population sizes in the standing setting; neither difference removes any case.'}", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.proposition2_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GP022B1b.lean#L999", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:d801fda1809384cf5ff3721eaf40b05567e7f02cc1c696a177cee7495b36c36f", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T16", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T16" }, "html": { "body": "

p. 863 (PDF p. 14); Remark 3 on p. 856 (PDF p. 7); proof on pp. 863–869 (PDF pp. 14–20). Bounded outcomes give both stated scaled tail limit-superior bounds uniformly over admissible sequences with the same limiting law and allocation parameters.

", "headline": "Proposition 2 (with Remark 3)", "location": "p. 863 (PDF p. 14); Remark 3 on p. 856 (PDF p. 7); proof on pp. 863–869 (PDF pp. 14–20)" }, "id": "P022-T16", "text": { "body": "p. 863 (PDF p. 14); Remark 3 on p. 856 (PDF p. 7); proof on pp. 863–869 (PDF pp. 14–20). Bounded outcomes give both stated scaled tail limit-superior bounds uniformly over admissible sequences with the same limiting law and allocation parameters.", "headline": "Proposition 2 (with Remark 3)", "location": "p. 863 (PDF p. 14); Remark 3 on p. 856 (PDF p. 7); proof on pp. 863–869 (PDF pp. 14–20)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T16" }, { "declarations": [ { "claim_revision_id": "catalog:ed31831dd759d02c51a6a8b01164196b3c4bbf689932ef9e588f9db5b956e94e", "conditional_on": [], "credit_reason": "{'source': 'adjudication+second review', 'adjudicator': 'P022/Wave1/Gap/TailBCorollaryCoverage.lean:199 copies the type of corollary1_and_coverage at P022/P022_Variance.lean:764, whose conclusions establish the CLT, variance limit, and nominal-coverage inequality but contain no optimality assertion. The supplied target additionally says in Remark 4 that the interval is “asymptotically the narrowest Wald-type confidence interval assured to have at least the nominal coverage.” Showing coverage of this interval does not establish that every smaller marginal-based variance input fails to guarantee coverage across compatible joint laws. The record and row descriptions expressly exclude that conclusion, but this exclusion does not amend the supplied claim text; the reviewer correctly identified the omission but understated its severity. The finite-variance and square-root totalizations themselves are harmless under the positive limiting variance and consistency.', 'suggested_fix': 'Add an optimality conclusion: any asymptotic variance input determined only by the marginal laws that is strictly smaller than the sharp upper variance limit fails to guarantee nominal Wald coverage for some admissible compatible joint law. The quantification must cover alternative variance inputs and compatible laws, using endpoint attainment or admissible approximation where needed; the current coverage inequality alone is insufficient.', 'second_review': \"P022/Wave1/Gap/TailBCorollaryCoverage.lean:199 takes the type of corollary1_and_coverage at P022/P022_Variance.lean:764, which concludes the CLT, variance limit, and coverage limit and inequality but no optimality assertion. Target T17 expressly includes Remark 4's claim that the interval is 'asymptotically the narrowest Wald-type confidence interval assured to have at least the nominal coverage'; the exclusion in the row description does not remove that target requirement. Although limitingBound uses a supremum over compatible laws at P022/P022_Variance.lean:672, the exported theorem does not establish that a smaller marginal-based variance input loses the coverage guarantee for an admissible compatible law. A substantive optimality conclusion is therefore needed in addition to the existing coverage conclusions.\", 'severity': 'major'}", "credit_status": "narrower", "fidelity_status_baseline": "narrower", "lean_name": "AronowAudit.P022.Wave1.TailB_corollary1_and_coverage", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/TailBCorollaryCoverage.lean#L199", "status": "proved", "support_scope": "limited", "used_in_solution": true }, { "claim_revision_id": "catalog:ed31831dd759d02c51a6a8b01164196b3c4bbf689932ef9e588f9db5b956e94e", "conditional_on": [], "credit_reason": "The exact narrowed corollary row supplies the CLT and coverage conclusions directly.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.third_moment_tail_bound", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L969", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:ed31831dd759d02c51a6a8b01164196b3c4bbf689932ef9e588f9db5b956e94e", "conditional_on": [], "credit_reason": "The exact narrowed corollary row supplies the CLT and coverage conclusions directly.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.allocation_coefficient_product", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1254", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:ed31831dd759d02c51a6a8b01164196b3c4bbf689932ef9e588f9db5b956e94e", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T17", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T17" }, "html": { "body": "

Corollary 1, p. 856 (PDF p. 7), Remark 4, p. 857 (PDF p. 8). Conditions 1–3, uniformly bounded third moments and a nonsingular limiting law give the actual and upper variance limits, gamma in (0,1], the studentized normal limit, and the stated Wald coverage limit at least nominal. No positivity of every finite estimate is assumed; division is totalized on exceptional rows. The sharp upper variance is the least leading variance input assuring nominal Wald coverage uniformly over admissible joint laws with the given marginals. Every smaller nonnegative input is defeated by a nonsingular compatible law and a nested fixed-outcome population realizing it. The usual measure-theoretic conditions are assumed, and existence of the moments and extrema is concluded.

", "headline": "Corollary 1 (with Remark 4)", "location": "p. 856 (PDF p. 7); Remark 4 on p. 857 (PDF p. 8)" }, "id": "P022-T17", "text": { "body": "Corollary 1, p. 856 (PDF p. 7), Remark 4, p. 857 (PDF p. 8). Conditions 1–3, uniformly bounded third moments and a nonsingular limiting law give the actual and upper variance limits, gamma in (0,1], the studentized normal limit, and the stated Wald coverage limit at least nominal. No positivity of every finite estimate is assumed; division is totalized on exceptional rows. The sharp upper variance is the least leading variance input assuring nominal Wald coverage uniformly over admissible joint laws with the given marginals. Every smaller nonnegative input is defeated by a nonsingular compatible law and a nested fixed-outcome population realizing it. The usual measure-theoretic conditions are assumed, and existence of the moments and extrema is concluded.", "headline": "Corollary 1 (with Remark 4)", "location": "p. 856 (PDF p. 7); Remark 4 on p. 857 (PDF p. 8)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T17" }, { "declarations": [ { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.empirical_cdf_bounds", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L603", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.origin_cdf", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L613", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.zero_atom_distance", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L621", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reciprocal_below_threshold", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L649", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reciprocal_finite_errors", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L666", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reciprocal_atom_values", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L709", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reciprocal_outcome_limit", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L727", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reciprocal_weak_limit", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L734", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reciprocal_allocation_limit", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L762", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.origin_cdf_continuous_off_atom", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L794", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "The selected aggregate theorem supplies the target component directly; this constituent fact is not separately invoked.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.reciprocal_conditions123", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L811", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "Used only through the aggregate counterexample: its exact statement is the infinite limsup, without a standalone probability-one conclusion.", "credit_status": "narrower", "fidelity_status_baseline": "narrower", "lean_name": "AronowAudit.P022.reciprocal_tail_diverges", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L828", "status": "proved", "support_scope": "limited", "used_in_solution": true }, { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "conditional_on": [], "credit_reason": "This is an exact witness for the printed universal lemma, not a positive proof of it. Witness B and the optional condition-4 observation are unnecessary for this refutation and are not separately asserted here.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.lemma2_counterexample", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L849", "status": "counterexample_proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:3c769757fb7d3920e289c91124c980b16fb66fb2f3fbfb920eb42e7e9dbe7ca7", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T1", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T1" }, "html": { "body": "

The printed claim is false: P022-T1; p. 860 (PDF p. 11), proof on pp. 860–861. The fixed negative-reciprocal witness satisfies conditions 1–4 but has population and sample CDF error one, certain tails, and a persistent error at the required partition knot zero. The bounded perturbed four-corner witness satisfies conditions 1–4 with a nonsingular limit and has CDF error at least one half and certain tails. Thus both conclusions of printed Lemma 2 fail, even with a nonsingular limiting law. The usual measure-theoretic conditions are assumed.

", "headline": "Lemma 2, as printed", "location": "p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12)" }, "id": "P022-T1", "text": { "body": "The printed claim is false: P022-T1; p. 860 (PDF p. 11), proof on pp. 860–861. The fixed negative-reciprocal witness satisfies conditions 1–4 but has population and sample CDF error one, certain tails, and a persistent error at the required partition knot zero. The bounded perturbed four-corner witness satisfies conditions 1–4 with a nonsingular limit and has CDF error at least one half and certain tails. Thus both conclusions of printed Lemma 2 fail, even with a nonsingular limiting law. The usual measure-theoretic conditions are assumed.", "headline": "Lemma 2, as printed", "location": "p. 860 (PDF p. 11); proof on pp. 860–861 (PDF pp. 11–12)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T1" }, { "declarations": [ { "claim_revision_id": "catalog:68fce7399f43fc477a2635b1d719c3c18766b00d7357d99f69f1fe478d4ea3fc", "conditional_on": [], "credit_reason": "The omitted sign assumptions strengthen the auxiliary algebraic identity.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.corrected_allocation_sum", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1040", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:68fce7399f43fc477a2635b1d719c3c18766b00d7357d99f69f1fe478d4ea3fc", "conditional_on": [], "credit_reason": "The event-inclusion glue invokes seven_term_budget, which already incorporates this triangle bound.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.seven_total_abs_le", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1049", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:68fce7399f43fc477a2635b1d719c3c18766b00d7357d99f69f1fe478d4ea3fc", "conditional_on": [], "credit_reason": "This is the deterministic event-inclusion lemma; R144-09 relates the signed total to the estimator and supplies the probability inequality.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.seven_term_budget", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/P022_Variance.lean#L1061", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:68fce7399f43fc477a2635b1d719c3c18766b00d7357d99f69f1fe478d4ea3fc", "conditional_on": [], "credit_reason": "Covers the finite H-endpoint parts (a)-(b), including the decomposition. The pointwise event inclusion also follows directly by combining its identity with R144-12.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.proposition2_corrected_finite_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GP022B2b.lean#L463", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true }, { "claim_revision_id": "catalog:68fce7399f43fc477a2635b1d719c3c18766b00d7357d99f69f1fe478d4ea3fc", "conditional_on": [], "credit_reason": "This is the fixed-xi conditional assembly in T6(c)-(d), not a proof of printed Lemma 2, a minimization theorem, or a law-only constant uniform in all sequences and parameters. The selected K positivity/monotonicity is not itself a conclusion of this covered component. These limitations were already recognized in the blind backtranslation and claim-stage judgment.", "credit_status": "ok", "fidelity_status_baseline": "ok", "lean_name": "AronowAudit.P022.Wave1.proposition2_corrected_limit_proof", "manifest_use": "used", "named_axioms": [], "says": null, "source": "https://github.com/pmaronow/llm-audit-lean/blob/b9e9df44831afed4f2b8f63be5c0cc9169a5fda4/AronowAudit/P022/Wave1/Gap/GP022B3b.lean#L678", "status": "proved", "support_scope": "historical declaration", "used_in_solution": true } ], "formalization": { "claim_revision_id": "catalog:68fce7399f43fc477a2635b1d719c3c18766b00d7357d99f69f1fe478d4ea3fc", "crosswalk_url": "../../data/lean-crosswalk.json#/targets/P022-T6", "decision": "whole", "release_id": "sap-2026-11-08-assured-release", "target_id": "P022-T6" }, "html": { "body": "

pp. 865 and 867–868 (PDF pp. 16 and 18–19), Proposition 2 on p. 863. The corrected upper seven-term budget uses sum nu=epsilon1/2 and last tolerance epsilon1/2, and is centered at the sharp upper bound. The finite decomposition, event inclusion, union bound and last-term CDF bound hold. Conditional on the Lemma 2 conclusions, universal positive nonincreasing integer functions give the corrected last-term limit; conditional also on (17)–(22), the concluding bounds hold. The cubic-root allocation attains their minimum and gives a positive rate function independent of the sequence. The lower endpoint retains sum nu=epsilon1 and last tolerance epsilon2+epsilon3, with its decomposition, finite budget and conditional limits. The reciprocal thresholds require C>0; at C=0 both last integrals vanish. The usual measure-theoretic conditions are assumed.

", "headline": "Proof of Proposition 2: tolerance of the last term, equation (23) and the concluding display for V^NH\\hat V_N^HV^NH​, corrected", "location": "pp. 865 and 867–868 (PDF pp. 16 and 18–19); statement of Proposition 2 on p. 863 (PDF p. 14)" }, "id": "P022-T6", "text": { "body": "pp. 865 and 867–868 (PDF pp. 16 and 18–19), Proposition 2 on p. 863. The corrected upper seven-term budget uses sum nu=epsilon1/2 and last tolerance epsilon1/2, and is centered at the sharp upper bound. The finite decomposition, event inclusion, union bound and last-term CDF bound hold. Conditional on the Lemma 2 conclusions, universal positive nonincreasing integer functions give the corrected last-term limit; conditional also on (17)–(22), the concluding bounds hold. The cubic-root allocation attains their minimum and gives a positive rate function independent of the sequence. The lower endpoint retains sum nu=epsilon1 and last tolerance epsilon2+epsilon3, with its decomposition, finite budget and conditional limits. The reciprocal thresholds require C>0; at C=0 both last integrals vanish. The usual measure-theoretic conditions are assumed.", "headline": "Proof of Proposition 2: tolerance of the last term, equation (23) and the concluding display for $\\hat V_N^H$, corrected", "location": "pp. 865 and 867–868 (PDF pp. 16 and 18–19); statement of Proposition 2 on p. 863 (PDF p. 14)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T6" }, { "declarations": [ { "claim_revision_id": "catalog:0534351b1d72d52e78a7a3f84536db38f24993c910d8bcb0e3988751012295cf", "conditional_on": [], "credit_reason": "At epsilon1=epsilon2=epsilon3=1 and epsilon=3, printed (23) has a certain zero-threshold event. For the separate wrong-V_N display, choose any 0The printed claim is false: P022-T7; p. 868 (PDF p. 19), Proposition 2 on p. 863. The nested four-corner population satisfies the stated assumptions. The admissible split (1,1,1) makes (23) a probability-one event. On N=4r the exact moments give a scaled centering gap tending to one half; the centered error has the stated 51200/(N delta^2) tail bound. The wrongly centered absolute error tends to one half in probability, its small-threshold probabilities tend to one, and their N-scaled values diverge, refuting both finite printed bounds. The usual measure-theoretic conditions are assumed.

", "headline": "Proof of Proposition 2: equation (23) and the concluding display for V^NH\\hat V_N^HV^NH​, as printed", "location": "p. 868 (PDF p. 19); statement of Proposition 2 on p. 863 (PDF p. 14)" }, "id": "P022-T7", "text": { "body": "The printed claim is false: P022-T7; p. 868 (PDF p. 19), Proposition 2 on p. 863. The nested four-corner population satisfies the stated assumptions. The admissible split (1,1,1) makes (23) a probability-one event. On N=4r the exact moments give a scaled centering gap tending to one half; the centered error has the stated 51200/(N delta^2) tail bound. The wrongly centered absolute error tends to one half in probability, its small-threshold probabilities tend to one, and their N-scaled values diverge, refuting both finite printed bounds. The usual measure-theoretic conditions are assumed.", "headline": "Proof of Proposition 2: equation (23) and the concluding display for $\\hat V_N^H$, as printed", "location": "p. 868 (PDF p. 19); statement of Proposition 2 on p. 863 (PDF p. 14)" }, "url": "https://pmaronow.github.io/llm-audit/works/P022/#P022-T7" } ] }, "missing_sources": [], "reproduction": { "agreement": "agrees_in_part", "status": "selected_recalculation", "targets": [ { "agreement": "not_compared", "location": "p. 857 (PDF p. 8)", "method": "I could not regenerate the application estimates without the Harrison and Michelson data. The Blocked and Not compared labels refer to that data-based rerun. I checked the printed table for internal consistency; the separate efficiency row reports the comparison of its variance-reduction and additional-subject claims.", "status": "blocked", "target": "Table 1 and application estimates" }, { "agreement": "agrees_in_part", "location": "p. 857 (PDF p. 8)", "method": "I recalculated the application efficiency claims from Table 1. The printed variances and confidence intervals together bound the variance reduction between 6.045% and 6.305%, assuming Wald intervals with a common critical value, so it rounds to 6%, rather than the reported 7% (P022-N003). Under nominal inverse-sample-size scaling, the additional-subject bounds are about 100.4–105.1 and 75.3–78.5, supporting “over 100” and “over 75”. I could not regenerate the underlying estimates without the application data.", "status": "recalculated", "target": "Application efficiency claims" }, { "agreement": "agrees_in_part", "location": "p. 858 (PDF p. 9); hypotheses continued p. 859 (PDF p. 10)", "method": "I recalculated internal relationships from the printed numbers: the mean interval widths are compatible with the mean variance estimates, and the Heterogeneity A true variance and mean conventional estimate follow from the Sharp Null variance and the ten modified outcomes. The quoted Sharp Null bias of −0.007 USD² for the Neyman upper bound cannot be reconciled with Table 2 (P022-N005); the replacement figure requires the simulation output. The table relationships agree; partial agreement reflects the incompatible quoted bias figure. I could not rerun the simulations without the experiment data and simulation code.", "status": "recalculated", "target": "Table 2" }, { "agreement": "to_rounding", "location": "p. 870 (PDF p. 21), setup p. 869 (PDF p. 20)", "method": "I recalculated all 36 ratios by numerical integration of the reported Beta quantile functions, using new R code. Every ratio agrees to the printed two decimals; I did not rerun the original Mathematica calculation.", "status": "recalculated", "target": "Table 3" } ] }, "summary": "The paper studies the variance of the difference-in-means estimator in a randomized experiment on a finite population. That variance depends on the covariance of the potential outcomes, which the data do not identify. The paper derives sharp bounds on it from the two marginal distributions, a consistent estimator of the bounds, and narrower Wald intervals. 9 entries are classified as errors (1 Moderate, 8 Minor), and 2 are clarity notes. The Moderate error is that Lemma 2 fails under its stated conditions (\\fid{P022-E001}); the conclusions of Propositions 1 and 2 survive with replacement proofs.", "what_was_checked": "I checked the following points, which held up subject to the qualifications stated.\n\\begin{itemize}\n\\item Equation (2) for the variance of the difference in means, and its case (3), re-derived and confirmed by exact enumeration of all assignments in small designs.\n\\item The unbiasedness of the marginal variance estimators on p.~852, and equations (4), (5) and (6).\n\\item Lemma 1 and the sharpness of the interval $[V_N^L,V_N^H]$, confirmed by enumeration of all pairings of the outcomes in populations of up to seven units.\n\\item Section 3: the quantile formulas, the partition $P_{m,n}$, the reduction of the integrals to the sums (8), and (9).\n\\item The ordering $\\hat V_N^{b-}\\le\\hat V_N^L\\le\\hat V_N^H\\le\\hat V_N^{b+}\\le\\hat V_N^a$ for $n=N$.\n\\item Proposition 1 apart from its use of Lemma 2: the limiting coefficients and steps (ii)--(iv) of the proof, given the convergence of the sampled distributions. The conclusion holds by the argument in \\fid{P022-E001}.\n\\item Proposition 2 apart from the points in \\fid{P022-E001}, \\fid{P022-E006} and \\fid{P022-N007}: the covariance representation over $[-C,C]^2$, the decomposition into seven terms, and the bounds (17)--(22) and (24) with their constants.\n\\item Corollary 1, given a central limit theorem for finite populations: bounded third moments imply condition 4, $\\lim NV_N>0$ when $H$ is not concentrated on a line, and $\\gamma\\le 1$. Freedman (2008), Theorem 1 and the remarks in its Technical Appendix, was read and supports the citation.\n\\item Table 1 for internal consistency: $\\hat\\tau_N=0.715-1.397=-0.682$, the three intervals share a centre and are compatible with the critical value $1.96$ after rounding, $\\hat V_N^{b+}+\\hat V_N^{b-}=\\hat V_N^a$, and the statements ``over 100'' and ``over 75'' follow from the table.\n\\item Table 2 for internal consistency: each mean interval width is compatible with its mean variance estimate, and the true variance $0.238$ and mean conventional estimate $0.279$ under Heterogeneity A follow from the Sharp Null variance and the ten modified outcomes. The discrepancies in the accompanying paragraph are \\fid{P022-N005} and the missing hat in \\fid{P022-N006}.\n\\item Table 3: all 36 ratios recomputed by numerical integration agree with the printed values at two decimals, with a largest difference of $0.0049$.\n\\item Appendix B: for finite $N$, with the broken line joined (\\fid{P022-E004}), the function follows (8)--(9), including the index for the lower bound, up to the defect in \\fid{P022-E002}. The infinite-population case requires the branch in \\fid{P022-E003}.\n\\item The reference list: every entry and every citation in the text was read, and no numbered reference is missing.\n\\end{itemize}\nThe following limits apply to my checks. The estimates in Table 1 and the simulations in Table 2 could not be recomputed, because the data of Harrison and Michelson (2012) and the simulation code are not available. Table 3 was recomputed with new code and not with the Mathematica calculation of the paper. Of the cited sources, only Freedman (2008) was read in full; only the abstract of Robins (1988) was accessible. Equation (3) of Robins (1988) was not read, and the statement that the estimator reproduces Robins's estimator for dichotomous outcomes was not compared with its source. Neyman (1923), Cochran's Theorem 2.4, van der Vaart and Wellner, Lehmann (1966) and Tchen (1980) were not consulted. The Stata implementation mentioned on p.~851 was not available. I did not establish the correctness of the remaining claims.", "work": { "arxiv_id": "1405.6555", "audit_complete": true, "authors": [ "P. M. Aronow", "Donald P. Green", "Donald K. K. Lee" ], "category": "Articles - Theory and Methodology", "citation": "P. M. Aronow, Donald P. Green, and Donald K. K. Lee (2014). Sharp bounds on the variance in randomized experiments. Annals of Statistics 42(3):850–871.", "doi": "10.1214/13-aos1200", "edition": { "pages_in_audited_file": 22, "sha256": "f8ca8ce6216b377f87c0d5bea18a8d33cd116aafc33ae6e72d2e51a662dd170a", "source_url": "https://doi.org/10.1214/13-aos1200", "venue": "Annals of Statistics", "year": "2014" }, "id": "P022", "later_materials": null, "title": "Sharp bounds on the variance in randomized experiments", "type": "Journal article", "url": "https://pmaronow.github.io/llm-audit/works/P022/", "venue": "Annals of Statistics", "year": "2014" } }