{ "claims": [ { "claim": "The intervals obtained by inverting two-sided randomization tests over possible binary potential-outcome tables (the Rigdon-Hudgens intervals) contain the sample average treatment effect with probability at least $1-\\alpha$ for every sample size and allocation with $00}}$ such that $k_2\\hl{>}k_1$, and a function $f:[\\![k_1,k_2]\\!]\\to\\{0,1\\}$ such that $f(x)=0$ if $x\\le r$ and $f(x)=1$ if $x>r$, where $r\\in[\\![k_1-1,k_2]\\!]$ is unknown. The algorithm returns $r$ through the following steps.\n\\begin{enumerate}\n\\item Initialize $a\\leftarrow\\hl{k_1}$ and $b\\leftarrow\\hl{k_2}$.\n\\item Set $c\\leftarrow\\lfloor(a+b)/2\\rfloor$ and evaluate $f(c)$. If $f(c)=0$, set $a\\leftarrow c$. If $f(c)=1$, set $b\\leftarrow c$.\n\\item Repeat the previous step until $b=a+1$. In this case, return $a$ if $a>k_1$ and $br$, where $r\\in[\\![k_1-1,k_2]\\!]$ is unknown. The algorithm returns $r$ through the following steps.\n\\begin{enumerate}\n\\item Initialize $a\\leftarrow\\hl{k_1-1}$ and $b\\leftarrow\\hl{k_2+1}$. \\hl{The function is never evaluated at these two points.}\n\\item Set $c\\leftarrow\\lfloor(a+b)/2\\rfloor$ and evaluate $f(c)$. If $f(c)=0$, set $a\\leftarrow c$. If $f(c)=1$, set $b\\leftarrow c$.\n\\item Repeat the previous step until $b=a+1$. \\hl{Then return} $\\hl{a}$.\n\\end{enumerate}\nLemma B.2. Algorithm B.1 terminates \\hl{and returns} $\\hl{r}$ in at most $\\hl{\\lfloor\\log_2(k_2-k_1+1)\\rfloor+1}$ evaluations of $f$.''\n\nThe bound in Lemma B.2 is at most the printed bound $\\lfloor\\log_2(k_2-k_1+1)+2\\rfloor$. The search for the left boundary and the search in Algorithm 4.5 are corrected in the same way.", "evidence": "The evidence is an exact counterexample, computed in integer and rational arithmetic, and a derivation of the corrected search.\n\n\\begin{audittable}{llll}{Inputs on which the printed search fails. The third row is the upper search of Algorithm 4.3 for $n=16$, $m=8$, observed counts $\\mathbf n=(7,1,0,8)$ and $\\alpha=1/20$.}\nRange & Values of $f$ & Printed search & Required value $r$ \\\\\n$\\{1,2\\}$ & $f(1)=f(2)=0$ & returns 1 & 2 \\\\\n$\\{1,2\\}$ & $f(1)=f(2)=1$ & never returns & 0 \\\\\n$\\{14,15\\}$ & $f(14)=f(15)=0$ & returns 14 & 15 \\\\\n\\end{audittable}\n\nIn the third row, I have $T(\\mathbf n)=7/8$ and $n\\max\\mathcal C(\\mathbf n)=15$, and the range is $\\{14,15\\}$. The count vector $\\mathbf v=(0,15,0,1)$ is possible given $\\mathbf n$ and has $\\tau(\\mathbf v)=15/16$. Over the 12870 assignments, $T$ takes the values $7/8$ and $1$, each 6435 times, and both are at distance $1/16$ from $15/16$, which is also the observed distance. I obtain $p(\\mathbf v,\\mathbf n)=1$ and $U_\\alpha(\\mathbf n)=15/16$. Enumerating all 144 possible count vectors gives $\\mathcal I_\\alpha(\\mathbf n)=[1/2,15/16]$. The value $14/16=T(\\mathbf n)$ is accepted as well, through the table $(0,14,0,2)$, and I have $f(14)=f(15)=0$. The printed search evaluates $f(14)$ twice, never evaluates $f(15)$, and returns 14. The upper endpoint of Algorithm 4.3 as printed is then $14/16$. If $(0,15,0,1)$ is the true table, these are the observed counts at 6435 of the 12870 assignments, and at each of them the printed upper endpoint is below the true effect $15/16$.\n\nFor the corrected search (given under Corrected), let $N=k_2-k_1+1$. The inequalities $k_1-1\\le a\\le rn$, the steps are repeated for the next $\\tau_0\\in\\mathcal C(\\mathbf n)$. After the last $\\tau_0$, the algorithm returns $\\mathcal I_{\\alpha,K}$.", "evidence": "The evidence is a check of pp.~11--12 and a small exact example of the literal reading. Let $n=2$ and $m=1$, with observed counts $(0,1,0,1)$, the estimand $\\theta(\\mathbf v)=v_{10}+v_{01}$ and the constant statistic $R=0$, so that every test accepts. If the search stops at the first $j$ with a possible table, I obtain the tables $(0,0,1,1)$, $(0,0,0,2)$ and $(0,1,0,1)$, and the set $\\{0,1\\}$. The full loop also reaches $(0,1,1,0)$ and returns $\\{0,1,2\\}$. The table with one subject of type $(0,1)$ and one of type $(1,0)$ produces these data and has $\\theta=2$. The released code loops over every $j$ (\\texttt{for (int j=0; j}\\varepsilon$, so (B.76) gives the same conditional bound. Thus, with probability at least $1-(n+1)\\lfloor\\log_2(n+1)+3\\rfloor\\exp(-2K\\varepsilon^2)$, Algorithm 4.5 accepts no such table, and on this event $U_{\\alpha-2\\varepsilon}(\\mathbf Y,\\mathbf Z)\\ge U_{\\alpha,\\varepsilon,K}(\\mathbf Y,\\mathbf Z)$. Both passages use the binary search as corrected in \\fid{P003-E001}.", "evidence": "I base this finding on an exact counterexample, computed in rational arithmetic, and a derivation of the lemma by a different argument.\n\nLet $n=30$, $m=15$, $\\alpha=1/20$, $\\varepsilon=1/200$ and $K=1$, and let the observed counts be $\\mathbf n=(2,13,1,14)$. With $K=1$, an approximate test accepts a table with probability equal to its exact $p$-value. The upper search runs over the effects $x/30$ with $x\\in[\\![2,16]\\!]$, and $U_\\alpha=U_{\\alpha-2\\varepsilon}=10/30$.\n\n\\begin{audittable}{lll}{The exact upper search at level $\\alpha=1/20$ and one path of the approximate search, which has positive probability.}\nEffect tested & Exact search & Approximate search, one path \\\\\n$9/30$ & rejects 1 table, accepts $(2,9,0,19)$ & accepts $(1,9,0,20)$ \\\\\n$12/30$ & rejects 5 tables & rejects 5 tables \\\\\n$10/30$ & rejects 2 tables, accepts $(3,10,0,17)$ & rejects 7 tables \\\\\n$11/30$ & rejects 6 tables & not tested \\\\\nEndpoint returned & $10/30$ & $9/30$ \\\\\n\\end{audittable}\n\nAt $9/30$ the exact search rejects $(1,9,0,20)$, whose $p$-value is $412/10005<1/20$, and accepts $(2,9,0,19)$, whose $p$-value is $752/10005$. On the path in the table, the approximate search accepts $(1,9,0,20)$, so $J=1$ and $b_J=0$. It then rejects every table that it tests at $12/30$ and at $10/30$, including $(3,10,0,17)$, which the exact test accepts with $p$-value $7143/126730$. It returns $9/30U_{\\alpha-2\\varepsilon}$. The event (B.85) holds, and the event $\\{U_{\\alpha-2\\varepsilon}\\ge U_{\\alpha,\\varepsilon,K}\\}$ does not. Summing over the whole search tree gives the probabilities below.\n\n\\begin{audittable}{lr}{Probabilities given the observed data in the same example, rounded to four places. The proof equates the first with the second, and the third with the fourth.}\nEvent & Probability \\\\\n$U_{\\alpha,\\varepsilon,K}U_{\\alpha-2\\varepsilon}$ & 0.0300 \\\\\n$b_J=0$ and $J\\le N$, with $\\mathbf a$ at level $\\alpha-2\\varepsilon$ & 0.0038 \\\\\n\\end{audittable}\n\nFor (B.81), I keep the data and $\\alpha$, and set $\\varepsilon=1/50$ and $K=5000$. Then $N=16$ and $a_2=0$, and an approximate test accepts when $S\\ge\\alpha-\\varepsilon=3/100$. The $p$-value $412/10005$ of $(1,9,0,20)$ is above this threshold. If the first approximate test accepts this table, the second approximate test is the test of $(1,12,0,17)$ at $12/30$, whose $p$-value $407/190095$ is below the threshold. Applying Chebyshev's inequality to each of the two tests, I find that $\\mathbb P(b_2=1\\text{ and }a_2=0)>9/10$, while $N\\exp(-2K\\varepsilon^2)=16e^{-4}<48/103$.\n\nFor the replacement argument, fix the observed data, and set $D=(n+1)\\lfloor\\log_2(n+1)+3\\rfloor$ and $\\delta=\\exp(-2K\\varepsilon^2)$. With the corrected search, the upper search performs at most $D$ table tests, whatever their outcomes. I number the tests that the approximate search performs. The table $\\mathbf W_i$ tested at step $i$ is determined by the outcomes of the earlier tests, and the draws used at step $i$ are independent of them. Conditional on the earlier tests, (B.75) implies that a table with $p(\\mathbf W_i,\\mathbf n)\\ge\\alpha$ is rejected with probability at most $\\delta$, and (B.76) implies that a table with $p(\\mathbf W_i,\\mathbf n)<\\alpha-2\\varepsilon$ is accepted with probability at most $\\delta$. By a union bound over at most $D$ steps, each of the following events has probability at least $1-D\\delta$: no table with $p$-value at least $\\alpha$ is rejected, and no table with $p$-value below $\\alpha-2\\varepsilon$ is accepted.\n\nOn the first event, every value $x/n\\le U_\\alpha$ in the range is declared compatible when it is tested. Indeed, by the proof of Theorem 4.1, the list of tables tested for such a value contains a table with $p$-value at least $\\alpha$, and the approximate search accepts either an earlier table of the list or that one. The upper pointer $b$ of the search then stays above $nU_\\alpha$, and the search returns $U_{\\alpha,\\varepsilon,K}\\ge U_\\alpha$. On the second event, every table that the approximate search accepts has $p$-value at least $\\alpha-2\\varepsilon$, and every value it declares compatible is at most $U_{\\alpha-2\\varepsilon}$. The lower pointer $a$ then stays at or below $nU_{\\alpha-2\\varepsilon}$, and the search returns $U_{\\alpha,\\varepsilon,K}\\le U_{\\alpha-2\\varepsilon}$. The lower endpoints are treated in the same way after every outcome is replaced by its complement, which changes the signs of $T$ and $\\tau$ and preserves the $p$-values. By a union bound over the two endpoints, each part of Lemma B.15 holds with probability at least $1-2D\\delta$.", "problem": "In Algorithm 4.3, the compatibility of one value $\\tau_0$ is decided by testing several tables in turn and stopping at the first acceptance. Algorithm 4.5 does the same with a fresh set of Monte Carlo draws for each table. The sequences $\\mathbf a$ and $\\mathbf b$ record the decisions on single tables, and $J$ is the first index at which they differ. The proof asserts that the direction of this first disagreement decides the order of the final upper endpoints. It does not. Suppose that $b_J=0$, that is, the approximate test accepts a table that the exact test rejects. The exact search may accept the next table for the same $\\tau_0$. Both searches then declare $\\tau_0$ compatible and continue from the same midpoint, and a later rejection by the approximate test can still produce $U_{\\alpha,\\varepsilon,K}0$. Without this convention, the printed interval is undefined at this input. This example does not refute (B.79) or either bound for the corrected search.", "what_survives": "Lemma B.15 and Proposition 4.6 hold as stated, with the same constants, when the search is the corrected one and the intervals in their statements are defined (the second part of the lemma needs $\\alpha-2\\varepsilon>0$). This proof error does not affect the statements of Lemma B.14, Lemma 4.2 or Theorem 4.1, or the exact test. The simulations and the application do not use the false identities, and no reported number is shown to be affected." }, "url": "https://pmaronow.github.io/llm-audit/works/P003/#P003-E003", "work": { "arxiv_id": "2305.09906", "doi": "10.1016/j.jeconom.2025.106056", "title": "Fast computation of exact confidence intervals for randomized experiments with binary outcomes", "url": "https://pmaronow.github.io/llm-audit/works/P003/" }, "work_id": "P003" }, { "before_review": null, "class": "error", "date_modified": "2026-10-25", "date_published": "2026-10-25", "evidence_type": "exact_counterexample", "exhibits": [ { "alt": "Crop of the second half of the proof of Proposition 6.2, with displays (D.3) to (D.7). Two red boxes: display (D.3), 'J alpha(M, Z) is a subset of I circle alpha(M, Z)', and the final 'less than alpha' at the end of display (D.7), which bounds a probability involving T(Y(+), Z). The last line reads 'so w is incompatible with (Y prime, Z), as desired.'", "caption": "Appendix D, proof of Proposition 6.2, p. 29 (PDF p. 29), from (D.3) to the end. The marked passages are the inclusion (D.3), which fails in the example below, and the last inequality of (D.7), which does not follow from (D.5) when $\\mathbf w$ is not possible given $(\\mathbf Y^{(+)},\\mathbf Z)$.", "url": "https://pmaronow.github.io/llm-audit/works/P003/exhibits/P003-E004.png" }, { "alt": "Crop of the end of Definition 6.1, Proposition 6.2 and Remark 6.3. Two red boxes: display (6.2), which defines I circle alpha(M, Z) as the interval from min(L alpha(Y(-), Z), T(Y(-), Z)) to max(U alpha(Y(+), Z), T(Y(+), Z)); and the words 'Suppose m less than n in the positive integers,' at the start of Proposition 6.2, whose display (6.3) states that the interval contains tau(y) with probability at least 1 - alpha.", "caption": "Definition 6.1, Proposition 6.2 and Remark 6.3, p. 14 (PDF p. 14). The marked passages are the interval (6.2), built from the two extremal imputations, and the hypothesis of Proposition 6.2, which allows every $m