Assumptions and verification
Claim revision: catalog:ce91178c3af0acea1d5bccc45e2f122d0d4b21e4d315b4dd8e49077fee03b5e3
Standing assumptions: The work is a three-page comment (Statistical Science 32(3), 2017, pp. 346–348; PDF pp. 1–3) on Ding (2017), ‘A paradox from randomization-based causal inference’. It has no numbered theorem, proposition, lemma or equation, and no proof, table, figure or appendix; its mathematics is in unnumbered displays and sentences of Sections 2–4. All three page images were read, and everything quoted below was transcribed from them.
(S1) Scope. Introduction, p. 346 (PDF p. 1): ‘Note, we consider only asymptotic results, and do not discuss the finite $N$ differences between exact tests and tests that are only known to be asymptotically valid. Finite $N$ differences may account in part for Ding’s (2017) simulation results; while perhaps of practical importance, such differences are in our view theoretically uninteresting as the source of a potential paradox.’
(S2) Design and regularity conditions. Section 2, p. 347 (PDF p. 2), in brackets: ‘[We follow Ding’s, 2017 notation throughout. We will also assume complete random assignment with the asymptotic scaling and regularity conditions of Freedman, 2008, primarily bounded fourth (cross) moments on the limit distribution of potential outcomes with $\lim_{N\to\infty}N_1/N=p$, where $0<p<1$. These regularity conditions appear to be acknowledged in the final paragraph of Ding, 2017.]’
(S3) Notation that the comment prints. p. 346 (PDF p. 1): ‘Fisher’s randomization test (FRT)’; ‘the FRT (with the difference in means as a test statistic)’; ‘the null of zero average causal effect (Neyman’s null)’; ‘the sharp null of zero individual causal effects (Fisher’s null)’; the Wald-type analogue is ‘a Wald test assuming a homoskedastic linear model’ and is ‘equivalent to a pooled-variance two-sample $z$-test’. p. 347 (PDF p. 2): ‘the null hypothesis, $\tau=\overline{Y}_1-\overline{Y}_0=0$’; ‘using as a working assumption $\tau=\widehat{\tau}$’; ‘the variance of the FRT reference distribution, $\widehat{V}(\text{Fisher})$’; ‘the variance estimator associated with the Wald-type analogue’, referred to as $\widehat{V}(\text{OLS})$; ‘one of Neyman’s variance estimators, $\widehat{V}(\text{Neyman})$’; $V(\widehat{\tau})$; ‘where $p=\lim_{N\to\infty}N_1/N$’. p. 348 (PDF p. 3): ‘Fisher’s null can be viewed as a joint hypothesis that implies two features of the underlying distribution of potential outcomes: $\tau=0$ and $S_1=S_0$. Neyman’s null implies no such assumption about $S_1=S_0$.’ The comment prints no definition of $N_1$, $N_0$, $S_1$, $S_0$, $\widehat{\tau}$, $V(\widehat{\tau})$ or of the three variance estimators: it takes them from Ding (2017), which is not staged.
(S4) The audit's reading of that notation (used by both fresh readers and by the validators; not printed in the comment). For each $N$ there is a finite population of units $i=1,\ldots,N$ with fixed real potential outcomes $Y_i(1)$ and $Y_i(0)$. $\overline{Y}_1=N^{-1}\sum_iY_i(1)$, $\overline{Y}_0=N^{-1}\sum_iY_i(0)$, $\tau_i=Y_i(1)-Y_i(0)$, $\tau=\overline{Y}_1-\overline{Y}_0$. $S_1^2=(N-1)^{-1}\sum_i(Y_i(1)-\overline{Y}_1)^2$, $S_0^2=(N-1)^{-1}\sum_i(Y_i(0)-\overline{Y}_0)^2$, $S_\tau^2=(N-1)^{-1}\sum_i(\tau_i-\tau)^2$; $S_1\ge0$ and $S_0\ge0$ are the square roots. Complete random assignment: $Z=(Z_1,\ldots,Z_N)$ is uniformly distributed on $\Omega_N=\{z\in\{0,1\}^N:\sum_iz_i=N_1\}$, with $N_0=N-N_1$; this is the only randomness, and $E$, $V$ and probabilities refer to it. Observed outcome $Y_i^{\mathrm{obs}}=Z_iY_i(1)+(1-Z_i)Y_i(0)$. Arm means $\overline{Y}_1^{\mathrm{obs}}=N_1^{-1}\sum_iZ_iY_i^{\mathrm{obs}}$ and $\overline{Y}_0^{\mathrm{obs}}=N_0^{-1}\sum_i(1-Z_i)Y_i^{\mathrm{obs}}$; the difference in means is $\widehat{\tau}=\overline{Y}_1^{\mathrm{obs}}-\overline{Y}_0^{\mathrm{obs}}$, and $V(\widehat{\tau})$ is its variance. Arm sample variances $s_1^2=(N_1-1)^{-1}\sum_iZ_i(Y_i^{\mathrm{obs}}-\overline{Y}_1^{\mathrm{obs}})^2$ and $s_0^2=(N_0-1)^{-1}\sum_i(1-Z_i)(Y_i^{\mathrm{obs}}-\overline{Y}_0^{\mathrm{obs}})^2$ (so $N_1\ge2$ and $N_0\ge2$ wherever they appear). $\widehat{V}(\text{Neyman})=s_1^2/N_1+s_0^2/N_0$. $SSR=(N_1-1)s_1^2+(N_0-1)s_0^2$ is the residual sum of squares of the least-squares regression of $Y^{\mathrm{obs}}$ on an intercept and $Z$, whose coefficient on $Z$ is $\widehat{\tau}$; $\widehat{V}(\text{OLS})=\frac{SSR}{N-2}\left(\frac{1}{N_1}+\frac{1}{N_0}\right)=\frac{N\,SSR}{N_1N_0(N-2)}$ is the classical homoskedastic variance estimate of that coefficient (the pooled-variance two-sample form). $s^2=(N-1)^{-1}\sum_i(Y_i^{\mathrm{obs}}-\overline{Y}^{\mathrm{obs}})^2$, with $\overline{Y}^{\mathrm{obs}}=N^{-1}\sum_iY_i^{\mathrm{obs}}$, is the variance of the combined observed sample. The FRT reference distribution is the distribution of $D(Z')=N_1^{-1}\sum_iZ'_iY_i^{\mathrm{obs}}-N_0^{-1}\sum_i(1-Z'_i)Y_i^{\mathrm{obs}}$ when the observed outcomes are held fixed and $Z'$ is uniform on $\Omega_N$; $\widehat{V}(\text{Fisher})$ is its variance, which equals $\frac{N\,s^2}{N_1N_0}$. Fisher's null: $Y_i(1)=Y_i(0)$ for every $i$. Neyman's null: $\tau=0$. Constant effects: $Y_i(1)-Y_i(0)=\tau$ for every $i$. The divisors ($N-1$ in $S_1^2$, $S_0^2$, $S_\tau^2$ and $s^2$; $N-2$ in $\widehat{V}(\text{OLS})$) are the fresh readers' conventions: the exact finite lemmas listed with the targets depend on them, the asymptotic statements do not.
(S5) The audit's reading of the asymptotic statements (the comment prints only S2). A sequence of finite populations indexed by $N\to\infty$, the potential outcomes and $N_1$ depending on $N$, with $N_1/N\to p$ and $0<p<1$; $S_1^2\to\sigma_1^2$, $S_0^2\to\sigma_0^2$, $S_\tau^2\to\sigma_\tau^2$ and $\tau=\tau_N\to\tau_\infty$, all finite; and $\sup_NN^{-1}\sum_i\left(Y_i(1)^4+Y_i(0)^4\right)<\infty$. For random variables $X_N$ defined on the randomization distributions, $X_N=o_p(N^{-1})$ means that $NX_N\to0$ in probability, and $\operatorname{plim}$ is the limit in probability. In its limit statements the comment writes $S_1$, $S_0$ for the limits $\sigma_1$, $\sigma_0$ (for example ‘$\lim_{N\to\infty}NV(\widehat{\tau})\le S_1^2/p+S_0^2/(1-p)$’, p. 347). Freedman (2008) and Ding (2017) are not staged, so the exact form of their conditions was not checked against the originals.
Effective claim (source TeX): As printed (p. 348): ‘Here, we state a weaker claim verifiable at full generality, which has been known since Copas [(1973), equation (9)]: for completely randomized experiments, if outcomes are binary, then under the sharp null hypothesis of no treatment effect, all test statistics may be written as a function of the difference-in-means statistic. Stronger statements are likely available.’ As printed (p. 346, item (iii)): ‘We also note an error in Ding’s (2017) Theorem 7 and suggest a refinement that is correct at full generality.’
Corrected wording (Sol's smallest repair, validated.json; doubt artifact, section 8). Section 4: ‘For completely randomized experiments, if outcomes are binary, then under the sharp null hypothesis of no treatment effect, all test statistics based on the two-by-two treatment-by-outcome table may be written as a function of the difference-in-means statistic.’ Introduction, item (iii): append ‘for statistics based on the two-by-two treatment-by-outcome table’ to ‘a refinement that is correct at full generality’.
Formal content, with every hypothesis. Let $N_1\ge1$ and $N_0\ge1$ be integers and $N=N_1+N_0$. Let $y=(y_1,\ldots,y_N)\in\{0,1\}^N$ be fixed: the outcomes are binary and, under the sharp null $Y_i(1)=Y_i(0)=y_i$ for every $i$, the observed outcome of unit $i$ is $y_i$ under every assignment. Let $M=\sum_iy_i$. Let $\Omega=\{z\in\{0,1\}^N:\sum_iz_i=N_1\}$ be the set of assignments of a completely randomized experiment (they are equiprobable, but no probability enters the statement). For $z\in\Omega$ let $a(z)=\sum_iz_iy_i$ be the number of treated successes, let $C(z)=\big(a(z),\ N_1-a(z),\ M-a(z),\ N_0-M+a(z)\big)$ be the two-by-two table (treated successes, treated failures, control successes, control failures), and let $\widehat{\tau}(z)=\frac{a(z)}{N_1}-\frac{M-a(z)}{N_0}$ be the difference in means. Then:
(a) for every $z\in\Omega$: $\widehat{\tau}(z)=\frac{N\,a(z)-N_1M}{N_1N_0}$ and $a(z)=\frac{N_1N_0\,\widehat{\tau}(z)+N_1M}{N_1+N_0}$;
(b) for all $z,z'\in\Omega$: $\widehat{\tau}(z)=\widehat{\tau}(z')$ if and only if $a(z)=a(z')$, if and only if $C(z)=C(z')$;
(c) [the corrected claim] for every set $\mathcal{S}$ and every function $h:\mathbb{Z}^4\to\mathcal{S}$ there is a function $g:\mathbb{R}\to\mathcal{S}$ such that $h(C(z))=g(\widehat{\tau}(z))$ for every $z\in\Omega$. The function $g$ does not depend on $z$; it can be chosen to depend on $y$ only through $M$, namely $g(t)=h\big(A(t),\ N_1-A(t),\ M-A(t),\ N_0-M+A(t)\big)$ with $A(t)=\frac{N_1N_0\,t+N_1M}{N}$ at the values $t$ that $\widehat{\tau}$ takes on $\Omega$, and arbitrary elsewhere. No monotonicity of $h$ is assumed, so the statement covers statistics such as $|\widehat{\tau}|$;
(d) [statistics of the two outcome samples are table statistics] let $F:\{0,1\}^{N_1}\times\{0,1\}^{N_0}\to\mathcal{S}$ satisfy $F(x_1\circ\pi_1,\ x_0\circ\pi_0)=F(x_1,x_0)$ for all permutations $\pi_1$ of $\{1,\ldots,N_1\}$ and $\pi_0$ of $\{1,\ldots,N_0\}$. For $z\in\Omega$ let $x_1(z)\in\{0,1\}^{N_1}$ list the outcomes $y_i$ of the treated units ($z_i=1$) in increasing order of $i$, and $x_0(z)\in\{0,1\}^{N_0}$ those of the control units. Then $F(x_1(z),x_0(z))=h_F(C(z))$ for every $z\in\Omega$, where $h_F(a,\cdot,b,\cdot)=F\big(c_{N_1}(a),c_{N_0}(b)\big)$ and $c_n(k)=(1,\ldots,1,0,\ldots,0)$ has $k$ ones followed by $n-k$ zeros; hence, by (c), $F(x_1(z),x_0(z))=g(\widehat{\tau}(z))$ on $\Omega$ for some $g$. Conversely, for every $h$ the map $(x_1,x_0)\mapsto h\big(|x_1|,\ N_1-|x_1|,\ |x_0|,\ N_0-|x_0|\big)$, where $|x|$ is the number of ones, has this invariance;
(e) [exact criterion] for an arbitrary function $T:\Omega\to\mathcal{S}$: there is $g$ with $T(z)=g(\widehat{\tau}(z))$ for every $z\in\Omega$ if and only if $T(z)=T(z')$ whenever $C(z)=C(z')$.
{S : Type*} : SAP.P053.T1Statement S
Theorem: SAP.P053.T1
Statement hash: 4cb31dac30722b86292bc0d0f1ecfdebf066e0f1d19a8a93daac740f6eb8ee16
Statement definition: SAP.P053.T1Statement
Certificate. 20261104T195825Z
Declaration review: faithful; 2026-11-04T20:18:17Z; binding v2
Evidence scope: Certification checks formal proof objects. Statement fidelity is a model judgment. Author review is separate.
Proof dependencies: Independently extracted
Manifest annotation reconciliation: Effective use follows the source-bound dependency observation; original annotations are retained in the crosswalk.
AronowAudit.P053.binary_equal_iff: observed used
AronowAudit.P053.binary_reconstruction: observed used
AronowAudit.P053.canonical_permutation: observed used
AronowAudit.P053.ordered_sample_successes: observed used
AronowAudit.P053.successes_compl: observed used
AronowAudit.P053.successes_permutation: observed used
AronowAudit.P053.symmetric_binary_samples: observed used
AronowAudit.P053.table_statistic_factors: observed used
Source and evidence metadata · sap-2026-11-08-assured-release
![Crop of four lines at the foot of the left column of page 348. They read: ference in means (e.g., the absolute value of tau-hat). For such test statistics, the resulting inferences may substantively differ. Here, we state a weaker claim verifiable at full generality, which has been known since Copas [(1973), equation (9)]: A red outline surrounds the words at full generality and the comma after them.](exhibits/P053-E001-b.png)
