Corrections to: The Regression Discontinuity Design under Interference: A Local Randomization-Based Approach

The problem

The first display defines $\overline{Y}_i(z)$ as an average of $y_i(\mathbf{z})$ over the distinct assignment vectors in $\operatorname{Supp}(\mathbf{Z})$ whose $i$th component is $z$. Each such vector counts once, whatever its probability. The expectation of the difference-in-means estimator weights each assignment vector by its probability. Under local randomization, by linearity of expectation, I have

This equals the printed target for all potential outcomes only if, for each subject $i$ in the window and each $z$, the conditional distribution of $\mathbf{Z}$ given $Z_i=z$ is uniform on its support. Local randomization does not imply this. Under it the arrangements of the running variable in the window are equiprobable, so that the subjects in the window have equal probabilities of treatment. However, the allocation outside the window is “left completely unspecified”, and the number of arrangements that produce a particular assignment vector of the whole population can vary from one vector to another. The distinct assignment vectors can then have unequal probabilities, so the identity $\mathrm{E}[\hat\tau]=\tau$ need not hold for the printed $\tau$ and cannot be justified by linearity of expectations alone.

The prose around the displays describes the probability-weighted quantity: the target is “the average outcome we would expect to see in treated subjects (across all subjects within the window and across all randomizations)” minus the same for control subjects. The display can disagree with the prose when assignment vectors have unequal probabilities, and the text does not settle whether the probabilities were omitted from the display or an assumption of uniform assignment was intended. To keep the allocation outside the window unrestricted, the correction inserts $\Pr(\mathbf{Z}=\mathbf{z})$ in both sums of the first display. The estimator is unchanged and the estimand changes.

Evidence an exact counterexample

The evidence is two exact counterexamples to the printed identity, and a derivation of the corrected one.

The first example. Let $N=4$, $b=2$ and $W_0=[-2,2]$. Let $(X_1,X_2,X_3)$ be a uniformly random arrangement of the values $-2$, $-1$ and $1$, and set $X_4=3$ if $(X_1,X_2,X_3)=(-2,-1,1)$ and $X_4=-3$ otherwise. The vector $\mathbf{X}$ has six support points, each of probability $1/6=1/N_0!$. Subjects 1, 2 and 3 are always in the window and subject 4 never is, the values in the window are distinct and the same in every case, and exactly one subject in the window is treated. Every condition printed on p. 130 holds, with the no-ties condition read for distinct subjects (P014-E003), and in addition all six arrangements are equiprobable. Each subject in the window is treated with probability $1/3$. For all sixteen assignment vectors $\mathbf{z}$, set $y_1(\mathbf{z})=y_2(\mathbf{z})=y_3(\mathbf{z})=z_4$ and $y_4(\mathbf{z})=0$.

The three subjects in the window always have the same outcome, so that $\hat\tau=0$ for every assignment and $\mathrm{E}[\hat\tau]=0$. The printed $\overline{Y}_i(z)$ counts each row of the table once. For subject 1 under control, the rows with $z_1=0$ are the first three, with outcomes $0$, $1$ and $0$, and the printed average is $1/3$. Given $Z_1=0$, these rows have probabilities $1/4$, $1/4$ and $1/2$, and the conditional expectation is $1/4$.

From the second and third columns, I obtain the printed target, and from the last two the probability-weighted target:

The printed identity fails by $1/18$.

The second example. Let $N=4$, $b=3$ and $W_0=[-3,3]$. Let $(X_1,X_2,X_3)$ be a uniformly random arrangement of $-1$, $1$ and $2$, so that two subjects in the window are treated, and set $X_4=4$ if $(X_1,X_2,X_3)=(-1,1,2)$ and $X_4=-4$ otherwise. Set $y_2(\mathbf{z})=12z_2z_4$ and all other potential outcomes equal to zero. The same conditions hold. The assignment vectors $(0,1,1,0)$ and $(0,1,1,1)$ have probability $1/6$ each, and $(1,0,1,0)$ and $(1,1,0,0)$ have probability $1/3$ each. The estimator equals $6$ at $(0,1,1,1)$ and $0$ elsewhere, so that $\mathrm{E}[\hat\tau]=1$. The printed $\overline{Y}_2(1)$ is $(0+12+0)/3=4$, and I find $\tau=4/3$. The conditional expectation $\mathrm{E}[y_2(\mathbf{Z})\mid Z_2=1]$ is $3$, and the probability-weighted target is $1$.

The corrected identity. Let $S$ be the set of subjects in the window and $n=N_0$. Suppose that $\sum_{i\in S}Z_i=m$ with probability one, where $0<m<n$, and that $\Pr(Z_i=1)=m/n$ for every $i\in S$. Both hold when the $n$ distinct values in the window, of which $m$ are nonnegative, are arranged uniformly at random: a subject receives each value in $(n-1)!$ of the $n!$ arrangements. I write $p(\mathbf{z})=\Pr(\mathbf{Z}=\mathbf{z})$ and

Since the numbers of treated and control subjects in the window are fixed, I may write

For each $i\in S$, by the definition of $\overline{Y}_i(a)$ and the identity $\Pr(Z_i=1)=m/n$, I have

Substituting these into the expectation of the previous display, I obtain

Note that this derivation uses no assumption on the allocation outside the window. In both examples the difference between $\mathrm{E}[\hat\tau]$ and the probability-weighted target vanishes identically as a linear form in the 48 potential outcomes of the three subjects in the window, and the difference between $\mathrm{E}[\hat\tau]$ and the printed target does not.

When the printed target is correct. If $\mathbf{Z}$ is uniform on $\operatorname{Supp}(\mathbf{Z})$, then the two definitions of $\overline{Y}_i(z)$ agree. This is the case when the assignments outside the window are fixed, since each assignment vector is then produced by exactly $m!(n-m)!$ arrangements. With $X_4$ fixed in either example, the printed identity holds for all potential outcomes. The two definitions also agree when there is no interference, since $y_i(\mathbf{z})$ then depends on $z_i$ alone.

What survives

The difference-in-means estimator, the potential-outcome model with arbitrary interference and the local randomization design are unchanged. The estimator is exactly unbiased for the probability-weighted target with no restriction on the allocation outside the window. This implies that the statement of the Introduction, that it is unbiased for a regime-specific causal effect, holds for that target. The printed target and identity hold as printed when $\mathbf{Z}$ is uniform on its support, and when there is no interference. The definition of the estimand changes; in the first example its value changes from $-1/18$ to $0$. The paper reports no simulations, application or numerical results that depend on the definition. The paragraph on inference on pp. 131–132 refers to the same target, and its sufficient conditions were not examined for this correction.

The problem

Under the marked clause, every support point of the full vector $\mathbf{X}=(X_1,\ldots,X_N)$ has probability $1/N_0!$. The sentence then says that its clauses imply that all permutations of the running variable values within the window are equiprobable. They do not. I define an arrangement to be an assignment of the $N_0$ values in the window to the $N_0$ subjects in the window. Two support points of $\mathbf{X}$ can have the same arrangement and differ outside the window. The clauses fix the number of support points at $N_0!$, but they do not fix how many support points have a given arrangement. Some arrangements can have probability zero and others probability above $1/N_0!$, and the subjects in the window can then have unequal probabilities of treatment.

The clause is also too strong once the conclusion drawn from it is granted. If the $N_0!$ arrangements are equiprobable and every support point of $\mathbf{X}$ has probability $1/N_0!$, then the running variables outside the window are a function of the arrangement. An allocation outside the window that remains random given the arrangement is excluded, although p. 131 says that this allocation is “left completely unspecified”. The words after the colon state the intended assumption, which is a uniformly random arrangement, and the correction states that assumption directly.

Evidence an exact counterexample

The evidence is an exact counterexample to the implication, together with a counting argument for the corrected condition.

The counterexample. Let $N=4$, $b=2$ and $W_0=[-2,2]$, and assign probability $1/6$ to each of the six vectors

Subjects 1, 2 and 3 are in the window with probability one and subject 4 is never in it, so that $N_0=3$. In each vector the values in the window are $-2$, $-1$ and $1$. They are distinct, and the empirical distribution function $F_{\mathbf{x},W_0}$ is the same for all six. Each support point has probability $1/6=1/N_0!$, and exactly one subject in the window is treated. Every printed clause holds. However, the arrangement $(-1,-2,1)$ has probability $2/3$, the arrangements $(1,-1,-2)$ and $(-1,1,-2)$ have probability $1/6$ each, and the other three have probability zero. Subjects 1, 2 and 3 are treated with probabilities $1/6$, $1/6$ and $2/3$. Let $y_1(\mathbf{z})=z_1$, and let all other potential outcomes be zero, so that there is no interference. Then $\tau=1/3$, while $\hat\tau=Z_1$ and $\mathrm{E}[\hat\tau]=1/6$.

The counting argument. Let $S$ be the set of subjects in the window, write $n=N_0$, and let $\mathbf{X}_S=(X_i)_{i\in S}$. For an arrangement $a$, let $k(a)$ be the number of support points $\mathbf{x}$ of $\mathbf{X}$ such that $\mathbf{x}_S=a$. Under the printed clauses, I have $\Pr(\mathbf{X}_S=a)=k(a)/n!$, and nothing determines $k(a)$. If in addition $\Pr(\mathbf{X}_S=a)=1/n!$ for every $a$, then $k(a)=1$ for every $a$, so that the values outside the window are a function of $\mathbf{X}_S$. For instance, let the arrangement be uniform over the six arrangements of $-2$, $-1$ and $1$, and let $X_4$ be independent of it, equal to $-3$ with probability $1/3$ and to $3$ with probability $2/3$. Then $\mathbf{X}$ has twelve support points, with probabilities $1/18$ and $1/9$, and none has probability $1/6$. Under the corrected condition, let $m$ be the number of nonnegative values in the window. A subject $i\in S$ receives a nonnegative value in $m(n-1)!$ of the $n!$ arrangements, which implies that $\Pr(Z_i=1)=m/n$.

What survives

The potential-outcome model, the threshold rule that sets $Z_i=1$ when $X_i\ge 0$, the fixed set of subjects in the window and the difference-in-means estimator are unchanged. The corrected condition is the assumption that the prose of the sentence already states, and it changes neither the estimator nor the intended estimand. Under it every subject in the window is treated with probability $m/n$, and without interference the unbiasedness argument on p. 131 holds as printed. This correction does not repair the definition of the target under interference, which is the subject of P014-E001.

The problem

The quantifier ranges over all pairs $i,j$ of subjects in the window, including $j=i$. Since $\Pr(X_i=X_i)=1$, the condition cannot hold as printed. The words that follow it, “no two observed age values”, state the intended condition, which concerns distinct subjects.

Evidence a derivation

The evidence is a check of the page and a one-line derivation. No restriction $i\neq j$ is printed. The window contains at least one subject, since the same paragraph assumes $0<\sum_{i:X_i\in W_0}Z_i<N_0$. For such a subject $i$, the printed condition with $j=i$ requires $\Pr(X_i=X_i)=0$, while the event $\{X_i=X_i\}$ has probability one.

What survives

Adding $i\neq j$ expresses the intended no-ties condition without changing the intended local randomization design. I did not examine whether “without loss of generality” is justified for every model of the running variable.

The problem

The surname in the 2016 entry is spelled “Vasquez-Bare”, while the text on pp. 129 and 132 has “Vazquez-Bare”, which is the spelling of the journal’s byline. The Rosenbaum entry prints “American” twice. Three titles end with a closing quotation mark that has no opening mark. The Neyman entry runs two citations of one item together. Its title is printed twice, the date appears as “[1923.] 1990”, and “erzy Splawa”, a fragment of the author’s name Jerzy Splawa-Neyman, stands among the translators. The page range 465–480 of that entry is not an error: it is the range on the masthead of the translation, which covers the translation (465–472) and the discussion that follows it.

Evidence a check of the printed page

The evidence is a check of the pages against public sources. The article page of The Stata Journal names the third author as Gonzalo Vazquez-Bare. The title page of the 1990 translation in Statistical Science names the author as Jerzy Splawa-Neyman, carries the credit “Translated and edited by D. M. Dabrowska and T. P. Speed”, and has the masthead “1990, Vol. 5, No. 4, 465-480”. The contents of that issue list the comment by D. B. Rubin separately at pp. 472–480.

What survives

Every cited work remains identifiable, and no mathematical content or result of the paper is affected.

The problem

Hudgens and Halloran (2008) define the individual average potential outcome as a conditional expectation. The outcomes of an individual are averaged over the assignments of the other individuals, each weighted by its conditional probability given the individual’s own treatment. They define the individual direct causal effect as the average under control minus the average under treatment. The target $\tau$ differs from the average of that effect over the subjects in the window in two ways. First, the printed $\overline{Y}_i(z)$ weights the distinct assignment vectors equally and is not their conditional expectation in general; this is the error corrected in P014-E001. Second, $\tau$ is treated minus control. Once the probabilities are inserted, $\tau$ is therefore the negative of their average direct effect for the subjects in the window. The sign is a convention. The paper uses treated minus control in both $\tau$ and $\hat\tau$, and no result on estimation depends on it.

Evidence an exact counterexample

The evidence is a reading of the cited paper and an exact example. In the public author manuscript of Hudgens and Halloran (2008), the definition in Section 3.1 multiplies each potential outcome by $\Pr_\psi(\mathbf{Z}_{i(j)}=\omega\mid Z_{ij}=z)$ and is followed by the statement that the “individual average potential outcome is the conditional expectation”. In Section 3.2, equation (3) defines the individual direct causal effect as $\overline{Y}_{ij}(0;\psi)-\overline{Y}_{ij}(1;\psi)$. I did not check the page and equation numbers of the version of record.

For the example, let $N=4$, $b=1$ and $W_0=[-1,1]$. Let $(X_1,X_2,X_3)$ be a uniformly random arrangement of $-1$, $-1/2$ and $1/2$, and set $X_4=2$ if $(X_1,X_2,X_3)=(-1,-1/2,1/2)$ and $X_4=-2$ otherwise. Set $y_1(\mathbf{z})=z_4$ and all other potential outcomes equal to zero. The conditions printed on p. 130 hold and the six arrangements are equiprobable. The assignment vectors $(0,0,1,0)$ and $(0,0,1,1)$ have probability $1/6$ each, and $(0,1,0,0)$ and $(1,0,0,0)$ have probability $1/3$ each. The printed $\overline{Y}_1(0)$ is $1/3$, while $\mathrm{E}[y_1(\mathbf{Z})\mid Z_1=0]=1/4$, and all other averages are zero. I obtain

and the average direct effect in the convention of Hudgens and Halloran is $1/12$. The printed $\tau$ equals neither $1/12$ nor $-1/12$. To isolate the sign, fix the assignment of subject 4 at control and set $y_i(\mathbf{z})=z_i$ for the subjects in the window. Then the two definitions of $\overline{Y}_i(z)$ agree, $\tau=\mathrm{E}[\hat\tau]=1$, and the average direct effect in their convention is $-1$.

What survives

The estimator, the design and, with the definition corrected in P014-E001, the unbiasedness of the estimator are unaffected. The reference to Hudgens and Halloran (2008) remains the right one for the probability-weighted target. The identification with their average direct effect needs a remark on the sign.

The problem

The two terms after the summation sign are not bracketed. Read literally, the sum and the factor $N_0^{-1}$ apply to $\overline{Y}_i(1)$ alone, and the index $i$ in $\overline{Y}_i(0)$ is free. The prose that follows, and the display for $\mathrm{E}[\hat\tau]$ on the same page, show that both terms belong to the summand.

Evidence a check of the printed page

The evidence is a check of the page. The display that defines $\tau$ has neither brackets nor a second summation sign. The later display reads $\mathrm{E}[\hat\tau]=N_0^{-1}[\sum_{i:X_i\in W_0}\overline{Y}_i(1)-\sum_{i:X_i\in W_0}\overline{Y}_i(0)]$, and by linearity of the finite sum its right side equals the bracketed form below.

What survives

The intended target and its agreement with the display for $\mathrm{E}[\hat\tau]$ are unaffected, and nothing depends on the missing brackets. The definition of $\overline{Y}_i(z)$ inside the sum is corrected separately in P014-E001.

The problem

The range is one page too long. Page 1408 of that issue is the first page of the next article.

Evidence a check of the printed page

The evidence is a check against the publisher. Its table of contents for the Journal of the American Statistical Association 101(476) lists Sobel’s article at pages 1398–1407 and the next article, “Exceedance Control of the False Discovery Proportion”, at pages 1408–1417. The publisher’s page for the article has the same range.

What survives

The cited work remains identifiable, and nothing in the paper depends on the last page.