Corrections to: Data-Adaptive Causal Effects and Superefficiency

The problem

Proposition 1 asserts that, for every empirical distribution $F_n$, the target $\theta_{F_n}$ of Definition 2 equals $\mathrm{E}_G[w\tau]/\mathrm{E}_G[w]$ for some nonnegative weighting $w$. Definition 2 clips $\hat\theta$ to the closed interval $[\mathrm{E}_G[\tau]-c,\ \mathrm{E}_G[\tau]+c]$. When $c$ is finite, at least one end of this interval is the infimum or the supremum of the support of $\tau$, and the target equals that value whenever $\hat\theta$ lies at or beyond it. A weighted mean with nonnegative weights attains every value strictly between the infimum and the supremum of the support, but it attains a finite endpoint only if the endpoint has positive probability. The fact given as the proof of Proposition 1 includes the endpoints without this condition. If, for example, $\tau$ is bounded and has a continuous distribution, the endpoint has probability zero. On the realizations where $\hat\theta$ lies at or beyond it, the target is a value that no weighting attains, and it is not the average causal effect of any subpopulation. These realizations have probability tending to zero as $n\to\infty$, but in the example below they have positive probability at every sample size. The abstract, the introduction and the discussion describe the target as a local average causal effect, and that description depends on Proposition 1.

Evidence a historical Lean attribution (Generated status: clause support not certified in this release)

The evidence is a derivation and an exact counterexample. The obstruction at an endpoint also has a Lean proof.

The values that a weighted mean attains. I call a weighting admissible if it is a measurable function $w\ge0$ of the full data such that $0<\mathrm{E}_G[w]<\infty$ and $\mathrm{E}_G[|w\tau|]<\infty$. Let $t$ be a real number. If $p_0=\Pr_G(\tau=t)>0$, then the weighting $w=\mathbf{1}\{\tau=t\}/p_0$ satisfies $\mathrm{E}_G[w]=1$ and $\mathrm{E}_G[w\tau]=t$. Suppose next that $p_-=\Pr_G(\tau<t)>0$ and $p_+=\Pr_G(\tau>t)>0$, and set

Then $m_-<t<m_+$ and $0<a<1$. The weighting

is bounded and nonnegative. Computing the two expectations, I find $\mathrm{E}_G[w]=a+(1-a)=1$ and $\mathrm{E}_G[w\tau]=am_-+(1-a)m_+=t$. Conversely, suppose that $\Pr_G(\tau=t)=0$ and $\Pr_G(\tau>t)=0$, so that $\tau<t$ almost surely. If an admissible weighting $w$ had weighted mean $t$, then $\mathrm{E}_G[w(t-\tau)]=t\,\mathrm{E}_G[w]-\mathrm{E}_G[w\tau]=0$. Since $w(t-\tau)\ge0$, this implies that $w(t-\tau)=0$ almost surely. Since $t-\tau>0$ almost surely, I have $w=0$ almost surely, which contradicts $\mathrm{E}_G[w]>0$. The case $\Pr_G(\tau<t)=0$ is the same, with $w(\tau-t)$ in place of $w(t-\tau)$. I conclude that an admissible weighting with weighted mean $t$ exists if and only if

Every $t$ strictly between the infimum and the supremum of the support of $\tau$ satisfies the second condition. A finite endpoint of the support satisfies neither condition unless it has positive probability.

The counterexample. Let $\tau$ be uniform on $(0,1)$, let $B$ equal $-1$ or $1$ with probability $1/2$ each, and let $T$ be a fair treatment assignment, with $\tau$, $B$ and $T$ independent. Let $Y(0)=B$, $Y(1)=B+\tau$ and $Y=TY(1)+(1-T)Y(0)$. Then $\mathrm{E}_G[\tau]=1/2$, the support of $\tau$ is $[0,1]$ and $c=1/2$, so that Definition 2 clips $\hat\theta$ to $[0,1]$. For $n$ independent units, I take the inverse-probability estimator $\hat\theta=n^{-1}\sum_{i=1}^nX_i$ with $X_i=2T_iY_i-2(1-T_i)Y_i$. The summands are bounded, with mean $1/2$ and variance $53/12$. By the central limit theorem, I have $\sqrt n(\hat\theta-1/2)\Rightarrow N(0,53/12)$. On the event that $(T_i,Y_i)=(0,-1)$ for all $i$, every $X_i$ equals $2$, and therefore $\hat\theta=2$ and $\theta_{F_n}=1$. Since $\tau<1$ almost surely, no admissible weighting has weighted mean $1$, by the characterization above.

The Lean proof covers the following statement: for an integrable weighting $w\ge0$ with positive expectation, on a population in which $\tau<1$ almost surely and $w\tau$ is integrable, the weighted mean is less than $1$. It also covers the uniform distribution on $(0,1)$, for which the value obtained by clipping the estimate $2$ to $[0,1]$ has no such representation. The Lean statements also specify the Gaussian limit, the probability $4^{-n}$ of this event at every $n\ge1$, and the identity expressing the estimator as an integral against the empirical distribution of the observed pairs. A second example has a failure event of probability $16^{-n}$: its effect is discrete, with values that accumulate at $-1$ and $1$ and do not reach them.

What survives

The conclusions of Propositions 2, 3 and 4 and the equality $\Pr(\hat\theta=\theta_{F_n})=1$ of Corollary 1 hold for the target as printed, since they can be derived without Proposition 1. Proposition 1 holds as printed when $\tau$ has finite support or support unbounded in both directions, and for the printed target the weighted representation exists with probability tending to one. The statement that the printed target is a local average causal effect in every realization does not survive; in particular, under the hypothesis of Corollary 1 the common value of $\hat\theta$ and $\theta_{F_n}$ need not be a weighted average of causal effects. With the radius $d$ the target differs from the printed one on the realizations with $|\hat\theta-\mathrm{E}_G[\tau]|>d$. No new assumption on $G$ is needed, the conclusions of Propositions 2, 3 and 4 hold with $d$ in place of $c$, and the claim that $\hat\theta$ is superefficient for a data-adaptive local average causal effect stands.

The problem

Definition 2 defines $\theta_{F_n}$ as the clip of the estimator $\hat\theta=\tilde\theta+u$ to the interval $[\mathrm{E}_G[\tau]-c,\ \mathrm{E}_G[\tau]+c]$. Both proofs compute with the normal part $\tilde\theta$ as though $\theta_{F_n}$ were the clip of $\tilde\theta$. On the event $|\hat\theta-\mathrm{E}_G[\tau]|\le c$ the target equals $\hat\theta$, which implies that $\tilde\theta-\theta_{F_n}=-u$, and $u$ is not zero in general. In the proof of Proposition 3, the probability $\Pr(\tilde\theta-\theta_{F_n}\ne0)$ need not equal $2\Phi(-c\sqrt n/\sigma)$, and $\tilde\theta-\theta_{F_n}$ need not be $o_p(a_n)$ for every positive sequence $(a_n)$. In the proof of Proposition 4, the squared deviation need not vanish when $|\tilde\theta-\mathrm{E}_G[\tau]|\le c$; when $|\hat\theta-\mathrm{E}_G[\tau]|\le c$, it equals $u^2$. The Gaussian tail integral does not bound $\mathrm{E}[(\tilde\theta-\theta_{F_n})^2]$ in general. The three steps do hold for some remainders, for instance a remainder that is zero when $\tilde\theta$ is in the interval and moves $\tilde\theta$ away from the interval otherwise. They do not follow from the assumptions of the two propositions. Both propositions are true as stated, but the printed proofs do not establish them.

Evidence an exact counterexample

I give an exact counterexample to the three steps and derive both propositions in the corrected proofs below.

Let the effects equal $-1$ or $1$ with probability $1/2$ each, so that $\mathrm{E}_G[\tau]=0$ and $c=1$, and let $X_1,\ldots,X_n$ be independent $N(0,1)$ observations. Set $\tilde\theta=\bar X_n$ and $\hat\theta=\tilde\theta+1/n$. Then $\sigma=1$ and $u=1/n=o_p(n^{-1/2})$. Further, $\sqrt n\,\hat\theta=N(0,1)+\varepsilon$ with $\varepsilon=n^{-1/2}$ and $\mathrm{E}[\varepsilon^2]=1/n=o(n^{-1/2})$, and the assumptions of both propositions hold. The difference $\tilde\theta-\theta_{F_n}$ equals $-1/n$ when $|\hat\theta|\le1$, and outside this event it vanishes only if $\tilde\theta=1$, which has probability zero. I therefore have $\Pr(\tilde\theta-\theta_{F_n}\ne0)=1$ for all $n$, while the printed value is $2\Phi(-\sqrt n)=\Pr(|\tilde\theta|>1)\le1/n$. For $n\ge2$, the event $|\tilde\theta|\le1/2$ has probability at least $1-4/n$, and on it $\tilde\theta-\theta_{F_n}=-1/n$. It follows that $n(\tilde\theta-\theta_{F_n})\to-1$ in probability, and $\tilde\theta-\theta_{F_n}$ is not $o_p(a_n)$ for $a_n=1/n$. At $n=25$, using the same event, I obtain

while the printed tail integral equals $\mathrm{E}[\tilde\theta^2\mathbf{1}\{|\tilde\theta|>1\}]$, which is at most $\mathrm{E}[\tilde\theta^6]=15/15625$. The printed inequality fails by at least $6/15625$.

The corrected proofs use three facts. First, Definition 1 implies that $\sqrt n(\hat\theta-\mathrm{E}_G[\tau])=O_p(1)$, and hence that $\hat\theta$ is consistent. Second, the function $r(x)=x-C(x)$, where $C$ is the clip to the interval, is continuous and piecewise linear with slopes $1$, $0$ and $1$, so that $|r(y)-r(x)|\le|y-x|$ for all $x$ and $y$. Third, a variable $W\sim N(0,\sigma^2/n)$ has $\mathrm{E}[W^4]=3\sigma^4/n^2$. Note that the argument for Proposition 4 does not use independence of $\tilde\theta$ and $\varepsilon$.

What survives

Propositions 3 and 4 hold as stated under their printed assumptions, with no further assumption, and the superefficiency and mean square rates stated in the abstract, the introduction and the discussion hold for the Definition 2 target. Proposition 3 holds for any consistent estimator of $\mathrm{E}_G[\tau]$, and then $\hat\theta-\theta_{F_n}=o_p(a_n)$ for any positive sequence $(a_n)$. The closed-form evaluation of the Gaussian tail integral is correct, and the intermediate conclusion $\mathrm{E}[(\tilde\theta-\theta_{F_n})^2]=o(n^{-1})$ is true, although the displayed bound does not establish it. Definition 2, Proposition 2 and Corollary 1 do not use these steps. The corrected proofs also apply to the target of P017-E001, with a fixed finite radius $0<d<c$ in place of $c$.

The problem

The hypothesis of Corollary 1 places the support of $\hat\theta$ inside the interval $[\mathrm{E}_G[\tau]-c,\ \mathrm{E}_G[\tau]+c]$, so that the event $|\hat\theta-\mathrm{E}_G[\tau]|\le c$ has probability one. The proof sentence assigns it probability zero, which contradicts the corollary and the sentence that follows the proof. In the proof of Proposition 4, the integral is printed without $dx$. Both are misprints. In the same display the expectation carries a subscript $G$ that the statement of Proposition 4 and the sentence after the display omit. This is a difference of notation and not an error, since the sampling expectation is the one induced by $G$.

Evidence a derivation

The evidence is a check of the page and a derivation. Let $\mu=\mathrm{E}_G[\tau]$. The hypothesis $c\ge\max(\sup(\operatorname{Supp}[\hat\theta])-\mu,\ \mu-\inf(\operatorname{Supp}[\hat\theta]))$ implies that $\inf(\operatorname{Supp}[\hat\theta])\ge\mu-c$ and $\sup(\operatorname{Supp}[\hat\theta])\le\mu+c$. Then $|\hat\theta-\mu|\le c$ almost surely, and by Definition 2, I have $\Pr(\hat\theta=\theta_{F_n})=1$. For the integral, set $s=\sigma/\sqrt n$ and $a=c/s$, and let $\phi$ be the standard normal density. Substituting $x=sz$ and integrating by parts, I obtain

which is the second line of the printed display. Numerical integration agrees with the printed closed form at three choices of $c$, $\sigma$ and $n$.

What survives

Corollary 1 and the closed-form evaluation of the integral are correct as printed, and no result changes. The inequality in which the integral appears is a separate matter, treated in P017-E002.

The problem

The sentence follows a proof in which the expression $2\Phi(-c\sqrt n/\sigma)$ appears, and in that context “quickly” says that the probability of zero estimation error tends to one at a fast rate. No rate follows from the assumptions of Proposition 3. Definition 1 requires only that $\sqrt n(\hat\theta-\mathrm{E}_G[\tau])=N(0,\sigma^2)+o_p(1)$, and the $o_p(1)$ term may be large on an event whose probability tends to zero arbitrarily slowly. For a decomposition $\hat\theta=\tilde\theta+u$ with $\tilde\theta=N(\mathrm{E}_G[\tau],\sigma^2/n)$, by the union bound, I have

and the condition $u=o_p(n^{-1/2})$ does not control the rate of the second term. The sentence prints no specific rate. If “quickly” is read as meaning only that the probability tends to one, then the sentence asserts no rate and needs no correction.

Evidence an exact counterexample

The evidence is an exact counterexample. Let $(p_n)$ be any sequence in $(0,1]$ that tends to zero. Let the effect $\tau$ equal $-1$ or $1$ with probability $1/4$ each and be uniform on $[-1,1]$ otherwise, so that $\mathrm{E}_G[\tau]=0$, the support of $\tau$ is $[-1,1]$ and $c=1$. Let the data contain independent observations $X_i\sim N(0,1)$ and $U_i$ uniform on $[0,1)$. Let $B_n$ be the indicator of the event that the fractional part of $\sum_{i=1}^nU_i$ is less than $p_n$. I have $\Pr(B_n=1)=p_n$, and $B_n$ is independent of $\bar X_n$. Let

The estimator $\hat\theta$ is a symmetric function of the sample. I have $\sqrt n\,\hat\theta=\sqrt n\,\bar X_n+\varepsilon_n$, where $\sqrt n\,\bar X_n\sim N(0,1)$ and $\Pr(\varepsilon_n\ne0)\le4/n+p_n\to0$, and Definition 1 holds with $\sigma^2=1$. If $B_n=0$, then $\hat\theta=A_n\in[-1/2,1/2]$ and $\theta_{F_n}=\hat\theta$. If $B_n=1$, then $\hat\theta\ge19/2$, $\theta_{F_n}=1$ and the error is at least $17/2$. I conclude that

For $p_n=1/(1+\lfloor\log_2n\rfloor)$, this probability is $1/10$ at $n=1000$ and $1/40$ at $n=10^{12}$, and it is not $O(n^{-\alpha})$ for any $\alpha>0$. The conclusion of Proposition 3 holds in the example, since $\Pr(\sqrt n|\hat\theta-\theta_{F_n}|>\delta)\le p_n$ for all $\delta>0$.

What survives

Proposition 3 is unaffected: its conclusion uses only that the probability of the event $|\hat\theta-\mathrm{E}_G[\tau]|\le c$ tends to one. No later result uses a rate for this probability. Proposition 4 makes its own stronger assumption on the remainder, and its conclusion holds under that assumption; the example above does not satisfy it.

The problem

Reference 10 names Princeton, NJ as the place of publication. The item is Working Paper 314 of the U.C. Berkeley Division of Biostatistics Working Paper Series, dated June 2013, and its place of publication is Berkeley, CA. Reference 1 has the journal and the year only, although Section 3 cites its p. 188. Both entries still identify the works, and no result depends on them.

Evidence a check of the printed page

The evidence is a check of the page against public bibliographic sources. The page of the series for the working paper (https://biostats.bepress.com/ucbbiostat/paper314/) reads “U.C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 314”, with the date June 2013. A later article by Balzer, Petersen and van der Laan cites the same report as “Berkeley: Division of Biostatistics, University of California; 2013”. The publisher’s page for the article of Crump, Hotz, Imbens and Mitnik has Biometrika 96(1):187–199, 2009. The words quoted in Section 3 appear on p. 188 in the indexed text of a copy in the journal’s format that one of the authors has posted. I did not obtain the working paper itself or the publisher’s full text of the article. The sources do not show at what stage the place of publication was misprinted.

What survives

The discussion and the results are unaffected. The year 2009, the citation of p. 188 and the quotation in Section 3 are correct as printed.

Intended meaning

By Definition 2, $\hat\theta=\theta_{F_n}$ exactly when $|\hat\theta-\mathrm{E}_G[\tau]|\le c$. This is the event that the sentence intends: the sentence after the proof of Corollary 1 names the interval $[\mathrm{E}_G[\tau]-c,\ \mathrm{E}_G[\tau]+c]$. The support of $\tau$ is a different set in general. If $\tau$ is exponential with mean $1$, then the support is $[0,\infty)$ and the interval is $[0,2]$, and an estimate $\hat\theta=3$ lies in the support with error $1$. If $\tau$ equals $-1$ or $1$ with probability $1/2$ each, then the interval is $[-1,1]$, and an estimate $\hat\theta=0.03$ has error zero and lies outside the support $\{-1,1\}$. With the interval in place of the support, the sentence names the correct event, and since $\hat\theta$ is consistent and $c>0$, the probability of this event tends to one. No result depends on the sentence.