Corrections to: Research Note: A More Powerful Test Statistic for Reasoning about Interference between Units

The problem

The two sentences give a reason to prefer the simple SSR statistic to the KS statistic: the KS statistic is said to ignore information about the joint distribution of treated and control outcomes, and the SSR statistic is said to include their correlation. This reason does not hold. The statistic meant by equation (1), and computed by the replication code, is the residual sum of squares from the regression of the adjusted outcome on an intercept and the treatment indicator (see P016-E002). It is determined by the group sizes and the first two moments of the adjusted outcome within each treatment group. Like the KS statistic, it is a function of the two group distributions and their sizes, and no pairing of treated with control units and no covariance between them enters either statistic. With the adjusted outcomes and the group sizes fixed, a smaller SSR corresponds exactly to a larger absolute difference in group means. The SSR test is then the test based on the absolute difference in means. A narrower statement is true: the randomization distribution of the SSR statistic reflects the dependence between the two groups that the assignment design induces. The same holds for the randomization distribution of the KS statistic, and the statement does not distinguish the two. The two statistics differ in power because they summarize the two group distributions differently.

Evidence an exact counterexample

I use a derivation and exact finite examples.

Let $a_i=\tilde y_{i,Z_i=0}$ be the adjusted outcomes for one hypothesis, held fixed. Let $z\in\{0,1\}^n$ have $n_1>0$ ones and $n_0>0$ zeros, with $n=n_0+n_1$. I write $m_g$ for the mean of $a_i$ over the units with $z_i=g$, $m$ for the mean over all units, and $T(a)=\sum_i(a_i-m)^2$. For any group $g$ and any constant $c$, I have

since the cross term vanishes. The least squares fit on an intercept and $z$ therefore has fitted values $m_0$ and $m_1$, and its residual sum of squares is $R(a,z)=\sum_g\sum_{i:z_i=g}(a_i-m_g)^2$. Taking $c=m$ and summing over the two groups, I obtain $T(a)=R(a,z)+n_0(m_0-m)^2+n_1(m_1-m)^2$. Since $m_0-m=-n_1(m_1-m_0)/n$ and $m_1-m=n_0(m_1-m_0)/n$, I conclude that

The term $T(a)$ and the factor $n_0n_1/n$ do not depend on the assignment. For any two assignments $z$ and $w$ with these group sizes, I have $R(a,w)\le R(a,z)$ if and only if $|m_1(w)-m_0(w)|\ge|m_1(z)-m_0(z)|$, ties included. This implies that, for every assignment law $P_Z$ on such assignments,

Further, I have $R=n_0(q_0-m_0^2)+n_1(q_1-m_1^2)$, where $q_g$ is the mean of $a_i^2$ in group $g$. The two group distributions and their sizes therefore determine $R$.

The table shows the example $a=(0,1,2,3)$ with two of the four units treated.

Three further exact computations bear on the two printed statements. First, for treated values $(0,2)$ and control values $(1,3)$, the SSR is $4$ and the KS statistic is $1/2$ whichever control value is paired with which treated value, although the empirical covariance of the pairs, with divisor $2$, is $1$ under one pairing and $-1$ under the other. Second, both randomization distributions depend on the design: at the assignment that treats units 1 and 2, both $p$ values are $1/3$ under complete randomization and $1/2$ when one unit is chosen uniformly and independently for treatment in each of the blocks $\{1,3\}$ and $\{2,4\}$. Third, the SSR is not a function of the KS value: when units 1 and 3 are treated, the KS statistic is $1/2$ and the SSR is $4$, and when units 1 and 4 are treated, the KS statistic is $1/2$ and the SSR is $5$.

What survives

The simple SSR statistic and the validity of its exact randomization test are unaffected by this correction, and no simulation, figure or part of the application depends on this explanation. The SSR and KS statistics remain different summaries with different power (Fig. 2). The statistics with network regressors do use information beyond the two group distributions: in an exact four-unit example, two outcome vectors with the same group distributions have SSR $0$ and SSR $4$ once degree is added to the regression. I correct only the two explanatory sentences here. I have not recomputed the reported power values.

The problem

The description of the application does not specify the model to which Fig. 4 and the rectangle on p. 402 refer. In the replication code the matrix is a similarity matrix $S$, with $S_{ij}=1-|s_i-s_j|/2$ for the W-NOMINATE scores $s_i$ and $s_j$, so that ideologically closer legislators have larger weights; the diagonal and undefined entries are zero. The exposure of legislator $i$ is not $(Sz)_i$ normalized to unit variance. The code first subtracts from $(Sz)_i$ its mean over those of 10,000 saved assignments in which legislator $i$ has the same treatment status, and only then centres and scales across legislators. This subtraction is not a recentring or a rescaling. The exposure used is not an affine function of the raw exposure, even when an intercept and a change in the direct effect are allowed, and in the application table of the replication files the two standardized exposures have correlation $0.07$. The indirect effect $\beta_2$ of Fig. 4 therefore multiplies a different variable from the one that the text describes, and its sign and its unit can be read only with the coded definition. The regression in the SSR statistic has an intercept, own assignment and this exposure, with no separate degree column, and the phrase “SSR + Design statistic as presented in this article” does not convey this.

The text also does not say how the $p$ values are formed. The simulations behind Figs. 2 and 3 use a two-sided $p$ value, twice the smaller of the two tail proportions, capped at $1$, and they count a rejection when $p\le.05$. The text and the caption of Fig. 3 say “less than .05”. The application uses one-sided $p$ values from 500 draws, the lower tail for SSR and the upper tail for KS. The two rules define different tests, and the note does not distinguish them.

Evidence an exact counterexample

I base this correction on the replication code, exact arithmetic on the application table of the replication files, and exact small examples.

Let $r_i=(Sz)_i$ be the raw exposure and $h_i$ the exposure after the subtraction. If the subtraction could be absorbed by an intercept, the direct effect and a rescaling of the indirect effect, then there would exist constants $a$, $b$ and $c$ such that $h_i=a+bz_i+cr_i$ for all $i$, and the four columns $(1,z,r,h)$ would have rank at most three. For the first four rows of the application table, the determinant of these four columns, computed in exact rational arithmetic from the decimals of the table, is $-1.4030\ldots\ne0$. No such constants exist. Across the 70 legislators the correlation between the standardized raw exposure and the standardized exposure used is $0.0665$, and the largest difference between the two is $4.76$ standard deviations.

The same holds in an example in which all assignments are enumerated. Let six units have scores $(-1,-3/4,-1/2,0,1/4,1)$ and form the pairs $(1,2)$, $(3,4)$ and $(5,6)$, with one treated unit in each pair, a design with eight assignments. At $z=(1,0,1,0,1,0)$ the raw exposures $r$ and the exposures $h$ after the subtraction are

Units 1 and 2 have equal exposure after the subtraction and unequal raw exposure, while units 1 and 3 differ in both. This implies that $h$ is not of the form $a+cr$, and the rank of $(1,z,r,h)$ is four. For the outcome $y=(0,1,0,1,1,0)$ the residual sum of squares is $4/3$ from the regression on $(1,z,h)$ and $655/708$ from the regression on $(1,z,r)$.

For the rejection rule, take the reference values $1,2,\ldots,1000$. At the observed value $25$ the lower tail proportion is $1/40$ and the two-sided $p$ value of the simulation code is exactly $.05$. The code counts a rejection, and the printed rule does not. At the observed value $40$ the lower tail proportion is $.04$ and the two-sided $p$ value is $.08$; at $980$ they are $.98$ and $.042$. Neither the one-sided rule nor the two-sided rule rejects whenever the other does.

The numerical comparison uses the application table that is among the replication files. The data file that the figure script loads is not among them, and I did not recompute Fig. 4.

What survives

This correction changes the description of the model and testing rules used by the replication code. The note’s comparison of a bounded SSR region and an unbounded KS region in Fig. 4 concerns the coded model. I have not established a numerical error in Fig. 4 or in the rectangle, and the absence of a degree column is not shown to make the test invalid. Figs. 2 and 3 are results for the two-sided rule with rejection at $p\le.05$; they cannot be relabelled as results for another rule without recomputation, and my recalculated null rejection rates are compatible with the reported rates of about 4% and 2% within Monte Carlo error; they do not establish a level guarantee. The meaning of $\beta_2$ changes: it is the coefficient of a centred, standardized similarity exposure, not of normalized sums of distances.

The problem

Steps (2) and (3) define the adjusted outcome $\tilde y_{i,Z_i=0}=Y_i-Z_i\tau_0$ and build the randomization distribution from it. Step (4) then writes the observed statistic with the raw outcome $Y_i$. In general $\mathcal{T}(Y,Z)$ differs from $\mathcal{T}(\tilde y,Z)$; the two coincide when $\tau_0=0$ or when the statistic is unchanged by the adjustment. As printed, the observed statistic is then compared with the distribution of a different quantity. Steps (2) and (3) show which statistic is meant, and the replication code computes it.

Evidence an exact counterexample

The evidence is a derivation, an exact counterexample to the rule as printed, and a check of the replication code.

Let $Z$ be the realized assignment and $Y$ the vector of observed outcomes. Suppose that the hypothesis $y_{i,Z_i=1}=y_{i,Z_i=0}+\tau_0$ holds, with no interference. Inserting it into the identity $Y_i=Z_iy_{i,Z_i=1}+(1-Z_i)y_{i,Z_i=0}$, I obtain $Y_i=y_{i,Z_i=0}+\tau_0Z_i$. This implies that $u_i=Y_i-\tau_0Z_i$ equals $y_{i,Z_i=0}$, which does not depend on the assignment. Under the hypothesis, the statistic at the realized assignment is $\mathcal{T}(u,Z)$, and its randomization distribution is that of $\mathcal{T}(u,Z')$, where $Z'$ follows the assignment law of the experiment and $u$ is held fixed. For the difference in means, I have $\mathcal{T}(Y,Z)=\mathcal{T}(u,Z)+\tau_0$.

For the counterexample, I take four units, two of them treated, with the six assignments equally likely, $u=(0,1,2,3)$, true effect $\tau_0=10$, and the difference in means as the statistic. The six values of $\mathcal{T}(u,Z')$ are $-2,-1,0,0,1,2$. At $Z=(0,0,1,1)$ the observed outcomes are $Y=(0,1,12,13)$, the raw statistic is $12$, and the adjusted statistic is $2$. The two-sided $p$ value is the proportion of the six values of $\mathcal{T}(u,Z')$ that are at least as large in absolute value as the observed statistic. It is $0$ for the raw statistic and $1/3$ for the adjusted statistic. Under each of the six assignments the raw statistic lies between $8$ and $12$, and its $p$ value is $0$. The rule as printed rejects the true hypothesis $\tau_0=10$ with probability one.

The randomization routine of the replication code adjusts the outcome once, evaluates the observed statistic on the adjusted outcome, and compares it with the statistic evaluated on the same adjusted outcome under the sampled assignments.

What survives

The procedure of steps (1) to (3) and the inspected randomization routine are unchanged. I found no reported result that depends on the raw statistic. Only the symbol in step (4) changes.

The problem

Read literally, the KS display differs from the statistic used, equation (1) leaves its mean undefined, and equation (2) omits the component index of its fitted vector.

1. Footnote 3 has no absolute value, and the display is the one-sided statistic $\max_i\{F_1-F_0\}$. The statistic described, which is sensitive to any difference between the two distributions, and the statistic coded are the two-sided $\max_i|F_1-F_0|$.
2. In equation (1) the overbar carries the unit index $i$ and is not defined. The statistic meant is the residual sum of squares from the regression of the adjusted outcome on an intercept and $z$, in which the mean subtracted from unit $i$ is the mean of its own assignment group. If the overbar is read as the mean over all units, the display is the total sum of squares, which does not depend on $\mathbf{z}$. The sentence before the display says “sums the difference” where the squared difference is meant.
3. In equation (2) the summand subtracts the whole vector $\mathbf{X}\hat{\boldsymbol{\beta}}$ from the scalar $\tilde y_{i,Z_i=0}$, since the row index is missing. The list of columns of $\mathbf{X}$ has no constant. The regression in the replication code has an intercept.

The surrounding text on least squares regression identifies the statistics that are meant.

Evidence an exact counterexample

The evidence is exact arithmetic on small examples, a derivation, and a check of the functions in the replication code on the same examples.

For the KS statistic, let the adjusted outcomes be $u=(0,1,2,3)$, with $z=(0,0,1,1)$. At the points $0,1,2,3$ the differences $F_1-F_0$ are $-1/2,-1,-1/2,0$. The display as printed gives $0$, and the two-sided statistic is $1$.

For equation (1), an intercept and $z$ fit one constant $c_g$ to each assignment group $g$. For any constant $c$, I have $\sum_{i:z_i=g}(u_i-c)^2=\sum_{i:z_i=g}(u_i-\bar u_g)^2+n_g(\bar u_g-c)^2$, where $\bar u_g$ is the mean of the $n_g$ units of group $g$. The fitted constants are therefore the group means, and the residual sum of squares is $\sum_i(u_i-\bar u_{z_i})^2$. With $u=(0,1,2,3)$ and two of the four units treated, the total sum of squares is $5$ under all six assignments, and the residual sum of squares takes the values $1,4,5,5,4,1$. At the assignment that treats units 1 and 2, the lower-tail $p$ value is $1/3$ for the residual sum of squares and $1$ for the total sum of squares.

For equation (2), take six units with the single edge $\{1,2\}$, $z=(1,0,1,0,1,0)$ and $u=(1,1,1,1,1,1)$. The residual sum of squares is $0$ with the intercept and $2$ without it.

On these inputs the functions of the replication code return $1$ for the KS statistic, $1,4,5,5,4,1$ for the simple SSR statistic and $0$ for the network SSR statistic. These are the values of the corrected definitions.

What survives

The inspected functions of the replication code compute the corrected definitions. These notation repairs do not change their outputs, and I found no error in the comparisons of Figs. 2 to 4 caused by these defects. Note that the signed maximum of footnote 3 is a valid one-sided statistic; it is not the statistic used. The power recalculations are described below under what I checked.

The problem

The passage identifies the property in the quotation, being small for similar distributions and large for divergent ones, with the effect increasing property. The two are different. A statistic is effect increasing if, at a fixed assignment, raising treated responses and lowering control responses cannot decrease it. This condition is directional. An unsigned measure of the difference between two distributions need not satisfy it, and the two-sided KS statistic does not. The next sentence gives a correct sufficient condition for validity, but validity holds for every test statistic calibrated under the known assignment law, and whether a test is unbiased against alternatives is a separate question.

Evidence an exact counterexample

The evidence is an exact counterexample and a derivation.

Fix the assignment $z=(1,1,0,0)$ and change $y=(0,1,2,3)$ to $y'=(2,3,2,3)$. Both treated responses rise by $2$ and the control responses are unchanged. The two-sided KS statistic is $1$ for $y$, where every treated value lies below every control value, and $0$ for $y'$, where both groups have the values $2$ and $3$. Since the statistic decreases, it is not effect increasing. The KS function of the replication code returns the same two values, and the signed maximum printed in footnote 3 also falls from $1$ to $0$.

For validity, let $\Omega$ be the finite set of assignments, with known probabilities $q(z)>0$. Under the sharp null hypothesis of no effects, the outcome vector $y^0$ is the same under every assignment. Any statistic then has fixed scores $s(z)=\mathcal{T}(y^0,z)$. Define the $p$ value $p(z)=\sum_{w\in\Omega:s(w)\ge s(z)}q(w)$. Let $v_1>\cdots>v_k$ be the distinct scores, let $m_j$ be the probability that $s(Z)=v_j$, and let $M_j=m_1+\cdots+m_j$. An assignment with score $v_j$ has $p(z)=M_j$, and $M_j$ is strictly increasing in $j$. Fix $\alpha\in[0,1]$. If $M_1>\alpha$, the event $\{p(Z)\le\alpha\}$ is empty. Otherwise, let $r$ be the largest index such that $M_r\le\alpha$. Then the event consists of the assignments with scores $v_1,\ldots,v_r$, and its probability is $M_r\le\alpha$. I conclude that $\Pr\{p(Z)\le\alpha\}\le\alpha$ for every statistic. The argument does not use the effect increasing property.

The cited propositions and the quotation were not compared with the two works cited.

What survives

The conclusion that the randomization test of the sharp null hypothesis is valid stands, for a more general reason. Nothing later in the note uses the effect increasing property: the statistics, the simulations and the application are unaffected, and the note does not claim that its SSR statistics are effect increasing. Only these explanatory sentences change.

The problem

The text and the replication code disagree about how the simulation behind Fig. 3 is divided. In the code, when no setting is supplied, the number of simulated experiments is 1000 and each test samples 10,000 assignments, and the driver files of the archive supply no other setting. The environment variable that the code provides changes only the number of experiments; with 10,000 experiments each test still samples 10,000 assignments, which again differs from the text. Since saved results and run logs are not among the replication files, the setting used for the published figure cannot be recovered. The difference bears on the precision of the reported rates: at a rejection rate of $0.04$ the Monte Carlo standard error is $0.0062$ with 1000 experiments and $0.0020$ with 10,000.

Evidence a recalculation from the source

The evidence is a recalculation from the replication code and a check of the page. Running the count expressions, the sampler and the test routine of both released versions of the code, I obtain a matrix of 1000 assignments of the 256 units, and each test receives 10,000 samples. The standard errors are $\sqrt{p(1-p)/N}$ at $p=1/25$, which is $0.006197$ for $N=1000$ and $0.001960$ for $N=10{,}000$. I obtained these counts from the code settings and calls, without simulating rejection rates.

What survives

No rejection rate, power comparison or conclusion is shown to be wrong. The discrepancy establishes that the text and the code disagree, and it does not establish which setting was run. Fig. 2 is unaffected, since its script sets 1000 sampled assignments itself, and so are the statistics, the framework and the application.

The problem

The validity statements on pp. 397 and 400 are correct for the rule as printed. With a fixed statistic and $B$ fresh reference draws, the rule that rejects when the $p$ value is less than $.05$ has null rejection probability at most $50/1001$ for $B=1000$ and at most $500/10001$ for $B=10{,}000$. The simulation code applies a different rule. Each tail is the proportion of the $B$ reference statistics that are at most, or at least, the observed statistic, and the observed statistic is not counted among them. The $p$ value is twice the smaller tail, capped at $1$, and a rejection is counted when $p\le.05$. This rule is not a test of level $.05$ in general: its size can exceed $.05$ slightly, and its $p$ value can equal $0$.

Evidence an exact counterexample

The evidence is an exact counterexample, computed by enumeration and rational arithmetic, and a derivation.

For the counterexample, I take sixteen units, eight of them treated, with all $\binom{16}{8}=12870$ assignments equally likely, the sharp null hypothesis of no effect with outcomes $2^i$ for $i=0,\ldots,15$, and the simple SSR statistic. The statistic takes 6435 distinct values over these assignments, each shared by an assignment and its complement. The observed assignment and the $B$ reference assignments are drawn independently from the design.

The probability that the coded $p$ value equals $0$ is $0.0018$ for $B=1000$.

For the derivation, let $T_0$ be the observed statistic and $T_1,\ldots,T_B$ the reference statistics, and suppose that these $B+1$ statistics are exchangeable under the null hypothesis. Set $L=\#\{b:T_b\le T_0\}$ and $U=\#\{b:T_b\ge T_0\}$. With ties broken at random, the rank of $T_0$ among the $B+1$ statistics is uniform on $\{1,\ldots,B+1\}$, and this rank is at most $L+1$. I therefore have $\Pr(L\le c)\le(c+1)/(B+1)$ for every integer $c\ge0$, and the same bound for $U$. The event $2\min(L,U)/B<\alpha$ requires that $L\le c$ or $U\le c$ with $c=\lceil\alpha B/2\rceil-1$, and by the union bound its probability is at most $2(c+1)/(B+1)$. At $\alpha=.05$ and $B=1000$, I have $c=24$, and the bound is $50/1001<.05$. The coded rule allows $c=\lfloor\alpha B/2\rfloor=25$, for which the same bound is $52/1001>.05$; the counterexample shows that the size can exceed $.05$. Next, set $p_L=(1+L)/(B+1)$ and $p_U=(1+U)/(B+1)$. For every $a\in[0,1]$, I have $\Pr(p_L\le a)=\Pr(L+1\le\lfloor a(B+1)\rfloor)\le\lfloor a(B+1)\rfloor/(B+1)\le a$, and the same holds for $p_U$. For $\alpha<1$ the event $\min\{1,2\min(p_L,p_U)\}\le\alpha$ is the union of $\{p_L\le\alpha/2\}$ and $\{p_U\le\alpha/2\}$. Its probability is at most $\alpha$.

What survives

The validity statement for the exact randomization test on p. 397 and the printed rule at level $.05$ stand, as do the statistics and the framework. No reported rejection rate, power ranking or figure is shown to change. Note that the counterexample is a constructed design of sixteen units and not the simulation of 256 units in the note, and the excess over $.05$ is a fraction of a percentage point. The $p$ values of the application are computed from a fixed saved set of assignments; they need a separate argument and were not examined here.

The problem

The helper find_breaks returns the positions that follow each change of the accepted or rejected indicator along a grid axis, and the script uses those positions as the endpoints of the accepted range. For a run of accepted grid points at positions $r,\ldots,s$ strictly inside the grid, the helper returns $r$ and $s+1$. The lower endpoint is right. The upper endpoint is the first rejected grid point, one step above the last accepted one; the step is $2/49\approx0.04$ on the grid of 50 points over $[-1,1]$. The defect is in the endpoint summaries that the script saves. The rectangle on p. 402 is rounded text in the manuscript source, and it is not shown to come from these endpoints.

Evidence an exact counterexample

I checked the derivation and an exact counterexample using the original code.

Let $a_i$ indicate that the largest $p$ value in row $i$ of the matrix exceeds $.05$, for $i=1,\ldots,n$. The helper sets entry $i+1$ of its output to the truth value of $a_i\ne a_{i+1}$ for $i=1,\ldots,n-1$, and it returns the positions of the true entries. If the accepted positions are exactly $r,\ldots,s$ with $1<r\le s<n$, then the changes occur at $i=r-1$ and at $i=s$, and the helper returns $(r,s+1)$. The script then saves the grid values $(g_r,g_{s+1})$. The extremes of the accepted grid values are $(g_r,g_s)$. On the grid of the script, $g_i=(2i-51)/49$, which implies that $g_{s+1}-g_s=2/49$.

For the counterexample, let the $50\times50$ matrix have $p_{ij}=1/10$ for $20\le i\le30$ and $22\le j\le27$, and $p_{ij}=1/50$ elsewhere. The accepted direct range is positions 20 to 30, that is $[-11/49,9/49]$, and the script saves positions 20 and 31, that is $[-11/49,11/49]$. The accepted indirect range is positions 22 to 27, that is $[-1/7,3/49]$, and the script saves positions 22 and 28, that is $[-1/7,5/49]$. The helper returns $r$ and $s+1$ for each of the 1176 interior runs of the grid. I did not recompute the application’s $p$-value matrices to establish this endpoint error.

What survives

The two matrices of $p$ values, the shading and the axes of Fig. 4, and the contrast between a bounded SSR region and a KS region that reaches the edges of the grid are computed before this step and do not use it. The correction changes only the saved endpoint summaries: for an interior run, the upper endpoint falls by $2/49$. It does not establish corrected values for the rounded rectangle on p. 402.

The problem

1. Without interference, the adjustment changes only the outcomes of the units assigned to treatment, as p. 397 says and Fig. 2 shows. The two hypotheses therefore differ in what they imply for the treated group, and the control group is the same under both.
2. The ECDF $F(x)$ records the proportion of the sample at or below the threshold $x$, not at or below a sample point $x_i$.
3. The two parameters vary separately, and the truth is $\tau=.5$, $\beta=2$. The family $\tau=\tau_0$, $\beta=\tau_0$ does not contain the truth.
4. The legislators who were randomized, and whose votes are analysed, are the 70 members of the New Mexico House of Representatives.
5. The 1999 second edition of Nonparametric Statistical Methods has two authors, Myles Hollander and Douglas A. Wolfe. In the Brady entry the name of the editor David Collier is printed in reverse order.
6. Three words are misprinted.

Evidence an exact counterexample

The evidence is a check of the page for each item, exact small examples, the replication files, and a library catalogue record. With $Y=(9,11,10,12)$ and $Z=(0,0,1,1)$, the adjusted outcomes are $(9,11,19/2,23/2)$ under the additive hypothesis $\tau_0=1/2$ and $(9,11,5,6)$ under the multiplicative hypothesis $\tau_0=2$: the control values are the same and the treated values differ. The simulation code tests every pair on a grid of 22 values of $\beta$ and 21 values of $\tau$, and the grid contains the truth $(2,0.5)$. The roll-call table and the application table of the replication files each have 70 rows, one for the representative of each of House districts 1 to 70. The catalogue record of the 1999 second edition (WorldCat, OCLC 301603607) lists Myles Hollander and Douglas A. Wolfe as authors.

What survives

None of these corrections changes an implemented formula, a computation or a reported number. The adjustment, the ECDF formula, equation (3) and the replication code already have the correct content, and the application uses the House data.

The problem

The comparison meant is between the mean of the adjusted outcome $\tilde y_{i,Z_i=0}$ in the treated group and its mean in the control group. The first term carries the subscript $Z_i=1$. The quantity $\tilde y_{i,Z_i=1}$ is not defined in the note: the preceding sentence and p. 396 define only $\tilde y_{i,Z_i=0}$. The conditioning event $Z_i=1$ identifies the treated group correctly.

Evidence a derivation

The evidence is a derivation with a numerical illustration, and a check of the page. Let $u_i=Y_i-Z_i\tau$. The comparison meant is $\operatorname{mean}(u\mid Z=1)-\operatorname{mean}(u\mid Z=0)$. If $\tilde y_{i,Z_i=1}$ is read as the treated potential outcome implied by the hypothesis, then it equals $u_i+\tau$, and the printed comparison equals $\operatorname{mean}(u\mid Z=1)-\operatorname{mean}(u\mid Z=0)+\tau$, which retains the hypothesized effect. With $u=(0,1,2,3)$, $\tau=10$ and $Z=(0,0,1,1)$, the corrected comparison is $2$ and the comparison under this literal reading is $12$. The replication code uses the same adjusted outcome for both groups.