Corrections to: Inference in Spatial Experiments with Interference using the SpatialEffect Package

The problem

The paper takes the distribution of the treatment assignment as known and derives its inference from that distribution (p. 142). It also says that the permutation test can be adjusted for blocking or clustering in the design (p. 145). The printed call for the application passes neither blockvar nor clustvar. With these defaults, the inspected implementations draw the permutations uniformly from the assignments that treat 60 of the 121 villages.

The experiment was not randomized in this way. According to the Supplementary Materials of the source study (Jayachandran et al., 2017, section 2.2), treatment was assigned by a public lottery in each of seven subcounties. Before each lottery the sample villages of the subcounty were divided into two lists. The sizes of the two lists differ by at most one, and the lists are balanced on four village covariates (the number of forest owners sampled, distance to a road, income per capita and landholding size), with a threshold of 0.25 on each standardized mean difference. In small subcounties all partitions were examined, and in larger ones 1000 were sampled. One eligible partition was selected at random before each public draw, and the lottery chose between its two lists. This is neither an unrestricted choice of 60 villages nor an independent assignment of each village. Conditioning on the total of 60 treated villages does not remove the subcounty and balance restrictions.

The unrestricted default does not generate the randomization distribution of the experiment. The reported rejection rests on a margin of $0.0017$, between $-0.0321$ and $-0.0304$, and does not establish significance at the 5% level under the design. This is the only formal test in the application. The restrictions also rule out independent assignment of the 121 villages, so a justification of the spatial-HAC intervals that relies on Bernoulli assignment needs a further argument for this design.

Evidence a derivation

I base this correction on the published protocol of the source study, the displayed call and an exact derivation. The protocol is quoted from the copy of the Supplementary Materials posted by the first author of the source study (Materials and Methods, section 2.2, p. 3). It refers to “each of the 7 subcounties” and says that, in advance of each lottery, the sample villages in the subcounty were divided “into two sets, balanced on covariates”. The printed call shows no blocking or clustering argument. In both inspected implementations, these arguments default to NULL and are passed to the permutation routine of the ri package, which then draws uniformly among the assignments with the observed number of treated units. I could not reconstruct the published computation, since I do not have the cited package release or the original application code.

Neither the size nor the direction of the change can be inferred without recomputing the test. Let four units be given, two of them treated, and let the statistic be the difference in means at the assignment $(1,0,1,0)$. With fixed outcomes $(0,1,0,1)$, the two-sided tail probability is $1/3$ under free permutation and $1/2$ when one unit is treated in each of the pairs $\{1,2\}$ and $\{3,4\}$. With outcomes $(0,1/3,2/3,1)$, it is $2/3$ and $1/2$. No corrected $p$-value for the application is established, and a test under the documented design could still reject at the 5% level.

What survives

The comparison $-0.0321 < -0.0304$ is arithmetically correct, and the circle averages and the estimated curve of Figure 8 do not depend on how the permutations are generated. The sharp-null method, the option of the package for blocked designs and the toy example are not affected. What changes is the conclusion. The significance statement, and with it the inferential support for a bandwagon effect “up to 2.5 km”, is not established by the displayed call’s inspected defaults until the test is recomputed under the documented design. A failure of the spatial-HAC intervals is not shown.

The problem

The toy example has four nodes, two of them treated. The permutation distribution of footnote 5 consists of six assignments that are equally likely. It is the Bernoulli design of Section 4, conditional on two treated nodes. On such a set no nonrandomized test can reject at $\delta=0.05$, since a rejection region that contains a single assignment already has probability $1/6$. For a difference in means with two nodes in each group more is true. The complementary assignment reverses the sign of the statistic, so at least two of the six values are as extreme as the observed one, and the exact two-sided p-value is at least $2/6$.

The printed rejection arises because the percentiles are interpolated. For six values, the default R quantile at level $0.975$ equals $0.125$ times the second largest value plus $0.875$ times the largest, which is below the largest value whenever the two differ. The rule of p. 146 rejects when the estimate is larger than this percentile or smaller than its counterpart at level $0.025$. It therefore rejects whenever the observed assignment gives the largest or the smallest of six distinct values. Under the sharp null this event has probability $2/6$, not $0.05$, and enumerating every permutation does not change it. The sentence on p. 144 attributes the error of the test to the number of simulated permutations; for this rule the error remains when all permutations are used.

In the toy example the observed cumulative statistic on $[0.1,0.5]$ is $8.00$ (the printed $12.38$ is corrected in P008-E003). It is the largest of the six values and exceeds the interpolated upper percentile by about $0.02$. Its exact two-sided p-value is $2/6\approx0.33$, and the sharp null is not rejected at $\delta=0.05$. The sharp null is likewise not rejected at single distances. The estimates at $d=0.4,\dots,0.7$ lie above the blue curve of Figure 5, and each has exact two-sided p-value $2/6$; the estimates at $d=0.1,0.2,0.3$ lie below it. As a statement about the randomization test, the words “marginally significant” are not supported at the 5% level at any distance.

Evidence an exact counterexample

The evidence is an exact counterexample to the rule, a derivation of the bound on the p-value, and a recalculation of the toy example.

Counterexample. Let four nodes have circle averages $(0,1,2,3)$ at each distance $d=0.1,\dots,0.5$ and under every assignment, and let two of them be treated. Such outcomes are realized by a spatial field in which successive nodes on a line are 10 units apart and the outcome equals $i-1$ within distance 1 of node $i$ and $0$ elsewhere, whatever the assignment. This field satisfies the sharp null. For the six assignments, I find the following.

The interpolated percentiles at levels $0.025$ and $0.975$ are $-15/8$ and $15/8$. The rule rejects for two of the six assignments, and its probability of rejection under the sharp null is $1/3$. For the sum over the five distances $0.1,\dots,0.5$ every value is multiplied by five, the percentiles are $-75/8$ and $75/8$, and the same two assignments are rejected.

Bound on the p-value. Under the sharp null the circle averages $m_i$ do not depend on the assignment. For an assignment $z$ with two treated nodes, the estimator at one distance is

and for the complementary assignment I have $T(1-z)=-T(z)$. The same identity holds for a statistic $S$ that is a linear combination of the estimators at several distances. The exact two-sided p-value $\#\{w:\lvert S(w)\rvert\ge\lvert S(z)\rvert\}/6$ counts both $z$ and $1-z$, so it is at least $2/6$.

Recalculation. I generated the toy data as in the package vignette (seed 123567); the observed assignment treats nodes 1 and 2. The data reproduce the thirty printed entries of the columns AMR_est, Per.CI.l and Per.CI.u. The six values of the cumulative statistic on $[0.1,0.5]$ are $\pm7.99649$, $\pm7.82109$ and $\pm4.44342$, and the observed value is $7.99649$. The interpolated percentiles are $\pm7.97457$, in agreement with the printed $-7.97$ and $7.97$, and two of the six values are at least as extreme as the observed one. With the outcomes held fixed at the untreated raster, the rule rejects for two of the six assignments at each of the twenty distances. Both copies of the code in the archived repository of the package compute the percentiles with the R function quantile at its default setting. I did not have access to the release cited in the paper.

What survives

The definitions of the spatial potential outcomes and of the AMR, the toy point estimates and the curves of Figure 5 are unaffected. The cumulative statistic can still be used in a randomization test with a calibrated rule. The spatial-HAC intervals and the asymptotic claims are separate and are not addressed here. I find no additional error in the phrase “when $d$ is small”: it can refer to $d=0.4,\dots,0.7$, where the estimates do exceed the upper percentile. The hypothesis tested is the sharp null, which Section 2.3 distinguishes from the weak null.

The toy rejection is withdrawn, and the rule of p. 146 and the sentence on p. 144 are replaced. What is shown is a failure of calibration for a design with six possible assignments. The application of Section 6 uses the same rule, and its p-value has to be recomputed on its own assignment distribution; whether its conclusion changes is not established here.

The problem

The paper describes cutoff as a distance: pairs of nodes farther apart than cutoff contribute no covariance terms to the variance estimate. The later version of the package is the one distributed with the replication files of Wang, Samii, Chang and Aronow (2024). In its loop over the radii, the spatial-HAC standard error of the unsmoothed estimator is computed with the bandwidth cutoff + d whenever cutoff is positive. Here d is the index $1,2,3,\ldots$ of the loop, and the radius itself is dUp. The bandwidth for a radius therefore depends on the place of that radius in the grid of radii dVec, rather than on the radius itself.

Two consequences follow. The standard error of the estimate at a given radius can change when another radius is inserted before it in dVec, although the estimate and the data are the same. Further, the bandwidth grows by one unit of the distance matrix with each entry of the grid, whatever the unit is. In the units of the toy example it exceeds every distance between two nodes at the sixth radius, $0.6$, and the variance there is exactly zero under the default uniform kernel and edf = FALSE. When spatial-HAC intervals are requested with a positive cutoff, the function passes this index-based bandwidth without warning.

Evidence an exact counterexample

I checked an exact counterexample in rational arithmetic and the output printed on p. 149. The unchanged R function in the later version, supplied with the counterexample’s circle averages and distances and an R implementation of the uniform-kernel variance formula, returns the same bandwidths and variances.

Counterexample. Let four nodes lie at $(0,0)$, $(2,0)$, $(4,0)$ and $(6,0)$, with treatments $Z=(0,0,1,1)$, and let the outcome equal $0$, $2$, $4$ and $6$ on the discs of radius $3/10$ around them. The circle averages are $\mu=(0,2,4,6)$ at each of the radii $1/10$, $1/5$ and $3/10$. I take Euclidean distance, the uniform kernel, edf = FALSE and cutoff $=2/5$. The regression of $\mu$ on an intercept and $Z$ has coefficients $(1,4)$ and residuals $r=(-1,1,-1,1)$, and the estimate is $4$. The weights of the treatment coefficient are $a=(-1/2,-1/2,1/2,1/2)$. With the scores $g_i=a_ir_i$, that is, $g=(1/2,-1/2,-1/2,1/2)$, the variance with bandwidth $c$ is

where $D_{ij}$ is the distance between nodes $i$ and $j$. If $c<2$, only the terms with $i=j$ remain, and $V(c)=\sum_i g_i^2=1$. If $2\le c<4$, the three pairs of neighbors enter as well, and $V(c)=1+2(g_1g_2+g_2g_3+g_3g_4)=1+2(-1/4+1/4-1/4)=1/2$.

The printed output. Since the four nodes of the toy example lie in the square $[0,4]^2$ (Figure 2), two of them are at most $\sqrt{32}$ apart. The printed call has cutoff = .4 and radii $0.1,0.2,\ldots,2$, and it leaves the kernel at its default, the uniform kernel. At the sixth radius the rule of the later version gives the bandwidth $0.4+6=6.4>\sqrt{32}$, and the kernel includes every pair of nodes. Then the variance is $(\sum_i g_i)^2$, which is zero for any data, since the residuals of a regression on an intercept and the treatment sum to zero in each arm. The interval at $d=0.6$ would have width zero. The printed interval is $[-0.038,\,3.438]$, of width $3.476$, and I conclude that this rule did not produce the printed table.

What survives

The sentence on p. 145 is correct as a description of the method. The definition of the AMR, the unsmoothed point estimates and permutation tests do not use this line, and the line is skipped when cutoff = 0. I attribute no published interval to the defect: the release that the paper cites is not archived, the earlier archived version of the package passes cutoff unchanged, and the printed toy intervals are incompatible with the rule. The later version’s requested spatial-HAC standard errors and intervals of the unsmoothed estimator depend on this bandwidth. With edf = TRUE, so do those of the smoothed estimator and its confidence bands, which reuse the degrees of freedom computed in the same loop.

The problem

The cumulative statistic is the sum of the AMR estimates at the distances in the window, each with weight one (the summation is described on p. 146). Each AMR $\tau(d)$ is a difference of two circle-average probabilities of forest loss at distance $d$, averaged over villages and over the other villages' assignments, with the selected village treated and not treated. A sum of $m$ such differences need not be a difference of probabilities. For a probability defined by mixing these circle averages over distances, the change is a weighted average $\sum_j w_j \tau(d_j)$ with nonnegative weights that sum to one. The weights of the printed statistic sum to $m$. The two parentheticals read the sum and its null percentiles as percentage-point changes in “the cumulative probability of forest loss”, and the unit weights of the sum do not define such a probability. Figure 8 shows four estimates in the window, at 1, 1.5, 2 and 2.5 km. For this four-distance grid, dividing the printed sum $-0.0321$ by four gives a mean estimate of about $-0.0080$, an estimated reduction of 0.8 percentage points. The value 3.21 is the only effect size quoted for the application.

Evidence an exact counterexample

My evidence is a derivation and an exact counterexample: for loss probabilities $Q_b$ defined by drawing a village uniformly, setting its assignment to $b$, drawing a distance with probabilities $w_j$, and using the paper’s circle-average measure, with other assignments averaged over the design, I obtain

For the counterexample, I take four village centres at least 20 km apart and 40000 points on each circle at distances 1, 1.5, 2 and 2.5 km, with independent assignments of probability $1/2$ and binary outcomes defined on all 640000 circle points and all four centres for every assignment. At the assignment $(1,1,0,0)$, each AMR and its Hajek estimate equal $-0.008025$, giving sum $-0.0321$, while the constructed pooled loss probabilities are $0.0175$ under control and $0.009475$ under treatment, a change of $-0.8025$ percentage points. Reading the sum as this probability change would give $0.0175 - 0.0321 = -0.0146 < 0$ under treatment; a companion binary model on the same spatial field gives sum $-4$ and probability change $-1$, ruling out a general conversion of the sum into a probability change.

What survives

The AMR, the Hajek estimator and the use of a fixed sum of AMR estimates as the statistic of a sharp-null test are unaffected by this interpretation error. Dividing the statistic and every permutation value by the same positive number preserves ranks and the printed comparison $-0.0321 < -0.0304$, provided that the statistic and the percentiles use the same grid; the original application percentiles and exact $p$-value have not been independently reproduced. The summed AMR statistic does not represent a 3.21-percentage-point reduction in a probability of forest loss; the grid in the printed code has no distance inside the window, which is the subject of P008-E004.

The problem

Section 3 (p. 146) says that SpatialEffectTest() “generates the null permutation distribution of the sum of the AMR estimates within the specified range”. The “actual cumulative effect” for the range $[0.1,0.5]$ is therefore the sum of the estimates at $d=0.1,\dots,0.5$. The five printed estimates sum to $7.996$, not to $12.38$. Note that the percentiles $-7.97$ and $7.97$ do belong to $[0.1,0.5]$. The sentence compares the percentiles of one range with a number that matches the observed sum on a wider range. It suggests a margin of about $4.4$ above the upper percentile, and the margin is $0.02$.

Evidence a historical Lean attribution (Generated status: clause support not certified in this release)

The evidence is a derivation from the printed table and a recalculation of the toy example. From the printed estimates, I have

Each printed estimate is within $0.0005$ of its unrounded value, so the unrounded sum lies in $[7.9935,\,7.9985]$. A number printed as $12.38$ lies in $[12.375,\,12.385]$, and the two intervals are disjoint. Adding the next three estimates, I obtain $7.996+1.700+1.678+1.002=12.376$, which rounds to $12.38$. A sum multiplied by the step $0.1$ of the grid would be $0.7996$.

For the recalculation, I generated the toy data as in the package vignette (seed 123567) and enumerated the six assignments with two treated nodes. These data reproduce the ten printed estimates and their permutation bounds, and give the following.

The SpatialEffectTest() function in the archived repository of the package returns the same values. The release of the package cited in the paper and the script that produced the number are not available. The agreement with the sum through $d=0.8$ is a numerical fact and does not show how $12.38$ arose.

What survives

The ten AMR estimates, their permutation bounds and the percentiles $-7.97$ and $7.97$ stand, and no later result is computed from this number. The corrected sum still exceeds the upper percentile, by $0.022$. The conclusion drawn from that comparison on p. 150, a rejection at $\delta=0.05$, fails for a separate reason given in P008-E002.

The problem

The printed grid seq(from=0, to=20/111, by=5/111) has five points, at 0, 5, 10, 15 and 20 km. The test range is $[1/111,\,2.5/111]$, that is, 1 to 2.5 km, and none of the five points lies in it. By Section 3 (p. 146), SpatialEffectTest() sums the AMR estimates at the grid points within the range. With the printed grid the observed sum and every permuted sum are empty sums, equal to zero, and the reported statistic $-0.0321$ and percentiles $-0.0304$ and $0.0336$ cannot come from the printed commands.

Figure 8 shows the grid that was used. Its unsmoothed curve has 41 points at a spacing of 0.5 km, and the four estimates at 1, 1.5, 2 and 2.5 km sum to $-0.0321$. The step $5/111$ is a misprint for $0.5/111$, in the text and in the code.

Evidence an exact counterexample

The evidence is a derivation and a check of Figure 8. The printed grid is $d_k=5k/111$ for $k=0,\dots,4$. The condition $d_k\in[1/111,\,2.5/111]$ is equivalent to $1\le 5k\le 5/2$, which fails for $k=0$ and for every $k\ge 1$. The SpatialEffectTest() function in the archived repository of the package, run with the printed grid on an example with four nodes and all six assignments enumerated, returns an observed sum of $0$ and both percentiles equal to $0$.

In the PDF, the black line of the Hajek panel of Figure 8 is a polyline with 41 vertices. Their horizontal positions agree with $0,0.5,\dots,20$ km to within $0.0002$ km. The values at the four vertices in the test range, read from the axes of the figure, are as follows.

The sum rounds to the reported $-0.0321$. These values are read from a published graphic and are not a recalculation from the data of the application, which are not available.

What survives

No reported number or conclusion changes: Figure 8 and the reported cumulative statistic agree with a grid of step 0.5 km. The correction concerns the grid only. The percentiles $-0.0304$ and $0.0336$ and the statement $p<0.05$ could not be recomputed and are not certified by it. The units of the cumulative statistic are the subject of P008-N016, and the layout of the printed call that of P008-E009.

The problem

Since the names in parentheses in Section 3 are arguments of SpatialEffect(), the sentence says that conley.se sets the level $\delta$. The next page says that Conley.SE and Conley.CI are returned “if conley.se equals 1”, and in both archived versions of the package the level is the argument alpha, with default 0.05. In those versions, setting conley.se = 0.05 skips the unsmoothed spatial-HAC output.

Evidence a check of the printed page

I checked the page against p. 146 and against the argument lists and manuals of both archived versions of the package. The condition conley.se == 1 in the source is false at the value 0.05, and the block that computes the unsmoothed spatial-HAC standard errors is then skipped. The release that the paper cites (version 0.1) is not archived, and I did not examine it.

What survives

Users can change the level, as the sentence says, and the first (conley.se) in the paragraph is correct. The two printed calls (pp. 148 and 153) set neither argument, and no reported number is affected.

The problem

Since the band is the set over which outcomes are averaged, it consists of points $x$ at which outcomes are defined. Section 2.1 indexes these points separately from the intervention nodes $i$, and the words “of intervention nodes” name the wrong set.

Evidence a check of the printed page

I checked the page against the definitions on p. 141 and the circle-average display on p. 143, which integrates $Y_x(\mathbf{z})$ over the points $x$ such that $d_i(x)=d$, for one intervention node $i$.

What survives

The formulas and the computations use outcome points, and nothing depends on the phrase. The width of the band is the subject of P008-N001.

The problem

A verb is missing on p. 140. On p. 144 two words are out of order, a preposition is missing and an article is misplaced. On p. 150 and p. 153 one word is wrong. The context fixes the intended meaning in each case.

Evidence a check of the printed page

I checked the four pages.

What survives

Nothing depends on these phrases. The correction on p. 144 repairs the wording only. Whether the test described there is exact in finite samples is the subject of P008-E002.

The problem

Section 3 (p. 145) says that for polygon interventions the treatment status comes from the attributes of the polygon object ras_Z. The call on p. 153 accordingly passes ras_Z and treatment, and no Zdata. The paper cites version 0.1 of the package (Samii and Wang 2020), which is not available. The archived repository of the package holds two copies of the code, an older one and a later one, and only the later one accepts polygons. Its SpatialEffect() first tests whether Zdata is NULL, which is the default, and stops with “No treatment specified” before it reads ras_Z. Deleting the test does not repair the function. The number of nodes and the treatment vector, nz and Zup, are set only from Zdata, and without them the function stops with “object 'nz' not found”.

There are two smaller points. First, both printed calls enclose strings in pairs of apostrophes, which R does not parse; ordinary quotation marks are meant. Second, for point interventions the default number of sampling points on a circle of radius $d$, for a raster whose smallest cell dimension is $r$, with the default $\texttt{evalpts}=1$, is in both archived copies

which grows in proportion to the radius. On the toy grid $d=0.1,0.2,\dots,2$ ($r=1$), it is 7 for $0.1\le d\le 0.7$, 10 for $0.8\le d\le 1.4$ and 13 for $1.5\le d\le 2$, and it would be 450 at $d=100$. Whether 7 to 13 is “roughly the same number” is a matter of wording, but as a description of the default the sentence on p. 145 is not accurate.

It is not known whether the call ran as printed with the release used for the paper. No reported number is affected.

Evidence an exact counterexample

The evidence is a run of the code and an exact computation. I called the SpatialEffect() function of the later copy, unchanged, with the printed arguments and ordinary quotation marks, on two square polygons with treatment attributes 0 and 1 and a 4-by-4 raster. It stops with “No treatment specified”. A copy of the function with the test deleted stops with “object 'nz' not found”. The R parser rejects each of the five printed strings (’’x’’, ’’y’’, ’’Z’’, ’’treatment’’ and ’’Geodesic’’) and accepts them with double quotation marks. Computed in exact arithmetic from the formula above, the counts are 7, 10, 13 and 450, and they agree with the counts used by both copies. The manual page of the package also describes the default as proportional to the radius.

What survives

The definitions, the estimators and every reported number are unaffected. The toy call supplies Zdata and is not stopped by this test once the quotation marks are corrected; its argument smooth = 1 is the subject of P008-N007. I did not make a complete run of the corrected polygon call on geographic data.

The problem

The two averages stand side by side with no sign between them, and as printed the display reads as their product. The second average is written with $\bar{\mu}_i$, although the page defines only $\mu_i(z;d,\alpha)$. The sentence after the display uses a set $\mathcal{N}$ that is not defined. By p. 141 it can only be the set $\{1,\ldots,N\}$ of indices of the intervention nodes.

Evidence a derivation

I derived the identity from the definitions that precede the display. On p. 143 the paper sets $\tau_i(d;\alpha)=\mu_i(1;d,\alpha)-\mu_i(0;d,\alpha)$. Substituting this into the first average, I obtain

and the two averages are equal. The missing sign is an equals sign. The product is a different quantity. For $N=2$ with $(\mu_1(1),\mu_2(1))=(3,7)$ and $(\mu_1(0),\mu_2(0))=(0,2)$, each average equals $4$ and their product is $16$. The estimator on p. 144 is written with $\mu_i$ without a bar.

What survives

The AMR as defined in the sentences before the display is unchanged, and so is the Hajek estimator on p. 144. Only the display and the index set after it change.

The problem

The assignment to dVec and the assignment to AMRest are printed on one line with no separator, the function name is broken into Spatial and Effect at the line end, and the strings are delimited by pairs of typographic apostrophes. R stops at AMRest. With the two assignments separated and ordinary quotes restored, R still reads AMRest <- Spatial followed by a separate call to Effect().

Evidence an exact counterexample

The evidence is a check of the page and a parse of the printed commands in R, which stops with an “unexpected symbol” error at AMRest. The corrected commands parse as the two intended expressions. No estimation was run.

What survives

The intended commands and the surrounding explanation are unaffected; this is a repair of syntax only. It does not address the grid of distances in the same call, which is the subject of P008-E004.

The problem

The first entry gives the coauthors Dean Eckles, Cyrus Samii and Stephanie Zonszein by given name and an initial of the surname. The second splits the single author Erin O. Sills into “Erin O” and “Sills”. The third, and the citation on p. 140, treat “Elizabeth” as part of the surname of M. Elizabeth Halloran.

Evidence a check of the printed page

The evidence is a check of the pages against public author records: the author list of the cited arXiv preprint, the publication record of the Land Economics article and the curriculum vitae of E. O. Sills, and the PubMed record of the article of Hudgens and Halloran together with the curriculum vitae of M. E. Halloran. The journal versions of record of the last two articles were not read.

What survives

The three works remain identifiable from their titles, years and identifiers, and no result of the paper is touched.

The problem

The two sentences describe the band and the disc inexactly, and the description differs from what the available code does.

First, the points within distance $k$ of the circle of radius $d$ are the points between $\max(0,d-k)$ and $d+k$ from the node. The endpoints on p. 143 are correct when $d\ge k$, but this band is $2k$ wide. Then $k$ is its half-width and not its width.

Second, $d$ is a radius and not a place, and a sector would need an angle, which neither the text nor the package specifies. The “disk” is the full disc of radius $d$ around the node. For a polygon it is the polygon together with all points within distance $d$ of it.

Third, the only archived version of the package that has the option cType is the one distributed with the replication files of Wang, Samii, Chang and Aronow (2024), which is later than the release that the paper cites (version 0.1, 2020). This later version has no argument $k$. For polygon interventions, its “donut” at the $j$th radius $d_j$ of dVec, $j>1$, averages the outcome cells that lie between the buffers of radii $d_{j-1}$ and $d_j$ around the polygon, and at the first radius it averages the full buffer. Such a band ends at $d_j$ and is not centered there. Someone who follows the text would place a donut estimate half a band away from the region it averages. The release that the paper cites is not archived, and I could not determine whether it behaved in the same way.

Evidence an exact counterexample

I derived the width of the band and ran an exact example through the function SpatialEffect() of the later version.

Width. Let $r$ be the distance of a point $x$ from the node. By the reverse triangle inequality, every point of the circle of radius $d$ is at distance at least $|r-d|$ from $x$, and a radially aligned point of the circle attains this bound. This implies that the points within distance $k$ of the circle are the points such that $\max(0,d-k)\le r\le d+k$. For $d=4$ and $k=1$ the band is $[3,5]$, of width $2$.

Example. Let the treated polygon be a small square, let dVec be $(2,4)$, and let the only outcome cells near the square lie at distances $0$, $19/8$, $27/8$ and $35/8$ from its boundary, with outcomes $4$, $2$, $0$ and $8$. Let all outcomes near the control polygon be $0$. The buffer of radius $2$ contains one cell, with sum $4$, and the buffer of radius $4$ contains three cells, with sum $6$.

The function returns $1$ for the donut and $2$ for the disk at the radius $4$, in agreement with the first and third rows, and not the value $4$ of the centered band.

What survives

The corrections of wording change no reported number. The definition of the AMR on exact circles, the estimator and the toy example do not use these options. The bottom panel of Figure 8 uses the donut option, and where its points are drawn is the subject of P008-N017.

The problem

Section 3 asks for a cutoff beyond which interference dependence does not exist. In the toy example the four nodes are between $1.43$ and $3.40$ coordinate units apart, and the simulated effect of a node does not vanish over these distances: at distance $2$ it is $-0.58$. The cutoff 0.4 is below the smallest distance between two nodes, so no pair of distinct nodes enters the variance estimator. The estimator then reduces to the heteroskedasticity-robust (HC0) variance of a difference in means, and all ten printed intervals equal the HC0 intervals to the three decimals shown.

The example presents these intervals as spatial-HAC intervals, in a tutorial on inference under interference, but they contain no adjustment for dependence between nodes. That dependence is present in the reconstruction below. Under the Bernoulli design of the example, the contributions of two distinct nodes to the estimator have positive covariance for every pair of nodes at $d=0.1$, and the sum of these covariances is positive at each of the 20 radii. The rule on p. 145 is correct as written, and the defect lies in the example. A user who chooses cutoff as the example does leaves these covariances out.

Evidence an exact counterexample

The evidence is a recalculation of the toy example from the vignette distributed with the package, and an exact counterexample for the general point.

Recalculation. Starting from the seed and the recipe of the vignette, I obtain the ten printed estimates and the twenty printed endpoints of the intervals, to three decimals. The smallest distance between two nodes is $1.4295>0.4$. With the uniform kernel, the spatial-HAC variance of the estimate is $\sum_{i,j}q_iq_je_ie_j\mathbf{1}\{D_{ij}\le 0.4\}$, where $q=(1/2,1/2,-1/2,-1/2)$, $e_i$ is the residual of the circle average of node $i$ from the mean of its arm, and $D_{ij}$ is the distance between nodes $i$ and $j$. Only the terms with $i=j$ remain, and the variance is the HC0 variance $\sum_i q_i^2e_i^2$.

The covariances left out. Since the effects are additive in the toy example, the circle average of node $i$ is $\mu_i=b_i+\sum_k a_{ik}Z_k$, where $a_{ik}$ is the average effect of node $k$ on the circle of node $i$. Let $m_1$ and $m_0$ be the averages over the nodes of the marginal circle means under treatment and under control, and let $\phi_i=2Z_i(\mu_i-m_1)-2(1-Z_i)(\mu_i-m_0)$ be the contribution of node $i$ to the linearized Hajek estimator. Under independent Bernoulli($1/2$) assignment, a direct computation gives

At $d=0.1$ the twelve coefficients $a_{ij}$ with $i\ne j$ are negative, which implies that the six covariances are positive. For nodes 3 and 4, which are $1.43$ apart, I have $a_{34}=-0.9235$ and $a_{43}=-1.1507$, and the covariance is $1.0627$. The variance of the mean of the four $\phi_i$ at $d=0.1$ is $1.4946$ with the covariances and $1.2680$ without them. The part left out is positive at each of the 20 radii, between $0.013$ and $0.227$. Note that these are variances of the linearization under the design, and not the finite-sample variance of the estimator with four nodes. I make no claim about the coverage of the printed intervals.

Counterexample for the general point. A cutoff equal to the range over which a node affects outcomes can be too small. Let two nodes lie at $(0,0)$ and $(2,0)$, let the outcome points be $(-1,0)$, $(1,0)$ and $(3,0)$, and let a treated node raise by $1$ the outcome of every point within distance $1$ of it. Under independent Bernoulli($1/2$) assignment and for $d=1$, the circle averages are $\mu_1=Z_1+Z_2/2$ and $\mu_2=Z_2+Z_1/2$, and the AMR is $1$. With $S_k=2Z_k-1$, I obtain $\phi_1=\phi_2=S_1S_2/2$, so that $\operatorname{Var}(\phi_i)=\operatorname{Cov}(\phi_1,\phi_2)=1/4$. The variance of $(\phi_1+\phi_2)/2$ is $1/4$, and the terms with $i=j$ contribute $1/8$. A cutoff of $1$ excludes the pair, which is $2$ apart, and leaves out half of the variance. The same ratio holds for any number of such pairs placed far from one another.

What survives

The rule on p. 145 is correct, and this cutoff correction leaves the estimates, the permutation bounds and the printed numbers unchanged. The toy intervals are nominal HC0 intervals, and they can no longer be read as adjusted for interference. I have not established an error in the application’s intervals; I could not obtain the package release and the exact transformed inputs used for them. Nor have I established an error in the asymptotic result that the paper attributes to Wang et al. (2022). The application bases its statement of significance on a permutation test, which does not use the cutoff.