My comments

P001-C03-E001 Moderate Finite variance does not make the population mean well behaved in the sense of footnote 31; the asymptotic normality of the sample mean follows from the Central Limit Theorem (Theorem 3.2.24), not from Hadamard differentiability with respect to the Kolmogorov distance.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P001-C04-E001 Moderate The conditions $J_n\to\infty$ and $J_n/n\to0$ do not imply that the polynomial least squares sieve estimator is consistent; the claim holds under a stronger condition on $J_n$, which for uniform $X$ reads $(J_n+1)^3/n\to0$.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P001-C05-N005 Moderate When only some of the component models of a mixture are needed, the parameters of the unused components are not identified, and the ML estimator of the full parameter vector is consistent and asymptotically efficient only under identification and regularity conditions that the sentence does not state.

This is sufficiently ambiguous that I think it is worth classifying as a moderate error for reader attention.

1. Classification before this comment: Moderate error.
2. I commented on 6 November 2026.
3. Author comment. The Moderate classification is retained.

P001-C06-E007 Moderate Footnote 15 should not say that bootstrap standard errors are likely to be adequate for hot deck imputation: for one-to-one, nearest-neighbor hot deck imputation, the bootstrap variance can be inconsistent even when the estimator has no bias.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P001-C07-A001 Clarity note In Definition 7.1.19 the derivative is taken in the treatment level $d$ of the potential outcome $Y_i(d)$, with the conditioning values held fixed, and is then evaluated at $d=D_i$; the printed formula writes $D_i$ for both.

I think this is worse than represented. This expression -- even once corrected is only equivalent to the AMCE under ignorability. The definition should not condition on D,X; rather it should be a separate theorem.

Suggested classification: Moderate error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): clarity note.
2. I commented and suggested a Moderate error.
3. Independent evaluation. Not adopted: the entry stays a clarity note, not the Moderate error I suggested.

P001-C07-E007 Moderate The $F$-test for adding the placebo variables $\mathbf{W}_i$ to a regression of $D_i$ on $\mathbf{X}_i$ can reject $\mathbf{W}_i\perp\!\!\!\perp D_i\mid\mathbf{X}_i$ with probability tending to one even when it holds.

Yes, this should be clarified.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: the entry already gives that clarification.

P001-C07-E010 Moderate Coding all the information in the applicant descriptions does not make gender conditionally independent of the potential outcomes by definition; conditional independence given $\mathbf{X}_i$ remains a substantive assumption.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P001-C07-E011 Moderate For nearest-neighbor matching the bootstrap can fail through its variance even when the estimator has no bias, so the assurance that the bootstrap standard errors “are likely to adequately characterize sampling variability” is unsupported.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P006-E001 Major Because the rewards $w_{\mathrm{ATO}}$ and $w_{\mathrm{ATT}}$ are zero when nobody is treated, their expected value is $[1-(1-q)^n]A/q$, where $q=E[\pi(X)]$, $A=E[s(X)\pi(X)]$ and $s$ is $\mu_1$ or $\tau$; the rule printed as the agent’s best response (treat only the values of $x$ with the largest positive $s(x)$) need not maximize it, and on the Horse Colic data the best response to $w_{\mathrm{ATT}}$ keeps 48.03% of the achievable welfare, where the printed rule keeps 5.34%.

I agree.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented.
3. Independent evaluation. My comment agrees with the entry; no change.

P008-N012 Major The statement “significant with $p < 0.05$” is not established under the design of the experiment: in the inspected implementations, the printed call permutes treatment freely over the 121 villages, but treatment was assigned by seven subcounty lotteries between lists of villages balanced on covariates.

I agree. This needs further investigation -- the correct reference distribution should be used here.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented.
3. Independent evaluation. My comment supports further investigation; no change.

P010-N011 Moderate The estimated bounds do not overlap across all treatment conditions for both outcomes: for “prefer nuclear use” the bounds for 90–90 lie wholly below those for 90–70 and for 90–45, in both figures.

I think this is a minor expositional error -- the plots make it clear.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Not adopted: the entry stays a Moderate error, not the Minor rating I suggested.

P011-E001 Critical Under Assumption 2 the oracle estimator satisfies $n\widehat V_1(A^{\mathcal O};Z)-n\operatorname{var}(\hat\beta)=n^{-1}\delta^{\mathsf T}(I+A^{\mathcal O})\delta+o_p(1)$ with $\delta_i=\mathbb{E}[\theta_iY_i]-\beta$, and the term on the right can be negative; Proposition 1, Corollaries 1 and 3 and the sentence after Assumption 3 therefore hold only under a further condition on the unit means, on the graph, or on the known edges and remaining degree capacity.

I investigated this, in part with help from an LLM. I agree, but want to suggest a more broad type of correction. The following is LLM-generated output, generated at my supervision:

An additional condition for conservative inference

Retain Assumptions 1–2. Write $X_i=\theta_iY_i$, and let $A_R$ be the known-edge adjacency matrix. Choose fixed integer caps $\deg_{A^{\mathcal O}}(i)\le h_i\le\min\{d_i,n-1\}$, with $\max_i h_i\le D<\infty$; the original maximization may retain larger supplied caps. Define

$$r_i=\sum_j(A_R)_{ij},\qquad s_i=h_i-r_i,\qquad K_R=I+A_R+\operatorname{diag}(s_i).$$

Thus $s_i$ is the capacity left for unknown edges. Let $v$ be the variance inputs, and optionally add a fixed correction $Q\succeq0$, with $\|Q\|_{\mathrm{op}}=O(1)$:

$$\widehat V =\frac1{n^2}\max_{A\in\mathcal A(Z)}v^\top(I+A)v +\frac1{n^2}v^\top Qv.$$

For the published estimator, $v=X-\hat\beta\mathbf1$ and $Q=0$. Regression alternatives below change the variance inputs and retain the original point estimator.

Additional condition. Write $v=X-E[X]+\delta+a$, where $\delta$ is deterministic, $\|\delta\|^2=O(n)$, $\|a\|^2/n\to_p0$, and $\max_i v_i^2/n\to_p0$. Require

$$\boxed{\displaystyle \liminf_{n\to\infty}\frac{\delta^\top(K_R+Q)\delta}{n}\ge0.} \tag{C}$$

For published centering, $\delta=E[X]-\beta\mathbf1$, and the other requirements follow from Assumption 2. A sufficient spectral certificate, when $\delta\in\operatorname{col}(U)$ and $U^\top U=I$, is

$$\lambda_{\min}\{U^\top(K_R+Q)U\}\ge-\varepsilon_n, \qquad \varepsilon_n\to0.$$

Condition (C) gives asymptotic variance conservativeness and at-least-nominal Wald coverage. It restricts the relevant mean directions; it does not require the full matrix's negative eigenvalues to vanish.

The following are sufficient routes. The first two use published centering, whose mean deviations are uniformly bounded.

• Few known edges. The number of known edges is $o(n)$. This includes degree-only constraints, with no known edges.
• The known graph is nearly a union of disjoint cliques. Deleting $o(n)$ known edges leaves vertex-disjoint clique components. A simple sufficient case is that only $o(n)$ known edges belong to components that are not cliques. The remaining clique edges may number $O(n)$, and their degree caps need not be saturated. Exact disjoint cliques are a special case; overlapping cliques alone do not suffice.

• Enough remaining degree capacity. For every vertex, $h_i\ge2r_i-1$. Then

  $$K_R=A_R+\operatorname{diag}(r_i) +\operatorname{diag}(1+h_i-2r_i)\succeq0.$$
• A compatible mean space. The possible mean deviations lie in a specified space $\operatorname{col}(U)$, and $U^\top K_RU$ is positive semidefinite, or its minimum eigenvalue tends to zero from below. Common transformed means, $E[X_i]=\beta$, give $\delta=0$.

• An explicit correction is added. One universal choice is $Q=(-K_R)_+$, the negative spectral part of $K_R$ with its signs reversed. Alternatively, choose a known-edge set $F$ whose deletion leaves vertex-disjoint cliques, and add

  $$\frac1{n^2}\sum_{\{i,j\}\in F}(v_i-v_j)^2.$$

  This uses $Q=L_F$, the Laplacian of the deleted edges; $F$ need not be $o(n)$. Both corrections are added after maximization and preserve the original maximizing graph.

• Residuals from a well-specified regression. Suppose $E[Y_i]=f_i^\top\gamma$, with nonrandom regressors. Fit by $\theta_i^2$-weighted least squares and use $v_i=\theta_i(Y_i-f_i^\top\hat\gamma)$. It is enough that the transformed design $W$, with rows $\theta_i f_i^\top$, has rank $o(n)$; a fixed number of regressors suffices. The fitted weighted means have average squared prediction error tending to zero, so (C) holds with $\delta=0$.

• Residuals from a nearly well-specified regression. Use the same fit, require $\operatorname{rank}(W)=o(n)$, and assume

  $$\inf_\gamma\frac1n\sum_i\theta_i^2 \{E[Y_i]-f_i^\top\gamma\}^2\longrightarrow0.$$

  The remaining weighted mean error has $\|\delta\|^2/n\to0$, which implies (C). More generally, either residual case works whenever

  $$\frac1n\sum_i\theta_i^2 \{\hat m_i-E[Y_i]\}^2\to_p0.$$

The regression cases also give consistency of the true-graph residual oracle. The maximized bound is conservative and need not be consistent. Correct specification alone is insufficient for arbitrary unweighted fitting under the paper's assumptions.

Why this works. Proper-color the known graph, then form sorted cliques separately within each color and remaining-capacity class. This gives a compatible graph and establishes

$$n\widehat V\ge\frac1n v^\top(K_R+Q)v-o_p(1).$$

Moreover, $K_R-(I+A^{\mathcal O})$ is the Laplacian of the unknown true edges plus a nonnegative diagonal matrix. Its covariance contribution is therefore nonnegative; (C) controls the remaining mean contribution. The covariance/mean separation is inspired by Yap's Theorem 2.

Updated after LLMs rendered their decision.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Critical error.
2. I commented.
3. Independent evaluation. Incorporated. The correction was broadened to include the additional condition in my comment and the other sufficient routes to conservative inference that the independent check established. The entry remains Critical: these conditions do not restore the paper’s original unrestricted guarantees.

P012-E001 Moderate Line 22 of the edge proposal lies outside the conditional opened at line 13, so the listing as printed can return a graph outside the compatible set; moving line 22 inside that conditional, with two boundary rules, repairs it.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P012-N007 Moderate The extrema of $h$ and $p$ over the compatible set have closed forms that are computed in one pass over the data; the opening sentence of Web Appendix 1 says that there are none.

I think this is actually a major correction, because it obviates much of the motivation of the paper. People should know there is a closed-form solution.

Suggested classification: Major error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Major error.
3. Independent evaluation. Incorporated in part. The entry now says that, for these four endpoints, the closed-form formulas replace the search in the appendix and the convergence argument, and shorten the proofs of Results 2 and 3. My suggested rating (Major) was not adopted.

P013-E001 Moderate Proposition 6.2 needs a further condition, that the positive joint exposure probabilities that enter the variance estimator of equation (11) are bounded away from zero; Conditions 3, 5 and 6 alone do not imply the stated coverage.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P013-E003 Moderate The bound of Proposition 4.1 holds for potential outcomes of one sign with nonzero total; for general nonzero totals, multiplying its right-hand side by $\sum_i|y_i(d_k)|/|y^T(d_k)|$ gives a valid bound, and the rule for the number of replicates changes in the same way.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P013-N012 Moderate The sampler for Bernoulli designs in the later package rejects every assignment vector that repeats an earlier one, and the exposure probabilities are then computed with equal weights on the vectors kept, which distorts them (for $N=100$, $p=0.01$ and $R=1000$ the estimated probabilities of treatment average at least $0.01898$, against $0.01$ under the design); the sampler should keep every independent draw.

This was not a problem with this paper, but rather the later software package.

Suggested classification: not an error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested that this is not an error.
3. Independent evaluation. Incorporated in part. The entry now attributes the defect to the later package, gives the package’s full list of authors without assigning responsibility for the defect, and says that its use for the paper’s original calculations is not established. My suggestion that this is not an error was not adopted.

P013-N020 Moderate In a reconstruction on the public data, all twelve point estimates of Table 2 are reproduced with 342 of 2,050 students who have no recorded general wristband answer kept and scored zero; the article should state whether it uses this recorded orange-wristband outcome and disclose its missing-answer rule.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P014-E001 Moderate The difference-in-means estimator is unbiased for the probability-weighted average direct effect, in which $\overline{Y}_i(z)=\mathrm{E}[y_i(\mathbf{Z})\mid Z_i=z]$, and not in general for the target printed on p. 131, which weights the assignment vectors in $\operatorname{Supp}(\mathbf{Z})$ equally.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P015-E006 Moderate A regression weight that is small but positive does not show that positivity fails or that a representative effect cannot be estimated, and for a treatment with more than two values the effective sample need not satisfy positivity for a chosen pair $(d,d')$; an exactly zero population conditional variance does rule positivity out.

I think this is too literal a reading of the manuscript. I think it's clear from context what is meant.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Not adopted: the entry stays a Moderate error, not the Minor rating I suggested.

P015-E008 Moderate Substituting another inverse link into the logistic estimating equation of Appendix C does not yield the maximum-likelihood equations of probit and the other listed models, and the argument that carries the OLS weights over to nonlinear models holds as printed only for models whose likelihood score is a constant multiple of the equation so obtained, such as logistic regression and Poisson regression with the log link.

Agreed.

Suggested classification: Moderate error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Moderate error.
3. Independent evaluation. No change was needed: my comment agrees with the entry and its Moderate rating.

P017-E001 Moderate Proposition 1 fails when the target of Definition 2 is an endpoint of the support of $\tau$ that has probability zero; with a fixed clipping radius $0<d<c$ in place of $c$, the target is a nonnegative weighted average of causal effects for every empirical distribution.

Agreed. I think this is a fairly minor regularity condition, but it's worth flagging.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed. The entry remains a Moderate error: the printed target can fail to be a weighted causal effect, although the numerical results stand and the repair is routine.

P018-E003 Moderate Corollary 2 needs the weights to be bounded after a common rescaling and the rescaled weighted design to have a positive definite limit; weights that are only finite and fixed do not suffice. Corollary 1 also needs uniformly bounded $T(d)$, a positive definite limiting design matrix and a condition on products of dyad-score means when record kinds follow different laws.

Updated after the LLMs rendered a decision. I think this is mostly a regularity condition in context, but I'm fine classifying it as moderate.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Minor error.
2. Changed by a later check, before my comment was evaluated: Moderate error.
3. Revised. The repeated-data correction was expanded while the classification was still Minor. When record kinds follow different laws, the stacked normal equations center the total score but need not center each dyad’s score. The correction therefore adds a condition requiring the normalized sum of products of score means over overlapping dyads to tend to zero. The estimator and the score-covariance target are unchanged. P. M. Aronow delegated this decision to the model session that runs the audit, which adopted the amendment recommended by the two checks; this note was written by that session.
4. Revised. Later, the model session running the audit raised the classification from Minor to Moderate, after P. M. Aronow suggested Moderate and left the final decision to that session. The additional condition on score means is substantive when different record kinds follow different laws, a reading supported by p. 567; it holds automatically when every record follows one common law. The correction, estimator, cross-sectional result and weighted corollary are unchanged. This note was written by that session.
5. I commented.
6. Independent evaluation. My comment accepts the current Moderate classification; no change.

P023-E001 Moderate Full-sample OLS with the coarsened control of Proposition 1 relies solely on outcomes from close elections; a polynomial regression can estimate the same jump at the threshold while also using distant outcomes.

I disagree. I think this is a wording issue. Below is a response from a conversation with an LLM.

The literal claim that finite-degree polynomial OLS uses solely close-election outcomes is false. That fact is a property of the design weights, not a rare outcome realization. With residualized treatment r, the coefficient is r'Y/(r'r); a remote outcome with r_i != 0 enters for every realized outcome vector. The exact House-data certificates were independently rerun during this review and passed for both specifications and both sample conventions.

The handpicked outcome vectors in the original illustration can themselves be rare. They demonstrate data dependence and do not measure the frequency of a substantively large OLS–RD discrepancy. Under the exactly specified model and shared-consistency assumptions used in E001, for every fixed positive tolerance epsilon, P(|OLS - RD| > epsilon) tends to zero. Nonzero remote weights do not contradict that conclusion.

On the full-paper reading, the explicit separation of exact coarsened equivalence from approximate polynomial equivalence, and Appendix B's common-cutoff-target argument, support treating the “solely” sentences as loose wording. E001 does not establish a failure of those equivalence results, a changed application estimate, or different causal bias or external validity. In particular, the finished block's claim that the unqualified transfer of concerns “falls with it” exceeds what its weight calculation proves: concerns arising from the common cutoff target and its identifying assumptions can still be shared.

Recommendation: reclassify E001 as a clarity note with a targeted wording correction. Major is not justified by E001 alone. This reassessment is saved separately; canonical ratings, validation records, and generated publication files have not been changed by this chat.

Suggested wording: “The binned-control OLS estimate equals the close-election RD estimate. An exactly specified polynomial regression estimates the same cutoff jump asymptotically while also using distant outcomes. Concerns arising from that cutoff target and its causal interpretation can apply to both approaches.”

This recommendation is limited to E001. The separate approximation-condition issue in E002 is unaffected.

Suggested classification: clarity note.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented and suggested a clarity note.
3. Independent evaluation. Incorporated in part. The rating was lowered from Major to Moderate, the claim that the concerns shared with close-election studies fall with this point was withdrawn, and the entry was redrafted. My suggested classification (clarity note) was not adopted.

P026-E001 Moderate A sufficient condition for fitting $\theta_i$ from other units' outcomes while preserving unbiasedness is independence of $T_i$ from the whole vector of assignments used in the fit; independence from each assignment separately does not suffice.

Yes, agreed. This was informal about pairwise vs joint independence.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed. The independent check upheld the six-unit counterexample, and the entry remains a Moderate error.

P027-N002 Clarity note Totalling units that are always selected together need not reduce the bias of the corrected variance estimator: with $a_{kl}=b_{kl}=2$ it can raise the bias when the clusters differ in size, and it cannot raise the bias when every two clusters with zero joint inclusion probability have the same size.

"tend to" is deliberately heuristic. I believe this to be minor.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Incorporated in part. The heuristic reading was accepted, and the entry was reclassified from a Moderate error to a clarity note, which goes further than the Minor rating I suggested.

P027-N005 Moderate The $\widehat{\operatorname{Var}}_{C2}$ results of the periodic simulation (Table A3 and Figure A3) are for the estimator applied to the original units, without the totalling of always-co-selected units that p. 237 of the main text recommends, and the supplement should say so.

This is a clarity point, not an error.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Not adopted: the entry stays a Moderate error, not a clarity note or the Minor rating I suggested.

P032-E001 Moderate Under dependence between non-incident dyads, DCRSEs can overstate uncertainty as well as understate it, and the reanalysis is a lower bound on the consequences of inadequate clustering only under a sign condition that the article does not state.

Agreed. The sign condition was implicit to our understanding of the natural of these problems typically, though we did not state it explicitly.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed. The independent check found that the usual setting of these problems does not by itself ensure the sign condition, so the entry remains a Moderate error. My report that the condition was implicit is my own account and could not be verified independently.

P033-N006 Major The reanalysis shows that standardized standard errors become larger and absolute $t$-statistics smaller on average; it does not establish that the original studies used inappropriate inference, or that more than half of them did.

I disagree that this is an error. This is an implication of the motivation of the study. The implication is that the RD design is motivated by the semiparametric continuity assumption, not by stronger (e.g., linearity) assumptions. Thus, under those implicit (smoothness-only) conditions, the inference procedures used were inappropriately optimistic.

Updated after the LLM decision: I continue to believe this is a clarity note, not a major error. I think this is an example of the (current) LLMs being overly literal in interpretation.

Suggested classification: clarity note.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented and suggested a clarity note.
3. Independent evaluation. Incorporated in part. The entry now acknowledges the paper’s standard of continuity and smoothness and its warning about neglected bias, and qualifies the coverage claims of its affine example as pointwise. My suggested classification (clarity note) was not adopted.

P033-N010 Moderate For Coppock and Green (2016), the estimate is an effect on the probability of voting but is standardized by the standard deviation of counts of voters, so that its “$d=0.1$” is not Cohen’s $d$ for the individual voting contrast; under the paper’s normal approximation with the saved standard error, the study cannot reach 95% power at $d=0.1$.

I think this is minor. It's accurate as-coded even if not on the authors' original scale.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Incorporated in part. The entry now says that the code accurately computes the standardized slope that it targets, and describes a mismatch of comparisons rather than an arithmetic error. My suggested rating (Minor) was not adopted.

P033-N011 Moderate With the default variance of rdrobust and clustering, a cell of at least four observations that share a value of the running variable inside one cluster adds nothing to the variance; the reanalysis standard error of Broockman and Ryan (2016) and its saved power of $1$ come from this cancellation, and correcting that study alone under the normal approximation turns the 17% of p. 9 into about 14%.

This is a problem with rdrobust that we should have noticed. It's not an error of ours per se, but it does affect one of the conclusions.

Suggested classification: Moderate error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Moderate error.
3. Independent evaluation. Incorporated. The entry now attributes the defect to the default settings of the rdrobust package, which the paper’s call inherited. It remains a Moderate error, as I suggested.

P035-E001 Major The experiment estimates the effect of assignment to the Michelle Obama porch condition; it does not establish that her initiative to reduce childhood obesity caused the difference.

I think you are being far too literal in your reading of this. This is the inferred mechanism for the intervention. We are explicit that what we were randomizing, and why this is our inference from it.

Updated after decision rendered by LLMs. I continue to think this is a silly decision by the LLMs, and a warning to people using the (current iteration of) LLMs to judge severity of errors.

Suggested classification: clarity note.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented and suggested a clarity note.
3. Independent evaluation. Incorporated in part. The Introduction’s stated purpose and proposed mechanism are no longer treated as errors, and the quotation now includes the paper’s disclaimer about the nationwide campaign and its sentence on reinforcement. My suggested classification (clarity note) was not adopted.

P037-E001 Not confirmed The composition coefficient is identified under the supplement’s qualified causal-response model; randomization and the eligible-pool control alone do not identify it.

I disagree that this was an error. If the baseline treatment effect $\tau_0$ is constant across schools, this problem does not exist. This was the motivation (and was what was meant by the linear model expressed), in conjunction with trying to preserve efficiency with such a small number of schools. More generally, even if we don't believe this strictly, we just need a no-moderation-by-seed-composition assumption.

Below is an explanation produced by another LLM:

P037-E001: the intended baseline-effect assumption

The identification problem described in P037-E001 does not arise if the baseline treatment effect is constant across schools, under the maintained linear composition-response model. This constancy was the motivation for the model: the single coefficient $\tau_0$ was intended to express a common baseline treatment effect.

Let $D_i$ indicate school treatment, $C_i$ denote realized seed-group composition, and $q_i$ denote seed-eligible-group composition. The intended causal model is

$$Y_i(1,c)-Y_i(0)=\tau_0+\beta c.$$

Here $\tau_0$ is common across schools, and $\beta$ is the common linear effect of composition. Untreated outcome levels may differ across schools. With the study's randomized assignments and equal school-treatment probabilities, a fixed analysis sample, and the design expectation $\mu_i=\mathbb E[C_i\mid D_i=1,\text{school }i]$ represented by the included composition controls, the regression identifies $\beta$ through its expected normal equations, provided residual composition variation is positive. This is an identification statement, not a claim of exact finite-sample unbiasedness.

The decomposition emphasized by the finding,

$$D_iC_i=D_i\mu_i+D_i(C_i-\mu_i),$$

does not itself establish confounding. Under the intended model, both components enter the causal response with coefficient $\beta$, while the baseline treatment contribution is exactly $\tau_0D_i$, already included in the regression. There is no additional composition-related baseline treatment contribution for $D_iC_i$ to absorb.

The finding's counterexample instead assigns school-specific baseline treatment effects $\tau_{0i}=-q_i$. Its spurious composition coefficient arises from that systematic moderation. The example establishes the need for an identifying assumption, but does not contradict the intended constant-$\tau_0$ model. Differences between estimates from alternative specifications likewise do not establish that this assumption fails.

A weaker sufficient assumption

Strict constancy is stronger than necessary. Write

$$Y_i(1,c)-Y_i(0)=\tau_0+\eta_i+\beta c.$$

School-specific baseline deviations $\eta_i$ are permissible provided that their contribution to treatment effects is orthogonal to the composition variation used by the regression. Precisely, first remove realized block means from the regressors. Let $R_i$ be the residual from projecting the demeaned $D_iC_i$ on the other demeaned regressors, using design-expected cross-products to define the projection coefficients. The relevant condition is

$$\mathbb E\!\left[\sum_i R_iD_i\eta_i\right]=0.$$

Under the preceding assumptions, the coefficient $\beta^*$ from the expected normal equations satisfies

$$\beta^*-\beta= \frac{\mathbb E[\sum_i R_iD_i\eta_i]} {\mathbb E[\sum_i R_i^2]}.$$

Substantively, this is a no-moderation-by-eligible-composition assumption for the baseline treatment effect, understood in the precise orthogonality sense above. Baseline-effect heterogeneity must not track expected seed composition in a way that the regression attributes to $\beta$. This restriction covers variation both within and between blocks; block and covariate main effects do not automatically remove treatment-effect moderation. It permits heterogeneous baseline effects and the modeled causal composition effect $\beta c$.

The supplement's equation (2), repeated in equations (4) and (6), was intended to express this restricted causal response. E001 should therefore distinguish a concern about the plausibility or explicitness of that restriction from an identification failure under the intended model. The common-baseline model eliminates the specific problem, and the weaker orthogonality restriction is sufficient for the same purpose.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Incorporated in part. The entry is no longer classified as an error: it is now not confirmed, with no severity, rather than the Minor rating I suggested, and it is presented as a clarification of the scope of the model.

P037-E005 Moderate The twelve entries labelled “Bayesian Bootstrap SE” in Tables S6 and S7 were computed with an independent uniform weight for each school, which is not the Bayesian bootstrap; in a reconstruction on the public data, uniform-weight dispersions are 27% to 32% smaller than the corresponding dispersions under the Bayesian bootstrap. The rows should be relabelled as dispersions under uniform weights, or recomputed with Dirichlet$(1,\ldots,1)$ weights.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P037-N001 Moderate The reduction in disciplinary reports of peer conflict is 25%, not 30%; the authors' correction of 2018 makes this change in the four places where 30% is printed.

We already corrected this, no? It's fine to restate it here though.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: the entry already records that the official 2018 correction replaced 30% with 25%.

P037-N006 Major The HC0 normal-reference school tests reject a true null more often than their nominal 5% level in specified null experiments on the public-data design; a calibrated replacement for the original hypotheses remains unestablished.

I don't know that this is an error per se, so much as it was a suboptimal decision. HC0 SEs remain common -- in particular as raw influence function-based estimators (see, e.g., the DoubleML implementation). I agree that the correction presents more accurate SEs.

Updated after decision rendered by LLM. I think it is good that this is being flagged, as the SEs are misleadingly small.

Suggested classification: Moderate error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented and suggested a Moderate error.
3. Independent evaluation. Incorporated in part. The independent check confirmed that HC0 and influence-function variances remain in contemporary use; the entry, which makes no claim to the contrary, was kept. My suggested rating (Moderate) was not adopted.

P037-N007 Moderate The authors' code selects the 49 schools of the conflict analyses by a rule on the outcome (a school mean of the peer-conflict count above $0.02$), not by whether a school provided administrative data, which is the article’s description; on the public data 52 schools have disciplinary records, and the three further schools that the rule removes are treatment schools.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P037-N008 Moderate The article reports the overall reduction in disciplinary reports as significant ($P < 0.05$), the 2018 correction says that the same effect remains insignificant at $0.05$, and the sentence of the Results was never withdrawn; a conventional two-sided Wald test on the public data gives $P = 0.099$, supporting nonsignificance for that reconstruction.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P037-N009 Moderate The control mean of $0.20$ in the Results and the “unadjusted” 23% of the 2018 correction are computed in the authors' code over all 56 schools, including the seven schools outside the conflict analyses, whose counts are zero or nearly zero; on the 49-school sample selected by the code, the public data give raw means of $0.21$ and $0.18$ and a raw reduction of 13.6%.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P037-N019 Moderate The disciplinary outcome is described as the number of times each student was disciplined for peer conflict, but it includes writeups that name the student as the target of harassment: 903 of the 4,075 units of the outcome on the public data. The outcome is an index of administrative records, and item 28 of the SI list counts writeups, not incidents.

This is an important clarification.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change: my comment acknowledges the clarification's importance and adds no factual or mathematical correction; the classification is unchanged.

P037-N025 Moderate The network results are not computed on one population of 2,451 students with positive probability of all four exposures: for each outcome the authors' code keeps the students whose four estimated exposure probabilities exceed $0.01$, who were not seeds in the realised assignment and whose outcome and preprocessed pretreatment score are available, with a further school-mean cutoff for disciplinary reports, and it averages over that set.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P037-N026 Moderate The dashed bands of Fig. 2, and of the school-level panels of Fig. S6, are not the 95% confidence intervals for the two predicted means that the caption and the keys describe: the band around the treatment line uses the standard error of the treatment–control contrast, and the band around the control line uses the variance of the control schools' residuals divided by 56.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P037-N032 Moderate In the authors' primary social influence code, two students are peers when either nominated the other; the supplement defines a student’s exposure by “student $i$'s row in the network adjacency matrix” and does not say that the matrix of nominations was made symmetric.

I think this is a clarification point.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Not adopted: the entry stays a Moderate error, not the Minor rating I suggested.

P037-N033 Moderate The authors' code computes the numbers of Fig. 3 and Table S5 from one weighted regression over the four exposures with common covariate slopes, and not from the separate weighted regression within each exposure that the SI Appendix displays; the two procedures can give different estimates.

Agreed.

Suggested classification: Moderate error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Moderate error.
3. Independent evaluation. No change: I agreed with the entry and its Moderate rating.

P040-E001 Moderate The printed interval agrees with a nominal Wald interval with a conventional lower-endpoint interpretation, but its normal approximation is poorly calibrated with 15 responses: exact coverage at a detection probability of $.80$ is about 81.5%.

I agree that what is proposed is a better analysis, but this is a standard interpretation of a asymptotic Wald-type interval (even if the asymptotics weren't warranted).

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Incorporated in part. The claims that .81 cannot be a nominal 95% limit and that the paper misread its interval were withdrawn, and the entry was redrafted around the interval’s poor calibration in small samples. My suggested rating (Minor) was not adopted.

P041-E006 Major Study 3’s nondetection and smaller early absolute survival differences do not establish that editorial bias is limited to active politicians or is not systemic to Wikipedia, or exclude Wikipedia’s culture or its policy on biographies of living persons as explanations.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P043-E007 Moderate After the paper’s normalization, the control group means are 0.896, 0.884 and 0.805 in 2007, 2008 and 2009: the description “essentially unchanged” omits a decline of about 10% by the third wave, while the gap between the treatment and control means shrinks from 0.161 to 0.089 to 0.022.

I think this is largely a subjective interpretation of wording.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Not adopted: the entry stays a Moderate error, not the Minor rating I suggested.

P043-N014 Not confirmed Whether the favorable-mediation claim omits the sign premise needed for its comparison.

I think the context implies the hypothesized sign of the effect that would make this claim true.

Suggested classification: clarity note.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a clarity note.
3. Independent evaluation. Incorporated in part. The contextual reading was accepted, and the entry is now not confirmed, with no severity: under the classification rule, an allegation that is not confirmed is classed as not confirmed rather than as the clarity note I suggested.

P044-N005 Major The Horvitz–Thompson variance estimator is conservative with exact exposure probabilities and positive marginal support, but not in general with probabilities simulated from 10,000 assignments, as in the chapter: in a small-world network regenerated from the printed code its expectation is 0.617, 0.035 and 0.995 times the variance of the estimator for the three contrasts when the realized assignment is independent of the simulated ones.

I agree that this is an important practical problem. I think there's a bit of ambiguity in wording (the estimator is conservative, the Monte Carlo-based approximation doesn't have enough replicates in some applications). I think the answer is "K=10000" replicates is not sufficient for some applications, or some correction is needed to guarantee conservatism. This is not an unfamiliar problem (think, e.g., the bootstrap).

Comment after LLM decision: I don't know if I agree that this is a major error per se, but I do agree that it should be flagged, and I'm glad that this is getting attention.

Suggested classification: Moderate error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Major error.
2. I commented and suggested a Moderate error.
3. Independent evaluation. Incorporated in part. The entry now gives the analogy with the bootstrap and names more simulation draws, with accuracy checks or proved error control, as a possible remedy. My suggested rating (Moderate) was not adopted.

P045-N016 Clarity note The guarantee is exact with true marginal and joint exposure probabilities. The context clearly treats Monte Carlo probabilities as numerical approximations; the guarantee should state this scope explicitly.

I think it is clear that it is meant for the known pi case / large R case.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Incorporated in part. The contextual reading was accepted, and the entry was reclassified from a Moderate error to a clarity note, which goes further than the Minor rating I suggested.

P046-E001 Moderate The interval of Proposition 3 is not finite-sample valid at every level, and its proof centres a conditional Hoeffding bound at $\mu$ where the conditional mean of $\hat\mu$ is required; the interval with radius $(\sqrt{\xi}+1)\sqrt{\log(4/\alpha)/(2n)}$ is valid.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P046-E002 Minor Proposition 4(c) holds when the working family of outcome regressions contains the zero function and the fitted regressions converge; for other families the adjusted estimator can be less efficient than the Horvitz–Thompson estimator.

Agreed.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Minor error.
2. I commented.
3. Independent evaluation. No change was needed: my comment agrees with the entry.

P046-E004 Clarity note Assumptions on the outcome regression alone also permit a uniformly consistent estimator; the abstract names only assumptions on the propensity score function.

Yes, this is what we meant but it was a misstatement.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): clarity note.
2. I commented and suggested a Minor error.
3. Independent evaluation. Not adopted: my suggested rating (Minor) was not adopted; the claim is correct in context, so the entry stays a clarity note without an error severity.

P054-N001 Moderate Under the stated independent-respondent survey linearisation, all nine intervals widen and the 53% stricter-law interval includes 50%; the nine estimates and the three main positive comparisons remain.

I agree that this is a preferable analysis. The original analysis was just performed via weighted least squares (just intercept-only regression in R) with classical standard errors. But it doesn't materially affect any conclusions, and so I think this is minor.

Suggested classification: Minor error.

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented and suggested a Minor error.
3. Independent evaluation. Incorporated in part. The entry records my report that the original analysis used intercept-only weighted least squares in R with classical standard errors; this reproduces the rounded numbers but could not be independently identified as the original procedure. My suggested rating (Minor) was not adopted.

P061-N019 Moderate The “average direct effect” of the conclusion is defined for the individualistic exposure mapping $g_i(\mathbf z)=z_i$, where it equals the AEED under independent assignments; for the general exposure mappings that the paper allows, the expression $y_i(d;\mathbf D_{-i})$ need not be defined, and the AEED of a coarsened exposure can differ from the average direct effect at every population size.

I agree this was misstated. I'm not sure that it's moderate (maybe minor), but it should have simply referred to the conditions in https://academic.oup.com/biomet/article-abstract/111/1/1/7078801

1. Classification before my review (the classification when the audit was first completed, before I reviewed it): Moderate error.
2. I commented.
3. Independent evaluation. Not adopted: the entry stays a Moderate error, and a citation alone was not accepted as the repair.