Why ex ante diagnostics matter

Design-based inference is only as credible as its assumptions. Section 3.6 makes a strong point: you should diagnose the plausibility of identification assumptions before treatment is assigned and before post-treatment outcomes are observed. In observational settings, you do the same checks ex post, but their role shifts from design selection to credibility assessment.

The goal is not to “prove” assumptions, but to catch red flags early and pre-commit to diagnostic logic rather than post-hoc storytelling.

When identification relies on parallel trends, the most visible diagnostic is the pre-trend check. In an event-study framework with leads $\theta_k$ for $k<0$, test:

$$ H_0: \theta_k = 0 \quad \forall k<0. $$

Interpretation:

  • Lead coefficients are falsification checks, not causal parameters.
  • Failure to reject $H_0$ is supportive but not definitive.
  • Divergent pre-trends are a warning sign that parallel trends is implausible.

Because panels have clustered and serially correlated outcomes, use cluster-robust SEs, wild cluster bootstrap, or randomization inference for these tests. Also beware of pre-testing bias: choosing specifications based on pre-trend tests can distort inference for post-treatment effects.

If pre-trends look bad, consider alternative strategies such as factor models or synthetic control.

Balance checks on covariates

Balance checks assess whether treated and control units are similar on observed covariates. A standard metric is the standardized mean difference:

$$ \Delta X^{(j)} = \frac{\bar{X}_1^{(j)} - \bar{X}_0^{(j)}}{\sqrt{(S_1^{2}+S_0^{2})/2}}. $$

Common thresholds are 0.1 or 0.2 in absolute value, but the right cutoff depends on how strongly a covariate predicts outcomes. Large imbalance on a weak predictor may be less problematic than small imbalance on a strong predictor.

Balance is necessary but not sufficient: good balance does not rule out unobserved confounding. When balance is marginal, report both unadjusted and covariate-adjusted estimates to assess sensitivity.

Placebo checks and negative controls

Placebo in time: assign a fake treatment date in pre-treatment periods and run the same estimator. Placebo effects near zero are supportive; large placebo effects raise concern. But placebo checks have low power, so a null placebo does not prove validity.

Negative controls: outcomes that should not be affected by treatment but share the same confounding structure. If treatment “affects” the negative control, confounding is likely.

Negative controls are valuable but difficult to choose. A valid negative control must be unaffected by the treatment and share confounders with the main outcome. If either condition fails, the diagnostic is weak or misleading.

Overlap and support planning

Conditional independence designs require overlap: treated and control units must have similar covariate distributions. Use propensity score plots to check support.

When overlap is weak:

  • Trim units with extreme propensity scores.
  • Reweight or match to focus on the overlap region.
  • Model outcomes flexibly to reduce extrapolation risk.

Trimming changes the estimand. The effect becomes the ATE over the overlap region, not the full population. Document the trimming rule and report the fraction excluded, especially if the trimmed units are substantively important.

Planning for spillovers and interference

If SUTVA is likely to fail, you must plan spillovers ex ante with an exposure mapping $h_i(D_{-i,t})$:

$$ Y_{it}(d, h_i(D_{-i,t})). $$

The exposure mapping determines:

  • Which other units’ treatments affect unit $i$.
  • How spillovers decay in space or across networks.

Design implications:

  • Collect buffer-zone outcomes if spillovers are geographic.
  • Collect network data if spillovers are relational.
  • Pre-specify alternative exposure mappings and run sensitivity checks.

Mis-specifying the spillover channel biases both direct and spillover estimates. The exposure model is part of the design, not an afterthought.

What ex ante diagnostics actually deliver

These diagnostics do not make assumptions “true.” They help you:

  • Detect weak designs early.
  • Choose alternative estimators or redesign the study.
  • Pre-commit to a diagnostic workflow before outcomes are known.

In observational settings, the same checks are still essential, but failure should shift how confident you are in the causal interpretation.

Practical checklist

  • Pre-trend plots with joint tests for $k<0$.
  • Covariate balance with standardized differences.
  • Placebo checks in time and negative controls.
  • Propensity score overlap plots and trimming plans.
  • Pre-specified exposure mappings if spillovers are plausible.

Takeaway

Ex ante diagnostics are a discipline, not a box-check. They force clarity about identifying assumptions and provide evidence on whether those assumptions are plausible. The earlier you run them, the more you can shape the design to avoid fragile causal claims.

References

  • Shaw, C. (2025). Causal Inference in Marketing: Panel Data and Machine Learning Methods (Community Review Edition), Section 3.6.
  • Roth, J. (2022). Pre-test bias in event-study designs.
  • Rambachan, A., and Roth, J. (2023). A sensitivity analysis for parallel trends.
  • Austin, P. C. (2011). Balance diagnostics in observational studies.
  • Crump, R. K., et al. (2009). Dealing with limited overlap in estimation.