A Practical Protocol for DiD

Synthesising the methodologies throughout Chapter 4, Section 4.10 provides a compact, end-to-end checklist for conducting staggered Difference-in-Differences (DiD) analyses in marketing and econometrics. Following this workflow helps ensure that analyses are rigorous, assumptions are stated clearly, and conclusions are robust to alternative specifications.

1. Define Estimands and Map Cohorts

  • Define the Target Estimand: Clarify if you are targeting the overall Average Treatment Effect on the Treated (ATT), event-time effects ($\theta_k$), cohort-specific effects ($\tau_g$), or calendar-time effects. Your choice must align with the business question.
  • Map Adoption Cohorts: Visualize the adoption pattern across units and time. Confirm that there is sufficient variation in adoption timing and that valid comparison groups (never-treated or not-yet-treated) are available for all treated cohorts.

2. Choose Estimators and Pre-Specify Aggregations

  • Select Heterogeneity-Robust Estimators: Avoid relying on Two-Way Fixed Effects (TWFE) as the primary estimator. Instead, use modern alternatives (e.g., Callaway–Sant’Anna, Sun–Abraham, or Borusyak–Jaravel–Spiess). Use TWFE only as a benchmark.
  • Pre-specify Weights: Decide how you will aggregate cohort-time effects. Base weights on cohort size, treated unit-periods, or weighting uniformly. Document these choices early in a pre-analysis plan to avoid ex-post “cherry-picking.”

3. Specify Event-Time Windows and Binning

  • Determine the scope of pre-treatment and post-treatment event times ($k$).
  • Longer pre-windows grant more insight into parallel trends but require longer historical data. Longer post-windows capture long-run dynamics but are constrained by late adopters.
  • Establish a reference period (usually $k = -1$) and outline any binning rules for sparse event times.

4. Run Pre-Trend and Placebo Diagnostics

  • Event-study Specification: Estimate leads prior to treatment. Verify visually that coefficient estimates are close to zero.
  • Joint Wald Diagnostics: Formally test if pre-treatment leads are jointly zero.
  • Placebo Tests: Run dummy analyses replacing the treatment period (placebo-in-time) or treated units (placebo-in-units). Estimates should be virtually zero to cast doubt on spurious effects. If these tests raise flags, explore Synthetic Control or factor models.

5. Assess Covariate Balance and Overlap

  • Check the Standardised Mean Differences (SMDs) for key covariates across treatments and controls. Imbalances over 0.1 to 0.2 standard deviations (especially on outcome-predicting variables) require correction.
  • Adjust through regression, propensity score weighting, or matching. Validate that overlap exists in the propensity score distributions.

6. Choose Clustering and Inference Procedures

  • Standard practice is to cluster standard errors by unit.
  • Expand to two-way clustering (by unit and time) if cross-unit correlation is expected.
  • For scenarios with few clusters ($< 20$), transition to wild cluster bootstraps or randomisation inference.
  • Choose a relevant multiplicity adjustment (Bonferroni, False Discovery Rate, Romano-Wolf stepdown) if you are examining many subgroup effects or event-time coefficients simultaneously.

7. Estimate and Report Aggregated Effects

  • Estimate the chosen summary measures and produce well-labelled event-time plots tracing $\hat{\theta}_k$ against $k$.
  • Clearly demarcate confidence intervals, pre/post-treatment sections, and the reference period.
  • Compare these estimates across multiple methodologies. Convergence across estimators grants robustness, while divergence prompts investigation into weighting and model assumptions.

8. Conduct Sensitivity Analyses

  • Iterate alternative choices (covariate sets, varying control pools, different windows) to produce specification curves.
  • Examine influence via leave-one-cohort-out or leave-one-period-out procedures.
  • Address plausible spatial or network spillovers by testing models with and without buffer zones excluding adjacent units.

9. Document and Report Transparently

  • Register analysis plans ex ante if possible. Document and rationalize any deviations from the initial plan.
  • Produce comprehensive reporting covering the data structure, diagnostics, primary estimates, sensitivity results, and inference procedures.
  • Accompany reports with reproducible assets (cleaned data, code scripts, and software version histories).