Implementing Randomization and Assignment
Section 5.8 covers the final practical step: how to actually randomize and assign treatments in carryover (switchback) experiments.
Randomization Points:
- Decide on the number and timing of randomization points based on the previous design steps (metric, granularity, lag order, duration).
- Sample the assignment path from the appropriate randomization distribution.
Optimal Design:
- If the time horizon $T$ is predetermined and $T/m$ is not an integer, you can still find the optimal design by solving the relevant optimization problem (see Theorems 1 and 2).
- In “imperfect” cases (e.g., $T$ not divisible by $m$), closed-form solutions may not exist, but numerical optimization or subset selection can be used.
Practical Guidance:
- Use commercial optimization software if you want to avoid discarding periods.
- If you prefer not to solve an optimization problem, discard a few periods and use explicit solutions where available.
Summary
The final step in carryover experiment design is to implement randomization and assignment according to the optimal design. Even when the time horizon is not perfectly divisible by the lag order, practical solutions exist—either through optimization or by discarding a small number of periods. Careful attention to these details ensures valid and efficient experiments.