Menu
Techniques

Switchback experimentation

Oct 2020
Assess?

Controlled experiments using A/B testing is a great way to inform decisions around product development. But it doesn't work well when we can't establish independence between the two groups involved in the A/B test — i.e., adding someone to the "A" group impacts the "B" group and vice versa. One technique to address this problem space is Switchback experimentation. The core concept here is we switch back and forth between the "A" and "B" modes of the experiment in a certain region at alternating time periods instead of both running during the same time period. We then compare the customer experience and other key metrics between the two time buckets. We've tried this to good effect in some of our projects — it's a good tool to have in our experiments toolbelt.