Experimentation System Audit: How to Know Whether Your A/B Test Results Can Be Trusted
An A/B result is only as trustworthy as the system that produced it. This audit framework examines eligibility, assignment, bucketing, identity persistence, exposure, sample-ratio mismatch, metric definitions, conversion windows, and statistical analysis. It also introduces the Zero Experiment: a production A/A calibration test designed to reveal whether the system creates lift where no treatment exists.
