On the sales page, the saved display of 20,796 exposed users recorded a 7.36% checkout rate for the six-minute, lightly edited video, versus 2.94% for the ten-minute version. The historical probability of being best rounded to 100%, but unequal exposure and changed production prevent a verdict about video length. Verify allocation before testing duration with one script, and use completed purchases to decide whether the video earns more revenue.
What the two videos changed
| Dimension | Control | Treatment |
|---|---|---|
| Length | Ten minutes | Six minutes |
| Production | Heavier editing and background music | Lighter editing |
| Delivery | Deeper explanation of the core message | More direct delivery |
| Offer angle | Same core offer and benefit | Same core angle |
| Page unlock | 2:28 from page load | 2:28 from page load |
Both videos explained the same core offer, but the shorter cut delivered its message more directly. Our hypothesis was that a more direct pitch would bring more people to checkout. The test varied the full video package, not duration alone.
Which outcome did the video test measure?
We followed experiment exposure to a checkout-page visit. This was an interest-to-checkout test, not a completed-purchase result. Keep those steps separate when deciding whether a video earns more revenue.
A different question is whether viewers should be able to navigate the pitch. Our video-controls test kept the video fixed and changed the player experience.
Why we standardized the page-unlock timer
The original ten-minute video used a 5:40 timer before the rest of the sales page appeared. The platform could not assign a different timer to each arm, so we set both to 2:28. That held page availability constant between the two tested versions, while changing the original long-video experience.
The timer started on page load, not after that much video had been watched. A shared timer therefore does not mean viewers heard the same material before the offer appeared; script order and time to the key claim still matter.
Checkout results and unequal exposure
The report summary records a 24-hour run with 20,350 participants and checkout rates of 3.00% for control and 7.52% for treatment. The saved PostHog result display contains 20,796 exposed users and the following result.
| Arm | Exposed users | Checkout visits | Did not visit | Rate | Relative change | Historical 95% relative-lift interval | Probability of being best |
|---|---|---|---|---|---|---|---|
| Ten-minute control | 1,665 | 49 | 1,616 | 2.94% | Baseline | Baseline | — |
| Six-minute treatment | 19,131 | 1,408 | 17,723 | 7.36% | +150.08% | +137.80% to +162.95% | 100.00% |
The saved display rounded its historical probability of being best to 100% and marked treatment significant. That output does not resolve allocation, observation-window or content differences.
The six-minute arm received 91.99% of recorded exposure (19,131 of 20,796 users). Before interpreting the checkout gap, compare recorded exposure with the configured allocation using PostHog’s exposure diagnostics. (+) The intended allocation and the relationship between the observation windows remain unverified, so this is a descriptive comparison of video packages.
Design the next VSL test around one decision
- To test length, start with one script, voice, visual style, music choice and CTA. Remove preselected repeated or optional passages while keeping the central claims and their order intact.
- To test production, keep the script and duration fixed, then vary editing or music. To test the message, hold production steady and compare the promise.
- Before interpreting conversion, compare recorded exposure with the configured allocation. Investigate exposure-capture failures and delivery differences before deciding whether the videos caused the gap.
- Keep the page-unlock rule identical and record whether it uses elapsed time or actual playback.
- Use completed purchase or another business outcome to decide, with starts, progress, completion, CTA exposure and checkout visits explaining where the result comes from.
Choose a duration and sample target for your own traffic with the PostHog experiment guide, and define the business outcome with our metrics guide.