How long should your paywall test run?
Enter your current conversion and daily paywall views to see how many days a test needs. Switch to revenue per view if your test changes which plan people buy.
Run the test for 35 days
You need 20,622 paywall views per variant, 41,244 in total, to catch a 15% change in conversion from a baseline of 3.5% conversion.
- Decide the length before you start and stop on that day. Stopping the first time the result looks good makes false winners much more likely.
Bigger changes need fewer views
| Change to detect | Views per variant | Duration |
|---|---|---|
| 5% | 177,290 | 296 days |
| 10% | 45,362 | 76 days |
| 15% | 20,622 | 35 days |
| 20% | 11,858 | 20 days |
| 30% | 5,500 | 10 days |
Two-sided test at 95% confidence with 80% power, normal approximation. How this works
Questions people ask
What is the smallest change worth detecting?
It's the smallest improvement that would make the change worth shipping. Smaller changes need far more traffic: halving the change roughly quadruples the views you need. Most paywall tests on small apps can only detect changes of 15% or more.
Should I measure conversion or revenue per view?
Measure revenue per view when the variants change plan order, plan prices or the default selection, because those tests move money between plans. It needs more traffic than conversion, since revenue differs a lot from one buyer to the next.
Why run for whole weeks?
People who install on a Saturday often behave differently from Monday installs. A test that runs for full weeks gives every weekday the same weight in both groups.
My app doesn't have enough traffic. What can I do?
Test bolder changes, such as a different price or a new plan lineup, instead of button colors. Put all eligible traffic into the test. Or measure a step that happens more often, like trial starts, and confirm the revenue effect afterwards.
What do confidence and power mean?
At 95% confidence, a test of two identical paywalls declares a false winner about 5% of the time. At 80% power, a test detects a real change of the size you entered about 80% of the time. Raising either one makes the test longer.
Get your test results in your inbox every Monday
We're building a weekly report for subscription apps. Connect RevenueCat or upload an export, and each Monday you get every running paywall test's revenue result, a check that the split is healthy, and what to test next.
Planned price: $49 a month, or $99 for teams.