Testing Parameters - A/B Testing Campaign
Recommended Test Group Size
Audience Size | Test Group | A | B | Winner |
300 or less | 100% | 50% | 50% | 0% |
400 | 80% | 40% | 40% | 20% |
500 | 70% | 35% | 35% | 30% |
600 | 65% | 32.5% | 32.5% | 35% |
700 | 55% | 27.5% | 27.5% | 45% |
800 - 900 | 50% | 25% | 25% | 50% |
1,000 | 45% | 22.5% | 22.5% | 55% |
1,100 - 1,200 | 40% | 20% | 20% | 60% |
1,300 - 1,400 | 35% | 17.5% | 17.5% | 65% |
1,500 - 1,800 | 30% | 15% | 15% | 70% |
1,900 - 2,300 | 25% | 12.5% | 12.5% | 75% |
2,400 - 3,200 | 20% | 10% | 10% | 80% |
3,300 - 4,900 | 15% | 7.5% | 7.5% | 85% |
5,000 or more | 10% | 5% | 5% | 90% |
Cochran's formula was used to calculate the recommended test size group.
Test Duration Recommendation
Based on research, the winner will be correctly chosen 80% of the time if you run your test 2- 5 hours. Your own institutional results may vary.