Hybrid Approach: The New Way to Calculate Statistical Significance
Our Hybrid approach combines Bayesian and Frequentist methods, calculating A/B Testing statistical significance with more speed and accuracy
A/B Test Statistics Reinvented
Anyone running an A/B test needs to know two things: 1) how significant are my results 2) how long until my data is reliable?
There are two types of A/B testing statistics: Frequentist and Bayesian. Frequentist statistics use a fixed sample size and show how often an event occurs. Bayesian statistics measure the probability that a hypothesis (B converts more often than A) is correct. It shows you the size of the effect and the trust you can place in it.
The Convertize Hybrid approach – combining Frequentist and Bayesian statistics – provides a series of metrics that help you decide when an A/B test is complete:
- Smart CR% – your conversion rate, weighted towards recent performance
- Confidence level – how many times (out of 100) the same test would produce the same results
- Duration estimate – how many days until your test is reliable
To get in-depth knowledge about the Hybrid approach developed by Convertize, download the ebook or get a paper copy on Amazon.
"What I particularly like is how easy it is to set up an experiment, and the visual representation of the key A/B testing statistics"
Test Hypotheses Trust Numbers
Using a unique Hybrid Approach to A/B testing statistics, Convertize tells you whether your data has captured a real change in your conversion rate. It then shows you how likely an edited page is to outperform the original. You can adjust your Confidence Level objective between 85 and 99%, shortening or extending the Duration Estimate for your experiment.
The statistical significance calculator incorporates more parameters than most A/B testing software platforms, in order to avoid false data and give you the most accurate results. This is why the reliability of your tests are described with a Confidence Level rather simply statistical significance:
- Avoid false positives
- Never waste traffic
- See how long a test will run
For more information about A/B testing statistics, explore our step-by-step guide.
How Convertize Makes A/B Testing Statistics Easy
Convertize incorporates Frequentist and Bayesian statistics, providing fast results that you can trust.
- Smart CR% responds to changes in customer behaviour
- The Hybrid Approach gives faster and more accurate results
- Automatic statistics make it easy to follow your Smart CR%, Uplift and statistical significance
- Our CRO Experts are available at any time to help you interpret your data
For more information on A/B testing statistics, explore our A/B Testing articles and best-practice guides.
