Results according to the Bayesian method

portrait de l'auteur Julie Trenque

Written by Julie Trenque

Updated on 03/22/2022

2 min


Was this content useful?

What does the Bayesian method promise you?

The Bayesian approach is a different model of thought where instead of considering the parameters we wish to estimate as unknown constants, we model each parameters as a random variable. It takes its name from the Bayes’ rule and it allows us to compute quantities which are not available in the frequentist framework.

The Bayes’ rule combine the experiments results with any prior information we might have, although in the case of A/B testing we use non informative priors. By choosing carefully the a priori distribution we can have a convenient posterior one. We can then leverage the posterior distribution to compute, for example, the probability for the variant to be better than the original.

Access your Bayesian results

When you click on “Results” from the dashboard, you are taken by default to the classic results page.

To access the results generated by Bayesian statistics, click on the “Actions” menu at the top right of the page > “Enable bayesian”.

It is not possible to access the Bayesian report page in the following cases: 100% of your traffic is diverted to the original; the number of visitors to your experiment is 0.

The Bayesian results page

Structure of the Bayesian reporting page is quite similar to the classic results page.

However, some elements are different:

  • New indicators appear such as the probability of beating the original, the reliability of the results according to Bayes, and the forecast.
  • Several graphs do not appear on the page, and only the conversion rate is displayed.

A few definitions

Probability to beat the original

This is the probability that a variation will beat the original page with a higher conversion rate for a given goal.

In the case where the traffic allocated to the original is 0%, the variations, sharing 100% of the traffic, do not compete with the original. We will then talk about the “Probability to be the winning variation”.

Reliability of results according to Bayes

This corresponds to the confidence rate attributed to the results. This rate is calculated on a 3-level scale, easy to interpret thanks to the legend that appears automatically in the “Reliability” column of the results tables. The results are totally reliable when the 3 boxes are full: this means that the reliability rate has stabilized over time. Be careful, to avoid any reversal of trend, it is advised not to exploit your results before having reached a sufficient reliability rate.


The forecast is the anticipation of a value towards which the results will converge. This data is only available for the conversion rate and the improvement rate.

The prediction of a future value requires the collection of a certain amount of data. This indicator may therefore not appear the first few days after the experiment is launched.

The forecasts appear directly on the table, after the actual values of the conversion rate and the improvement rate.

The forecast value also appears in the conversion rate graph, it is located in the grey area on the right. It is symbolized by a circle, and takes the color of the variation it represents.

Use the horizontal dotted lines to compare the actual evolution of your conversion rate with its future value.

My results are very different, is it normal?

Both statistical methods lead to equivalent results, but they do not guarantee a perfect similarity between the two. It is therefore normal that you may observe differences between certain rates.

In some cases, two different variations can be announced winners on a same experiment.

Make sure that the confidence levels are at their maximum in both methods before comparing the two data. If it is and there is still doubt, then we recommend that you use the results of the classic method.

Further reading

If you would like more details on how Kameleoon’s statistic engine works, you can read our Statistical paper.

  • In this article :