What is the difference between an A/B experiment and a MVT?

Beginner Experiment

A/B experiment

An A/B experiment allows you to make variations of one or several pages.
For instance, an A/B experiment can measure the performance of a product page A versus a product page B.

The disadvantage of this kind of experiment is that you need to change only one element per page (for example, the title of the download button) to know if this modification has an impact on the conversion rate.

Please note that you can change a multitude of elements with an A/B experiment but it would be impossible to know the impact of these changes individually.

 

Multivariate test (MVT)

The MVT allows you to change several elements of a page in order to analyze which combination converts the most.

For instance, you want to try on one A/B experiment your original version + 2 versions of the color + 2 versions of your button title and analyze which of the title-color combination suits the most.

If you choose an MVT, you will have 2 x 3 = 6 variations (including the original one) tested. Here is the main interest of an A/B testing solution such as Kameleoon, able to handle an unlimited number of variations and to statistically calculate which version is the more efficient.

For example, a simple MVT allows you to test the performance of these 6 buttons by creating 2 sections in the Kameleoon editor (one to change the title and one to change the color):

Note : The number of versions of a same page can quickly become important in the case of an MVT. You need to make sure to have enough unique visitors for the result to be statistically significant. For further information, you can read our article about Statistical significance.

To create your first MVT with Kameleoon, please read our article Creating a Multivariate test (MVT).