What is a multivariate test?
Difference betwen an A/B test and a MVT
The Multi Variate test, or MVT, allows you to change several elements on a page and analyze which combination converts the most. Unlike an A/B test with a reference A and a variation B, the MVT allows you to change several elements on a page and analyze which combination converts the most.
Note : the number of versions of a same page can quickly become important with a multi variate test. You need to make sure to have enough unique visitors for the result to be statistically significant. For further information, you can read our article about Statistical significance.
In addition to your original page, you would like to test 2 versions of a button, 4 titles and 4 images. Kameleoon will allow you to test and analyze which “Button-titlte-image” combination is the most efficient for your website.
In this example we have:
- 2 buttons + the original = 3 different buttons
- 4 titles + the original = 5 different titles
- 4 images + the original = 5 different images.
If you choose an MVT, you will have 3 x 5 x 5 = 75 combinations to test. Kameleoon can handle an unlimited number of variation and will allow you to statistically calculate which version is the most efficient. Note that you should make sure to have enough trafic on your website before testing 75 variations.
Using the editor to create a MVT
You can create Multi Variate tests with Kameleoon editor. Here is how to proceed.
Creating the variation
We would like to test several combination of colors/titles for this button:
For our multi variate test, we will test 2 different titles and 3 background colors.
- “Download Firefox” (original)
- “Download now”
- Green (original)
Therefore, we will test 6 combinations:
To do so, we have to create 3 variations in our test:
- Variation 1: change of title
- Variation 2: change of color
- Variation 3: change of color
To avoid a wrong combination, like red button + blue button, use this convention to name your variations:
Finalizing the test
Once your have created your variations, click on the “Finalize” button to launch your multi variate test.
Like for a classic A/B test, select the variations to include in the test. Under the list of your variations, check the “Execute a multivariate test” box.
Then, choose the targeting and the reporting tool for your test. For these steps, there are no difference with a classic A/B test.
Kameleoon will generate automatically the 6 possible combinations of your test.
Note: With Kameleoon, you can start an MVT in simulation mode (ie: only on your computer). Simply click on the “Simulate” link at the left bottom of the configuration window. Kameleoon will generate every possible combination according to the chosen variations.