The first phase of finalizing your A/B test is the allocation of traffic, i.e. the percentage of your visitors who will see your variation(s). By default, the traffic is evenly distributed among the variations; however, you can change this setting.
Two types of views are available: list and graph.
The list view shows sliders. In the example above, 50% of visitors will see variation 1; same for the reference version or original.
To change the traffic allocation, click and drag the slider of a variation by holding the click, or directly enter the percentage by clicking on the number displayed to the right of the slider. Traffic that you do not assign to any of your variations is automatically attributed to “Traffic excluded.”
The graph view represents the allocation of traffic between your variations as a graph. As in the list view, you can change this distribution, either by opening the drop-down menu and selecting or de-selecting one or more variation(s); or by clicking on a zone and dragging the slider. You can also directly enter a pecentage.
In each of these views, two ON/OFF buttons at the bottom left give access to additional options:
- You can ask Kameleoon to automatically optimize the traffic distribution based on test results. This means that the more conversions a variation generates, the more visitors are directed to it.
- You can also perform a multivariate test, or MVT. It enables you to combine several changes in one variation and analyze which combination generates the most conversions of your goal. For example, with three variations of an image and five variations of a button, an MVT tests the 15 (3 x 5) possible combinations. More information on this type of test.
Equal allocation per variation
When you run an A/B test on your production website, you have the possibility to specify a percentage of deviation of your traffic to the variations (step 2 in your test’s launch pop-in).
For example, if you have 3 variations in your test, setting a 75% deviation percentage will tell Kameleoon to deviate 75% of your traffic to your variations and 25% to your original page. By default, Kameleoon will then display 25% of the time Variation 1, 25% Variation 2 and 25% Variation 3.
Different allocation per variation
You also have the possibility to allocate different parts of your traffic to each variation. To do so, use the sliders to modulate the desired percentage for each variation.
You can also click on the percentage and directly enter the value you want to apply to the variation.
At any time, you can return to an equal distribution between the variations by clicking on the dedicated button, placed just below the list of variations.
Click on “Graph view” to switch to the second type of view.
When hovering over the graph, the portion of traffic allocated to each variation appears. Click to access the percentage and modify it if needed, in the same way as in list mode.
Advanced deviation options are available at the bottom of the pop-in.
Automatically optimize traffic allocation: multi-armed bandit
You can also let Kameleoon automatically manage your traffic allocation, in real time, based on the performance of the variations. Kameloon indeed embeds a multi-armed bandit algorithm, based on the epsilon-decreasing approach. It allow you to limit the opportunity cost of an A/B test made up of “lost” conversions on the least performing version. This alternative to the traditional A/B testing approach uses adaptive learning to select the best variation among several options. This type of traffic distribution is recommended if you are short of time.
By checking the “Automatically optimize traffic allocation” option, based on the first results observed during the test period Kameleoon will disable the worst-performing variations of your test and deviate traffic from these variations to the most performing ones.
Note : Dynamic traffic allocation has its limits. It does not take into account the preferences of each visitor and instead seeks to maximize the performance of certain variations. If an element has proven successful with a segment of visitors, but that element is part of a low-performing variation, these visitors will be redirected to a variation that has interested the greatest number of visitors.
- If you check a variation, the visitors previously associated with this variation will be re-bucketed. This means they could be affected to other variations than their original one.
- If you change the traffic allocated to a variation (without checking it), changes in repartition will only affect new visitors. Returning visitors for this unchecked variation will still be associated with this variation.
Once the changes are made, validate to reallocate the traffic.