During the configuration of your split URL experiment you have distributed your traffic between your variations, but you notice significant data discrepancies between Kameleoon results and a third party analytics tracking solution, especially regarding the number of visits on variation B.
It can be explained by several reasons:
- The Kameleoon tag is not installed on variation B of your experiment. This problem is common and the Kameleoon snippet has to be also installed on the new page (URL B) of your website so that Kameleoon can track your visits and conversions.
Note: In this case, you may notice some visits on variation B even though Kameleoon is not installed on page B. This is because Kameleoon will try to send the tracking call before doing the redirection to page B. So all visits counted on variant B are actually tracking calls from page A.
- The bounce rate metric could also be one of the reason. Indeed, with a split experiment and when a visitor has been bucketed to variation B, Kameleoon will either send the tracking data from the original page before redirecting the visitor to page B, or do it only when the visitor has landed on page B. This depends on how quick the redirection is being done. So if your bounce rate is high, you could see significant discrepancies between Kameleoon and your third party analytics results. Kameleoon won’t be able to trigger the third-party analytics tracking (as usually the tag won’t have loaded yet on the page), whereas it won’t be the case for Kameleoon (as usually our script will load first on your website). This usually explains why you have more traffic on variation B in Kameleoon that in the third-party analytics tool.
- If your consent policy is set to “Required”, you will need to follow these guidelines in order to have accurate metrics for your experiment.
One good test to identify if there is an issue or not is to compare the number of visits you have in your third party analytics tool for variation B with the number of visits you have for page URL B, independently of your experiment. You should get approximately the same numbers, assuming visitors cannot access page URL B without going through the experiment first (which is usually the case).
