Controlled-experiment Using Pre-Experiment Data (CUPED) leverages data on your existing visitors to accelerate time to reliable experimentation and personalization campaign results.
Key benefits
CUPED allows experimenters to make data-driven decisions faster and with more confidence, provides better estimates for the effect of variations, and reduces the risk of false positive results for tests with fewer data points.
When should you use CUPED
CUPED will have the most impactful effect if:
- You have been collecting data on your goal(s)l prior to the launch of the experiment, for at least 2 weeks.
- There is correlation between the goal conversions before the start of the experiment and during the runtime of the experiment.
Additionally, you should take into consideration the goals of your experiment before enabling CUPED. We recommend you activate CUPED for the main metrics of your site, such as transactions.
Enable CUPED
You can enable CUPED under the Actions menu on the results page of a campaign.
Click Enable CUPED. In the appearing pop-in, you’ll be prompted to select one or more goal(s).
Note: Data loading time may take slightly longer than for the classic algorithm. We advise you to limit applying CUPED only to the key goals in your decision-making.
Click Validate to start loading data with CUPED.
The data presented on the page will change depending on the goal(s) selected in the pop-in.
If you have selected your primary goal from the drop-down, the main results block of your results will present CUPED-adjusted data, and it will labeled as “CUPED enabled”.
Similarly, the results tables for each of your goals will be impacted, according to the goal(s) for which you enabled CUPED.
You’ll notice three differences in your results columns:
- “Conversion rate” becomes “Conversion rate adjusted”
- “Improvement rate” becomes “Improvement rate adjusted”
- “Reliability” becomes “Reliability adjusted”
Note that you can apply filter-based breakdowns to your data in the results tables only if you disable CUPED. When you click Breakdown on a table, you will be asked to confirm that you want to disable CUPED. You can reactivate CUPED after removing the breakdown filter.
Note: CUPED gets more effective over time. The more you use Kameleoon, the more experiment data you’ll have that our algorithm can use to improve the effectiveness of CUPED.
Manage CUPED
To modify your CUPED configuration, open the Actions menu and select Manage CUPED. (This menu option will only be available after enabling CUPED.)
You can select and unselect goal(s) in the pop-in window. Click Validate to save your changes.
Disable CUPED
To revert to the classic statistical method, open the Actions menu and select Manage CUPED.
You can unselect your goal(s) in the pop-up, then click Validate. After saving your changes, CUPED will no longer be applied to your results.
However, note that once you enable CUPED, you need to commit to relying on its results. Switching between the original and CUPED-adjusted results and making decisions based on the more favorable outcomes will increase your probability of overestimating the lift of your experiment and risk of getting false positive results.
You can read more about CUPED in our statistical paper.
