Actions on the results page

At the top of the results page, on the right of the web experiment’s main information, there is the page’s general actions menu.

The options in the Actions menu allow you to act on the entire results page :

• Refresh data;
• Reset data;
• Stop collection of data;
• Custom view;
• Save display settings;
• Create alerts;
• Print;
• Export;
• Share;
• Enable bayesian;
• Add multiple testing correction (only for experiments);
• enable CUPED (only for experiments).

Refresh data

This action updates all the data on the page without reloading the page itself. You are thus sure to view the most recent data!

Reset data

Reset data if you want to delete all the campaign results. To reset the data of a campaign is to start the campaign a second time, all the results are reset.

Warning! This action cannot be undone! It is offered only to authorized users, and a confirmation pop-in secures your data in case of an unintended click.

Stop collection of data

Note: This action is not available for feature flags / feature experiments.

Have you collected enough data on your campaign, and were you able to draw conclusions from your results? Then you can decide to stop collecting data on it. If you are used to leaving your campaigns online for long periods of time, this will allow you to have a better loading performance.

Visitors who enter this campaign will no longer be counted and data will not be collected or stored. You can resume the data collection later (in this case, your evolutionary graphs will show a gap between two curves).

This will not impact the status of the campaign, which will remain online.

Custom view

A pop-in opens, giving you the possibility to change the view of the results page.

View per goal/per variation

In the drop-down menu, choose a display of the results by goal or by variation. The presentation of the results in the graphs and tables will then be modified: the displayed curves will represent either the goals or the variations. In the same way, each table will correspond either to a goal or a variation.

View per audience type (personalization)

In the drop-down menu, choose the results display by goal or by audience. Results presentation in the graphs and charts will be modified: the displayed curves will represent either the goals or the audience. In the same way, each chart will correspond either to a goal or an audience.

Change the reference

By modifying the reference variation, the results (and especially the improvement rates) will no longer be calculated according to the original page of your website but according to the variation of your choice.

To set a reference variation different from the original, switch the button to ON. In the drop-down menu that appears, select the variation of your choice from the variations of your experiment.

Hide the results of one or several variations

If you only want to look at certain variations of your experiment, this option allows you to hide a selection of variations from the results page.

Click on the switch button to put it on ON; in the drop-down menu that appears, select the variation(s) you want to hide from the page.

Hide the results table of one or several goals

In the same way as with variations, you can hide the goals you do not want to see.

Click on the switch button to put it on ON then, in the drop-down menu that appears, select the goal(s) you want to hide from the results page. Associated targets tables will not be displayed anymore.

You can check or uncheck all the goals at the same time in the pop-in.

Save display settings

We offer a default view, but you can set your own display according to your needs.

You can then save it so you will find the same display on your next visit. It is also possible to load a predefined view to apply it to the page.

Save the current settings

To memorize the current display, just name it and, if you wish, set it as default view for this campaign. By checking this option, you will find this display with each new consultation of the results of this campaign.

Then click on Validate.

These elements are saved:

• Filters;
• Date ranges;
• Columns of the tables;
• View per goal/per variation;
• Reference variation;
• Display of the goal(s).

Load a predefined view

You can apply to your page previous display settings set up on this same page.

Select a view from the drop-down menu and, if desired, set it as the default view for this campaign.

Then click on Validate.

To the right of the selection field you will find a trash icon to delete a previously saved display.

To do so, select the display and click on the trash. Gone with the wind!

Create alerts

Configure alerts to be informed by e-mail of the evolution of your results.

First, enter the e-mail address to which you want to receive the alerts.

Note: You can’t enter more than 10 e-mail adresses.

Two predefined alerts are directly available, but you can also set custom alerts.

Predefined alerts

Choose to be informed when your campaign results are ready or when a variation is identified as a winner on the main goal.

Your campaign’s results are considered as ready when a variation has a reliability on the main goal superior to the minimum fixed. A reliable variation is a variation with a reliability higher to 95%. However, you can edit this rate from the setting page of your website.

Click on the switch button of the alert to put it on « ON » and activate it.

Custom Alerts

Custom alerts can be set on any of the data available in your results. Just click on Custom alerts to set up.

Setting up a custom alert includes three steps:

In the first drop-down menu, select the data for which you want to be informed of the evolution: improvement rate, conversion rate, reliability, number of visits, visitors or conversions.

• Conversion rate / Improvement rate / Conversions number: As soon as a variation reaches the value you set for each goal you have selected.
• The number of visits / The number of visitors: As soon as the addition of variations reaches the value you set.
• The reliability sequential: As soon as the reliability sequential is higher than the value you set on the variation(s) you choose.

Note: Data is based on converted visits. In order to garanty an appropriate alert, we don’t send any e-mail before 100 visits and 20 conversions. 

An alert will be sent to you as soon as the value reaches the threshold you set. Choose whether this threshold should be higher or lower than the value you want and fill in this value.

You can also be alerted when the value of the measured data increases or decreases in proportions that you set.

Finally, in the last drop-down menu, define the variations or goals to which the alert applies.

If you have chosen to be alerted about the improvement rate, the conversion rate or the number of conversions then you will define here the concerned goals. In other cases, you will have to choose among the variations of your experiment.

Print

The Actions menu allows you to print the results of your campaign.

Two options are available: the detailed printing of a part of your results or the printing of a global summary.

Print the detailed results

In the first drop-down menu, select the graphs you want to print. All the graphs available in your results page are also available here. In the second drop-down menu, select the goals or variations for which you want to print the tables.

If your results display is set on Goal view, the drop-down menu will let you choose the goals you want to print.

If the display is set to Variation view (experiment), it will allow you to choose which variations you want to print.

If the display is set to Type of audience (personalization), it will allow you to choose which type of audience you want to print.

Then click on Validate.

Print the recap

The impression of a summary of the results of an experiment includes:

• The name of the experiment and the project;
• The parameters of the page;
• The main results/winning variation;
• The graph for the main goal/variation;
• The table for the goal/variation.

The impression of a summary of the results of a personalization includes:

• The name of the personalization and the project;
• The parameters of the page;
• The main results;
• The graph for the main goal / main type of audience;
• The table for goal / type of audience.
• List of other goals.

Then click on Validate.

Export

The results of your campaigns can be exported as aggregated (Excel, PDF) or raw data.

Note: Raw export will include all of your campaign’s data we are getting from your website, before we transform it to display it on the results page, while the aggregated export will include the data as is in the results page, already transformed.

Aggregated data

Excel

To export your results in an Excel file, select Aggregated then Excel from the drop-down menu.

You can choose which type of data to export and your preferred display of Excel files: tabs per variation, tabs per goal or all data in the same tab.

Then click on Export to save the Excel file containing the result tables.

When opening the table, the first tab, Info, reminds you of the main information about the campaign: title, date range, and targeting. The following ones depend on the view you have chosen.

PDF

To export your results in PDF format, select Aggregated then PDF from the drop-down menu.

Then select the information you want to export in the same way as for printing.

Raw data

You can access the raw data of your campaign results and analyze it with any tool outside of Kameleoon. This can give you an independent way of analyzing your experiment results, notably with the freedom of using your own statistical models.

To export your raw data, select Raw from the drop-down menu.

Select the type of view (Visit/Visitor) and enter your email address.

Then click on Export. You will receive an email containing a download link for the file containing the data (.csv). This link is valid for 7 days only.

Note: This feature is also accessible via an API endpoint, enabling you to automate the retrieval and management of your raw data exports. Further details on accessing and utilizing this feature through our API endpoints can be found on our developers’ portal. You can use this endpoint to initiate an export and this one to obtain the download link once the export is successful.

The following columns are included in the CSV:

• Date (in customer time zone)
• Visitor Time Zone
• Visitor Code or your internal user ID if you use our cross device experimentation feature
• New or Returning (2 separate strings)
• Device Type Name
• OS name
• Browser name
• Referrer URL
• Landing page URL
• Country
• Experiment assignation – Variation name
• Goals informations for each goal attached to the test
  • The number of times it was converted in the visit (after being targeted by the experiment)
  • If we have revenue for this goal, the sum of the revenue associated with the conversions

You can also include any custom data as an additional column. Simply toggle the custom data you want to include in the dedicated dropdown menu. You will get, for each visit/visitor, a list holding the values which have been set for this custom data.

Note: Please note that the quantity of data units (visits/visitors) a customer can export is capped at one million per month. Should you wish to export a higher volume, we offer a premium version to cater to this need. For further information, please reach out to your Customer Success Manager.

Share

The results of a campaign can be shared by e-mail or by sending the URL of the page directly. The page thus shared only allows the consultation and does not give access to the editing options, the setting of the page or the various actions.

Share by e-mail

To share the results page by e-mail, select Send report by e-mail from the drop-down menu.

In the text box, enter up to 10 e-mail addresses separated by commas.

To delete an e-mail address, click on the cross next to it.

Then click on Share to send a report containing the results to the e-mail addresses.

Share a URL

To get a link to share your campaign results, select Share URL of this page from the drop-down menu.

The link appears directly in the pop-in and two options are available:

• Limit the validity of the link to 24 hours. 24 hours after the creation of the link, it will no longer be valid and will no longer allow access to the results;
• Secure access to this page with a password. Access to the results page from the link will only be possible with the password of your choice. Set this password by entering it in the text block.

Click on Copy and Close to confirm the creation of the link, copy it to the clipboard and close the pop-in.

Bayesian

By default, the results of your campaign are presented to you according to a Frequentist analysis of data. Kameleoon also offers you another statistical method: the Bayesian one, which is based on the the deduction of the results’ evolution according to trends seen previously on this experiment and/or on similar campaigns.

You can read more about the correction implementation in our statistical paper.

For the details of the Bayesian statistical method, and which one is best for you, check out our dedicated articles: Frequentist or Bayesian Approach? and Results according to the Bayesian method.

To switch the Frequentist results display to Bayesian, click the switch button to the right of the display settings to set it to ON.

Add multiple testing correction

This feature is only available for experiments.

This action is only available for experiments with more than one variation. When enabled, our multiple testing correction method will make sure your likelihood of getting a false-positive result stays under your configured level when determining the winning variation. (You can configure the required reliability on a Project-basis under Admin > Projects > [your site or app] > Experiment.)

You can turn on multiple testing correction for individual experiments. Once activated, Kameleoon will continue to display the “corrected” performance metrics on your experiment’s results page.

Note that when you apply this method to your experiment, its performance will likely look different from the results you are used to seeing. Namely, your experiment’s reliability will be lower when multiple testing correction is enabled. This is an expected outcome of enabling our method for multiple testing correction because it corrects for statistical significance to account for issues arising when a test has multiple variations. Furthermore, if you have previously used other correction methods, these numbers will likely differ from those as well. 

It’s up to you whether you want to enable multiple testing correction, although we highly recommend using it. (It’s better to use a foolproof correction method rather than an approximate one!) If you choose not to activate it and you want to feel fully confident in making decisions based on the results of an experiment that has multiple variations, we recommend that you wait to reach a higher reliability (e.g., 97% vs. 95%) before calling a winner. 

You can read more about the correction implementation in our article in our statistical paper.

Enable sequential testing

This feature is only available for experiments.

What is sequential testing ?

Sequential testing is a statistical approach used to monitor the progress of A/B tests over time and make data-driven decisions on when to stop a test.

With this feature, you can:

• Monitor the test’s progress in real-time
• Make informed decisions about when to stop the test based on statistical significance
• Reduce the risk of making premature decisions or continuing tests unnecessarily

How to use it?

In the Actions menu of the results page of an experiment, click on Enable sequential testing.

In the pop-in that opens, select the goal(s) on which you want to use sequential testing, then click on Validate.

Note: This action is reversible: you can return to this pop-in at any time to reactivate the classic algorithm.

Enable CUPED

This feature is only available for experiments.

Controlled-experiment Using Pre-Experiment Data (CUPED) is based on existing data on your former visitors, to speed up the time needed to obtain a reliable result.

You can read more about CUPED in our dedicated article and in our statistical paper.