Set up and launch a hybrid A/B experiment

In this article:

– An overview of hybrid experiments and their benefits

– How to create and launch a hybrid experiment

– How to access and analyze the results of your hybrid experiment using Kameleoon’s reporting tools, and best practices for ensuring the accuracy and reliability of your results

Hybrid experiments are SDK based experiments that benefit from client-side capabilities.

They allow you to :

• leverage most of our plug-and-play integrations to sync and unify your experimentation data in your preferred solutions
• push client-side (JS / CSS) changes on top of your your server-side experiments
• analyze server-side experiment data in real time with built-in client-side reporting including 25+ templated filtering and breakdown criteria.

Note: Kameleoon Hybrid™ is only available with Kameleoon Full Stack.

To know more about Kameleoon Hybrid™, click here.

Create a new hybrid A/B experiment

On the Experiment dashboard, click on “New Experiment”

Select the Code editor option.

Select the hybrid type of experiment (both server and client-side).

In the case of a hybrid experiment, the experiment configuration is also injected into the kameleoon.js application file, which is not the case for SDK based experiments.

Please note that hybrid experiments are only available for projects of type Site : SDK and Hybrid experiments and Site : All types of campaigns. More information on projects types

Click on “Create”: the Code editor opens.

Note: If you have both Kameleoon Web Experimentation and Kameleoon Full Stack, you will also be able to use our Graphic editor to push graphic changes to your hybrid experiment.

Configure a hybrid experiment from the Code editor

The Code editor allows you to push both client-side (JS/CSS) and server-side (JSON) updates for each variation of the experiment.

Header

The header displays the main information associated with the experiment:

• its name (you can double-click to edit it) ;
• its status (draft, online, paused, deviated or stopped);
• the last interaction (last save or update, publication date…).

You can check that the changes you have made have been saved. Two types of backup are available:

• automatic when the experiment is in draft, scheduled or paused;
• manual for online and deviated experiments (via the green button in the “Finalize” tab).

“Experiment” block

By default, the Variations block is displayed. To access the Experiment block, simply click on “Experiment”.

Test-specific code

For all variations, you can add, modify or remove JavaScript code, CSS and/or JSON data (select the corresponding tab at the top).

At the bottom right of the code window is an expand option.

Click on this button to expand the code area or use the following keyboard shortcut (while editing your code):

On Windows: CTRL + B

On MAC OS: Control + SHIFT + B

• Code options

Choose here to include or exclude the original of the changes you have just made. If the box is checked, JavaScript, CSS and/or JSON data will also be injected on the original variation.

Tags and description

Edit the description of your experiment or the associated tags if you wish.

URL of the tested page

You can edit the URL of the tested page here. By default, the URL is the one you entered in the pop-up window for creating your hybrid experiment.

Open Graphic editor

The Graphic editor will then open instead of the Code editor.

Experiment results page

If your experiment has already been launched, a link will take you to the results page.

If your experiment is stopped, a green button at the bottom of the editor offers you direct access to the results page.

“Variations” block

By default, the Variations block is displayed when you access the Code editor.

• In the left part you can add, duplicate, rename or delete a variation.
• On the right side, you can add, modify or remove code in JavaScript, CSS and/or JSON fields. It will apply to the selected variation. The code is checked in the same way as in the “Experiment” tab. The JSON feature allows you to update variables values set in your server-side code during runtime. To know more please refer to our SDK documentation.

At the bottom right of the code window is an expand option.

Click on this button to expand the code area or use the following keyboard shortcut (while editing your code):

On Windows: CTRL + B

On MAC OS: Control + SHIFT + B

Add a variation

Click on “+ Add a variation”.

The color circle associated with the variation fills up when the variation is selected. This allows you to navigate between the different variations of your hybrid experiment.

Variation menu

By clicking on the 3 points to the right of each variation, a menu opens.

The changes made via this menu then apply only to the selected variation.

Edit the options in the JavaScript code on the variation (you can choose between several display parameters).

If necessary, enter the redirection URL for the variation.

Find all the options available in the classic Graphic editor.

• JavaScript code options

1. Choose when the JavaScript code should run ;
2. If the JavaScript code is a pop-in, you can limit its display by enabling some of the options shown.

• Redirection URL

By default, no redirection is set on the variation. However, you can set up a global redirection or a parameter redirection.

• Preview

When you click, the preview URL opens in a new tab. You can then check the display of your variations to test the code added in the JS/CSS tab only.

• Rename

Edit the names of your variations.

• Duplicate

When you click, a copy of the variation is created.

• Delete

A pop-in asks you to confirm your choice.

“Finalize” block

Finalization steps

In the left part, as in the case of an A/B experiment on the Graphic editor, you must validate each of the 3 steps to be able to launch your experiment:

• traffic distribution;
• targeting;
• tracking and goals.

• Traffic distribution

Click the pencil icon to set up the traffic distribution on your A/B experiment.

Confirm your choice: the step is validated, a green check indicates it in the editor.

• Targeting

Click on the pencil icon to set the targeting of your hybrid experiment.

This is mandatory for hybrid experiments to make sure Kameleoon knows whenever an experiment has taken place in your back-end, in order to count in the analytics tool (Kameleoon, Google Analytics…) only visitors that have actually seen/triggered the experiment server-side. It is also needed if you want to run client-side changes on top of your server-side experiment.

There are two ways to achieve this:

• You can provide the correct targeting segment by using our segment builder. The targeting conditions should exactly match the triggering conditions on the back-end. For instance if your experiment runs on all your product pages, the targeting conditions should match this scope as well.
• You can use our Activation JavaScript API to trigger the experiment explicitly from your code, where the experiment’s triggering code is written. It can be for example embedded in the returned HTML page. Please read our developer documentation for more information about how to set up the back to front-end bridge. Once the logic is managed in your code, you will need to create a segment with the targeting criteria “Explicit trigger” to tell Kameleoon that the targeting is explicitly triggered by code.

• Tracking and goals

Select at least one goal that you want to associate with your hybrid experiment.

Confirm your choice: the step is validated, a green check indicates it in the editor.

Simulation

Kameleoon allows you to simulate client-side changes done to your hybrid experiment. To do so, simply click on the “Simulate” button to open the simulation mode in a new tab of your browser.

Note: Kameleoon will not be able to simulate your server-side changes as the code is managed in your back-end. To test your server-side changes in production, we recommend to set a custom data that contains internal user IDs and to activate the experiment first for selected users before pushing live the experiment for all visitors. Please read this documentation to know more about it.

Experiment status

At the top right, an insert reminds you of the status of your experiment: draft, online, paused, deviated or stopped.

Note: Planning is not available in the case of a hybrid experiment.

Once your experiment is launched, you can change its status by clicking on the arrow to scroll through the possible actions.

Information

The insert is unfolded by default. You will find the website to which your hybrid experiment is associated, as well as the creation date of your experiment.

IDs

The insert is unfolded by default. At a glance, you will find the ID of your experiment as well as the IDs of each of the variations of your hybrid experiment.

Publish

The “Publish” button becomes active when the 3 necessary finalization steps have been completed.

Click on it to immediately launch your hybrid experiment.

A summary pop-in shows the essential configuration of your experiment, as well as the procedure to follow. All you have to do is code your variations on the server side, using the generated IDs. You can follow these instructions.

You can copy everything at once by clicking on “Copy all” at the bottom left.

Click on “Publish”: a message will confirm that the experiment has been successfully launched and the status of the experiment will change from “Draft” to “Online”.

Note: There can be a latency time between the online release of the experiment and its visibility on the site. This is controlled by the actions_configuration_refresh_interval variable available in all our SDKs. This variable specifies the refresh interval, in minutes, of the configuration for experiments and feature flags. It means that once you launch an experiment, pause it, or stop it the changes can take (at most) the duration of this interval to be propagated in production to your servers. If not specified, the default interval is 60 minutes.

You stay on the Code editor and can make changes to your experiment if necessary.

Save and update

During the creation of your experiment, you can save your changes and exit the Code editor. Your experiment will be saved as a draft and you will be redirected to the A/B Dashboard.

Once your experiment is online, you can still make changes to it. The saving of your changes is indicated at the top left of the Code editor, just below the experiment name.

Two types of saving are available :

• automatic when the experiment is in draft or paused;
• manual for online and deviated experiments.

Find a hybrid experiment

Tag

The type “Hybrid” is indicated on the related column.

Filter

On your Experiment dashboard, you can apply one or more filters to find an experiment. The “Hybrid” filter is selectable in the “Type” filter category. Only the hybrid experiments will be displayed.

Experiment card

On the Experiment Dashboard, when clicking on “Finalize in the Code editor” (via the secondary actions of the related experiment card), the Code editor opens in a new tab.

As with all experiments cards, it is also possible to duplicate the experiment and its configuration, delete it (if it is in draft or stopped), manage its status and view the results.