Skip to main content

Mutually exclusive groups for feature experiments

This article explains how mutually exclusive groups work for feature experiments, why they’re essential for reliable analysis, and how to set them up in Kameleoon.

note

To read about how to use mutually exclusive groups for web experimentation instead, refer to this help doc.

Running multiple feature experiments simultaneously can lead to overlapping effects, especially when different teams are testing changes that impact the same areas of your product.

Setting up mutually exclusive groups​

  1. Define the experiment group: Determine which feature flags (and their experiments) should be mutually exclusive. For example, all experiments modifying the checkout flow could be grouped together.
  2. Tag the experiment: Use the naming convention “ME-GROUP-{GROUP NAME}” to tag each experiment in the group, such as ME-GROUP-A. This tells Kameleoon to enforce mutual exclusivity within the group.
  3. You can tag flags at creation, or by clicking an existing flag’s three-dots menu and selecting Manage tags.

Once done, each visitor will be exposed to only one experiment from each mutually exclusive group, ensuring accurate, non-overlapping results.

Example of a mutually exclusive group​

Imagine you are testing different variations of your checkout experience:

  • Group A: Experiment 1 (new checkout flow) and Experiment 2 (one-click checkout)
  • Group B: Experiment 3 (up-sell recommendations) and Experiment 4 (discount banner placement)

With a mutually exclusive setup:

  • A visitor will see either Experiment 1 or Experiment 2 from Group A, but not both.
  • The same visitor may see either Experiment 3 or Experiment 4 from Group B, but not both.

This setup ensures that visitors do not experience multiple conflicting changes within each group, allowing for more precise measurement of each experiment’s impact.

To maintain consistency, if a visitor was previously assigned to an experiment within a group, they will remain assigned to that experiment. If they are new to the group, assignment will be random, ensuring an even distribution.

This setup does not require any additional targeting conditions, making it easier to implement for flags containing experiment rules and helping you maintain the integrity of your feature experiments, leading to cleaner data and more confident decision-making.