Automatic Rollbacks is a useful safeguard for your feature flags that allow you to proactively manage feature releases. This functionality allows you to set up custom rollback conditions, or ‘triggers’ which monitor your features’ performance to automatically turn off the respective rule or flag.
This guide will walk you through the process of setting up Automatic Rollbacks and leveraging Kameleoon’s metric guardrails to prevent potential negative impacts on your application.
Setting up rollback conditions
To get started, create a new rule or click on an existing one. Then, navigate to the Rollback Condition at the bottom of the edit pane for your desired rule. Here, add a new condition and fill in all the fields:
- Select whether to turn off only the selected rule, or you can choose environment to turn off the toggle for the environment entirely. Choosing the latter will stop all active rules (deliveries and experiments) in that environment.
- Choose the performance outcome to check against: Uplift, Downlift or Conversation Rate.
- Define the condition (higher than or lower than) and enter the threshold percentage which triggers the automatic rollback.
- Select which goal to monitor and evaluate for performance. Here, you can pick from any Kameleoon goal that you’ve already created for this project.
- Lastly, select the minimum visitors evaluated before the rollback can be executed.
Once you save, Kameleoon will continuously monitor the specified metrics against the defined condition(s). You can always choose to add as many additional conditions as you’d like by repeating the same steps – but, whichever condition is first met will be instantly executed, and the rest will be ignored.
Setting up email alerts
You can also set up email alerts for specific conditions. This ensures your team stays informed about any unexpected adverse effects on your key performance metrics. After you’ve at least added one condition, check the box next to Send Email alerts and enter the email addresses of the recipients who should receive the alerts. You can add multiple email addresses. Save to finalize.
You’ll now receive instant notifications whenever any of the specified conditions is met, allowing you to respond promptly.
Conclusion
Automatic Rollbacks provide an additional guardrail of control and safety to your feature releases and experiments. When you have performance-sensitive features that require proactive rollbacks, you can ensure that any potential issues are swiftly addressed, minimizing the negative impact of poorly performing features.
Examples
Here’s some examples of scenarios where it might make sense to configure an automatic rollback and prevent unforeseen harm to product metrics.
| Condition | Scenario |
| Pause this Environment if Uplift is Lower than 15% for Conversion Rate after at least 10,000 evaluated visitors. | In an e-commerce website, a new feature aimed at increasing the conversion rate is introduced. If, after evaluating 10,000 visitors, the uplift (increase) in conversion rate is less than 15%, the feature is automatically rolled back to prevent potential revenue loss. |
| Pause this Rule if Downlift is Higher than 30% for Cart Abandonment after at least 50,000 evaluated visitors. | A new checkout flow experiment is launched to boost sales conversion. If, after evaluating 50,000 users, a drastic drop (downlift) in cart abandonment by more than 30% is observed, the feature is automatically disabled to prevent revenue losses. |
| Pause this Environment if Downlift is Higher than 25% for User Engagement after at least 15,000 evaluated visitors. | A new user engagement feature is added to a social media platform. If, after evaluating 15,000 users, the downlift in user engagement exceeds 25%, it might indicate a potential issue (like spammy behavior). Automatically pausing the feature helps maintain a healthy user experience. |
| Pause this Rule if Conversion Rate is Lower than 10% for Click-Through Rate (CTR) after at least 10,000 evaluated visitors. | Testing a new layout on a news website is undertaken to improve click-through rates. If, after evaluating 10,000 visitors, the conversion rate is less than 10%, it might indicate that the new layout isn’t effective. Automatically rolling back helps maintain engagement. |
| Pause this Environment if Uplift is Lower than 5% for Average Order Value (AOV) after at least 25,000 evaluated visitors. | In an e-commerce platform, a feature aimed at increasing the average order value (the average amount spent per transaction) is introduced. If, after evaluating 25,000 users, the uplift in AOV is less than 5%, it could indicate a pricing or UX issue. Automatically pausing the feature prevents potential issues. |
