Health menu

The Health menu helps you monitor your experiment's data reliability. It automatically runs a series of checks to detect potential issues in tracking, traffic balance, or data collection, giving you greater confidence in your experiment results.

A Health tooltip is also displayed in the Overview menu, so you can quickly see if everything is running smoothly.

What are Health Checks

Health Checks are automated diagnostics that monitor your experiment's overall health. They run hourly during the first 24 hours after an experiment is activated, and daily afterward for all active experiments. You can also trigger them manually each time you open the Results page.

These checks help you:

• Detect tracking or data anomalies early.
• Confirm that your experiment is still collecting data as expected.
• Identify potential biases that could affect your results.

There are four core checks:

Check name	Purpose
Visitor Tracking Check	Verifies that new visitors are still entering the experiment regularly.
Conversion Tracking Check	Ensures that all attached goals are still receiving conversions.
SRM Check	Detects sample ratio mismatches between variations and expected traffic distribution.
Overlapping Exposure Check	Identifies users exposed to multiple variations of the same experiment.

Health check thresholds

Each Health Check uses specific thresholds to determine its status. Understanding these thresholds helps you interpret the severity of any detected issues.

Sample Ratio Mismatch Check

This check uses a Chi-squared test to calculate a p-value that measures how closely your actual traffic distribution matches the expected split.

Status	Threshold	Meaning
Passed	p-value ≥ 0.01	Traffic allocation is balanced. The observed distribution matches the expected split, and randomization is working correctly.
Warning	0.001 ≤ p-value < 0.01	Traffic allocation shows minor imbalance. The user split is deviating slightly from the expected ratio. While not yet critical, this status could indicate an emerging issue.
Failed	p-value < 0.001	CRITICAL: Sample Ratio Mismatch (SRM) detected. The observed user split is significantly different from the expected ratio, indicating a high probability of a systemic issue (for example, tracking bug, implementation error).

Overlapping Exposure Check

This check monitors the percentage of users who have been exposed to multiple variations of the same experiment. The rate is calculated as: (users exposed to more than one variation) ÷ (total users in the test).

Status	Threshold	Meaning
Passed	< 2%	Sample contamination is minimal. The level of user crossover is within acceptable limits for maintaining high data integrity.
Warning	2% to 5%	Moderate sample contamination detected. A small percentage of users have seen multiple variations, introducing some noise and potentially reducing statistical power.
Failed	> 5%	CRITICAL: High sample contamination. A significant portion of users have been exposed to multiple variations, which can severely bias results and invalidate the experiment.

Conversion Tracking Check

This check monitors whether your goals are actively receiving conversions.

Status	Threshold	Meaning
Passed	Conversions recorded in the last 24 hours	Conversion tracking is active. Goal events are being recorded as expected.
Warning	Zero conversions for at least one goal in the last 24 hours	No conversions were recorded for at least one goal in the last 24 hours, which could be due to low traffic, but may indicate a tracking issue.
Failed	Zero conversions for > 48-72 hours	CRITICAL: Conversion tracking has failed. No conversions have been recorded for at least one goal for multiple days. The tracking implementation is likely broken.

Visitor Tracking Check

This check verifies that new visitors are still entering the experiment.

Status	Threshold	Meaning
Passed	Visitors recorded in the last 24 hours	Visitor tracking is active. Visitors have been successfully recorded in the last 24 hours.
Warning	Zero visitors in last 24 hours	No visitors have reached the experiment in the last 24 hours. Traffic may be low or targeting issues may exist.
Failed	Zero visitors for > 48-72 hours	CRITICAL: No new visitors detected for an extended period. The experiment may not be correctly targeted or technical issues may be blocking visit logging.

Health summary in the Overview menu

At the top of the Overview menu, you'll find a Health summary badge showing your experiment's overall health at a glance.

• The badge color represents the current status across all checks:
  • Red: Failed check
  • Yellow: Warning
  • Gray: Pending
  • Green: All checks passed
• The text inside the badge displays the number of non-passing checks (red + yellow + gray), or All passed if everything is green.

Hover over the badge to see a tooltip listing all four checks with their individual statuses.

Icon	Status	Description
✅	Passed	No issue detected
⚠️	Warning	Potential issue detected
❌	Failed	Issue impacting experiment
⚪	Pending	Check not yet evaluated

At the bottom of the tooltip, click View full health report to open the Health menu.

Navigating the Health menu

The Health menu provides a detailed, interactive view of all checks and their results.

Top banner

At the top of the menu, you'll see:

• X/Y checks run: Number of checks completed (X) out of the total (Y).
• Last updated: The most recent evaluation time.
• Status breakdown: A count of checks by status—Passed, Warning, Failed, or Pending.

Check details

This tab is divided into two sections:

• Left column:
  • A clickable list of the four Health Checks, each showing its current status icon.
• Right column:
  • A detailed description of the selected check, including:
    • What it monitors
    • What the current status means
    • Recommended Next Steps if the check has a Warning or Failed status

For the Conversion Tracking Check, you'll also see a table listing all goals in your experiment, along with their individual statuses and the timestamp of their last recorded conversion.

Special cases

Pending status

The Pending status appears only when the experiment has just launched and no Health Check job has run yet. This state is temporary and automatically resolves once the first evaluation (hourly or manual) is completed.

Paused or stopped experiments

When an experiment is paused or stopped, the Health menu becomes disabled. The Health summary badge in the Overview menu is also hidden to keep the Results page clear.

Data updates and refresh

• Health Checks run hourly during the first 24 hours after activation, then daily for all active experiments.
• When you open the Results page, checks are refreshed on-demand.
• Cached results display instantly and automatically update once new data is received.

Compatibility

Health Checks are available for all experiment types:

• A/B tests
• Feature flags
• Personalization campaigns