Experimenter warnings
There are a number of warning messages that you may encounter on Experimenter in the course of launching your experiment. These warnings are listed below, along with any restrictions that they may impose on an experiment.
Rollout bucketing warning
See the Rollout FAQ for general rollout information
If you are using the multiple-rollout approach, this warning can be safely ignored because you are explicitly creating more than one rollout in the same bucket.
However, if you are using live editability for rollouts, this warning tells you that there is a rollout that is already live that has the same application, channel, feature, and advanced targeting as the rollout that you are creating. This means that there will be collision, and your rollout will not function as-is. Users are already enrolled in the live rollout with the same criteria, and they will not be able to enroll in your rollout.
If you are encountering this and you would still like to proceed with your rollout, feel free to reach out to us in #ask-experimenter on Slack and we will be able to provide more guidance.
Rollouts and setPref Interaction (Desktop)
Rollouts that set prefs via setPref features can have unexpected interactions. The Nimbus client will always enroll in any available rollouts--including rollouts it was previously enrolled in--unless the user has specifically opted out via about:studies. If an enrolled user changes a pref set by the rollout they will be unenrolled from the rollout, but it will not be marked as an opt-out. This interaction will result in rollouts that will constantly override users' prefs unless targeting prevents re-enrollment.
To prevent this situation, check the "Prevent enrollment if users have changed any prefs set by this experiment" box on the branches page of your rollout.
Audience overlap
There are cases where the audience of two experiments/rollouts will overlap in ways that Experimenter can predict. Audience overlap will generally result in under enrollment which can potentially invalidate an experiment. There are three scenarios where we can detectable cases of audience overlap so that experiment owners can adjust their sizing accordingly.
The warnings list the slugs of the experiments that overlap with yours (the slug of an experiment is the hyphenated name listed underneath the experiment title). Easily navigate to one of these experiments by replacing the slug in your URL with one copied from the warning: https://experimenter.services.mozilla.com/nimbus/put-the-slug-here/summary.
These warnings do not prevent experiments from being launched.
Live experiments exist on a previous iteration of a namespace
Live multi-feature experiments exist on this feature
An experiment excludes other live deliveries
