Skip to main content

Rollouts and Experiment Interaction

tip

See the Rollout FAQ for general rollout information

Question​

Can a client be in a rollout and an experiment for the same feature at the same time?

Answer​

Yes, we have separate namespaces which means you can run rollouts and experiments simultaneously and each namespace does its accounting for users separately. You can run a 100% rollout and a 5% experiment and both will fill up, but everyone in the experiment is also in the rollout.

Question​

If there is a concurrent experiment and rollout on the same featureId, once the experiment ends, assuming the rollout is at 100%, a given client will immediately be put into the rollout. Correct?

Answer​

The short answer is “yes”, in most situations.

The long answer is: while both are running, an experiment client with actually be enrolled in both and the treatment they’ll receive will be the combination of both. This enables clients in subsequent experiments to easily get rolled out winners of prior experiments.

If both the experiment and the rollout are focused on the same fields of the treatment JSON, the experiment will take precedence and in this case the behavior will be as you state: effectively, the client immediately joins the rollout.

If they’re focused on different fields of the JSON, the behavior the clients gets will go from “experiment+rollout” to “rollout only”.

Question​

I have a feature that is now on by default in v115, and I want to wait until it has saturated the user base before shutting off the rollout. Do we have a standard around how long we should keep a rollout going in a situation like this?

Answer​

We typically wait 1-3 releases for the upgrade tail to catch up. You'll probably have the vast majority, so it just depends on the ramifications if people have it enabled on 114 and then lose it.

If it's not a big deal, turn off the rollout in 116 and most people will have it and the stragglers will get it when they upgrade. If it is a big deal - turn off in 117 or 118 and you'll have reached a few percent more with each release.