If you want to set configurations for a feature remotely for non-experiment users, you can do so with Nimbus Rollouts.
This feature is still in the early stages of development. The first version will be available in Firefox 89 for a limited set of early users and more extensively in Firefox 90. See our project plan for more information.
- Launching a winning branch of an experiment faster than the trains.
- Launching a configuration to non-experiment users during an experiment after a short period of verification
- Configuring different settings for a feature for different audiences remotely
- A "kill switch" if you want to launch a feature but then turn it off if something goes wrong.
If the feature has not yet been extensively tested, isn't production quality, or needs a period of validation on the trains.
- In Firefox 89 on Desktop, if you are an L3 Nimbus Reviewer or UJET engineer.
- In Desktop starting in Firefox 90
- (Mobile dates TBD)
It's possible, but bear in mind that rollouts are not measurement instruments. Experiments are.
If you have uncertainty about the effect of the feature, you may wish to be guided by experiment results instead of deploying the feature immediately.
Before you do this, you should consider:
- Future experimentation needs: once you deploy the feature to someone, you lose the ability to observe what happens when you introduce that feature to that user. Consider whether you have a need for holdbacks.
- Decision criteria: identify the risks you're trying to mitigate with a rollout and decide whether you need multiple stages or not. If you have multiple stages, how will you know whether to advance or roll back? What signals will help you make your decision? Where will they come from? If you are relying on the experiment to guide you, make sure that the timelines are compatible. Consult data science before relying on signals derived from the behavior of the rollout group, since rollouts are not measurement tools and lack a control.
You would need to:
- Launch an experiment that targets a fixed portion of the population (sized appropriately for whatever you are trying to measure)
- When you are ready, launch a rollout using the steps below at a low percentage of the population
- As the rollout proceeds, consult your decision criteria. Change the percentage of the roll-out by editing the configuration.
Keep in mind that if you do plan to release the experience to 100% of users, you should make sure it meets production quality standards.
If a client receives a rollout and enrolls in an experiment for the same feature, the Feature API will prefer the experiment configuration.
To be able to use rollouts, make sure you register your feature in the Nimbus manifest and add variables you want to be able to set remotely. This process is identical to what you would do for running Nimbus Experiments, so if you've already done that, you don't need to make any changes in Firefox.
This is for MVP testing only will be replaced by Experimenter UI by Firefox 90.
To rollout changes to a feature, you will need to update (or create, if none exists) the record that matches your feature id in the
nimbus-desktop-defaults remote settings collection:
Each entry in the
configurations list can have a targeting
JEXL expression that you can use to target a specific group of users. They will be evaluated in the order they are defined until one matches.
You can monitor users that receive your configuration by looking at the experiments environment. When a rollout is active, it will add the following:
<feature>is the id of your feature in the manifest
branch: The slug of the whatever rollout configuration is active
Remember that rollouts are not measurement tools. Rely on experimentation to measure effect sizes.
In the future, we will support a preview workflow like with experiments. For now:
- Add the config to the staging Remote Settings
- Use the Remote Settings devtools to switch to the staging endpoint.
For writing tests please see Testing with Desktop Rollouts