Getting Started

Most nimbus-cli commands need an --app and --channel parameter. These should correspond to the app as it is installed on the simulator or emulator.

App Name	--app value	Available values for --channel
Firefox for Android	fenix	developer, nightly, beta, release
Firefox for iOS	firefox_ios	developer,beta, release
Focus for Android	focus_android	developer, nightly, beta, release
Focus for iOS	focus_ios	developer,beta, release

The app and channel are how they are known to Nimbus. The channel corresponds to the build type: e.g.

• fenix developer is the Debug build variant of Firefox for Android.
• firefox_ios developer is the Fennec build configuration of Firefox for iOS.

You can also specify a --device-id for working with multiple simulator/emulator/device at the same time.

list

You can find experiments you want to use by using the list command.

nimbus-cli --app fenix --channel developer list

The above command gives a formatted table of experiments:

• with the targeted channel,
• the percentage of eligible users will be enrolled
• the affected features
• whether the recipe is an experiment or a rollout (R)
• names of the branches

Here is an example:

Experiment slug                                                   | Channel |     % | Features                      |   | Branches
------------------------------------------------------------------+---------+-------+-------------------------------+---+----------------------
 viewpoint-rolling-week-4-expansion-android                       | release |   3 % | messaging                     |   | treatment
 mobile-default-browser-cta-copy-test                             | release |  90 % | messaging                     |   | control, treatment
 lifestyles-images-onboarding-experiment-v3                       | release | 100 % | juno-onboarding               |   | control, treatment-a
 android-research-surface-validation                              | release |  10 % | messaging                     |   | control, treatment
 release-android-onboarding-redesign                              | release | 100 % | juno-onboarding               |   | control, treatment-a
 fx-release-android-re-engagement-notifications-114-rollout       | release | 100 % | re-engagement-notification    | R | control
 notification-worker-validation-experiment                        | nightly | 100 % | messaging                     |   | control, treatment-a

You can see the preview collection, by adding a preview parameter.

By default, this lists what the apps can see: this is coming straight from Remote Settings.

If you want to see what is available via the API, then use --use-api. It should be noted that this is shows both live and completed experiments, so is likely fairly large, and slow. It will not show preview experiments.

nimbus-cli --app fenix --channel developer list preview

You can also use:

• production or release, or nothing to specify the production server.
• preview or production/preview to specify the preview collection
• stage to specify the staging server
• stage/preview to see the staging server and the preview collection of the staging server.

tip

More information about the experiment can be found with the info command.

You can also filter the experiment list by slug, features, channel, or activity status:

Here we do a substring match on features, looking for onboarding in any of the features, for experiments in the release channel.

nimbus-cli --app firefox_ios list --feature onboarding --channel release

Here we look for any experiments active (either enrolling or observing). We use the --use-api flag because the experiments may not be live anymore.

nimbus-cli --app firefox_ios list --use-api --active-on 2023-05-01

We can use today as a shortcut for the current date.

nimbus-cli --app firefox_ios list --enrolling-on today

enroll

You can start the app enrolled in a branch of a given experiment.

nimbus-cli --app fenix --channel developer \
    enroll $EXPERIMENT_SLUG --branch $BRANCH

This will download the experiment from Experimenter, and then make changes to it to make it inevitable that the Nimbus SDK will enroll the experiment with that branch:

• changes the channel of the experiment to that of the installed app
• set the targeting "true"
• the bucketing for the particular branch as 100%
• set the enrollment paused flag to false.

Once these changes have been made, the app is launched, and the experiment is injected into the SDK.

nimbus-cli --app fenix --channel developer \
    enroll android-research-surface-validation --branch treatment

If you're downloading the experiment from preview, stage or stage/preview, you should prepend the server slug to the experiment slug:

nimbus-cli --app fenix --channel developer \
    enroll preview/mobile-default-browser-homepage-banner-copy-test --branch treatment-c

You can use the --preserve-targeting and --preserve-bucketing options to preserve the targeting and bucketing.

Enrolled experiments should persist across restarts of the app, and fetching of new experiments is disabled.

tip

Occassionally, you may want to get the experiment from Remote Settings rather than Experimenter. You can do this by adding the --use-rs flag.

Validating experiments

By default, the experiment branches will be validated against the feature manifest. This can be configured with --version VERSION, --ref REF and --manifest FILE parameters, or disabled with the --no-validate parameter.

Testing interactions between experiments and rollouts

In addition to an experiment slug and branch, zero or more experiment slugs can be specified.

These must be rollouts. This is to test the interactions between rollouts and experiments acting upon the same feature.

nimbus-cli --app fenix --channel developer enroll mobile-default-browser-cta-copy-test --branch treatment viewpoint-rolling-week-2-expansion-android

--preserve-targeting and --preserve-bucketing acts on these rollouts too.

Console output

The corresponding adb commands and xcrun commands are shown in yellow. These can be useful for attaching to bug reports as part of the Steps To Reproduce (STR). The nimbus-cli command used will also be useful for STR.

reset-app

The app can be put back into the just installed state. On Android, there is an adb command to do this, but not in iOS. For iOS:

• the app container for the given simulator is deleted.
• the app's group containers for the given simulator are all deleted.

For both the command is the same:

nimbus-cli --app fenix --channel developer reset-app

nimbus-cli --app firefox_ios --channel developer reset-app

unenroll

From time to time you may need to test how the app responds to unenrolling from an experiment. You can force the unenrollment from all previously enrolled experiments with this command.

nimbus-cli --app fenix --channel developer \
    unenroll

info

When you're starting from just the experiment slug, you can use the info command to get more information:

nimbus-cli --app fenix --channel developer info mobile-default-browser-cta-copy-test

Slug        mobile-default-browser-cta-copy-test
Name        Mobile Default Browser CTA Copy test
Description A copy test on the default browser message on homepage.
URL         https://experimenter.services.mozilla.com/nimbus/mobile-default-browser-cta-copy-test/summary
App         fenix
Channel     release
E/R         Experiment with 2 branches
Enrollment  2023-06-06 ➞ 2023-06-14 (paused)
Observing   2023-06-06, proposed ending after 29 days
Targeting   "(app_version|versionCompare('112.!') >= 0) && (language in ['en'])"
Bucketing    90 %
Branches    control, treatment
Features    messaging

Notice:

- `E/R` stands for Experiment or Rollout.
- the targeting string is a JEXL expression, generated by Experimenter, and is the test all clients must pass in order to be considered _eligible_ for this experiment.
- the bucketing is the percentage of eligible clients that will enroll in the experiment, across all branches.

You can also get this outputed to a file as JSON or YAML with the --output parameter.

nimbus-cli --app fenix --channel developer info mobile-default-browser-cta-copy-test --output info.json

Experiment Slugs

Experiment identifiers, or slugs, are used to identify experiments and rollouts throughout Nimbus and nimbus-cli.

You can copy the slug to the clipboard in Experimenter by pressing the icon next to the experiment slug at the top of the summary page:

Alternatively, you can use the link you get from Experimenter.

e.g.

nimbus-cli --app fenix --channel developer \
    enroll \
    https://experimenter.services.mozilla.com/nimbus/mobile-default-browser-cta-copy-test/summary \
    --branch treatment

or

nimbus-cli --app fenix --channel developer \
    info \
    https://experimenter.services.mozilla.com/nimbus/viewpoint-rolling-week-4-expansion-android