QA Sign-off

Feature testing alone is not enough for experiments or rollouts. A QA passthrough will also ensure that the feature is successfully turned on/off through Nimbus, that it works on all desired platforms and most important locales, that the targeting expression works as expected (only users that meet the criteria will be enrolled in the experiment/rollout), and that the telemetry that we’re basing the analysis on is successfully gathered while enrolled.

Unless you are 100% certain you don’t need QA - file a QA "Nimbus/Remote delivery" ticket here. You can use this document as a guide when filing the ticket to ensure that you pass all the relevant information that you can to QA. Going through QA is recommended, as they often find critical edge cases that were missed.

The only possible exceptions where you may not need official QA are below; if you choose to skip QA, you must self-test, learn more. Self-testing is recommended even before sending to QA, as it can avoid days of delay if QA discovers an easily corrected recipe error when they start testing.

1. You are launching on Nightly or early Beta and the feature itself was tested.
2. You are repeating an experiment with minimal changes that don’t involve new code. Ex: QA tested the original experiment - and now you are rolling out one of the winning configurations. You launched the original but there was a small error in targeting (be sure to consider localization with every country/language change, if there is user facing copy in your experiment!)
3. You are doing a simple message experiment/rollout (Infobar or Heartbeat) with no new code changes or new targeting, so only changing the copy / well tested fields. Note: Skipping testing for this is when we see easy-to-make human errors lead to sending the wrong language places. We also see more image resolution issues that you’d think (cropped, color issues, scaling issues). Swapping images can cause unexpected display issues.