Skip to main content

Product Managers

What is the role of Product in experimentation?#

Product Managers typically have the role of experiment owner in the experimentation workflow. Responsibilities include:

  • starting and completing the experiment brief - which aligns everyone supporting the experiment to the learning goals.
  • implementing the experiment in Nimbus console
  • coordinating with the QA team for sign-off
  • launching the experiment
  • building a set of reviewers trained and authorized to perform review sign-offs in your feature areas

Where do I start?#

  • Watch the 5 minute video on experiment workflow for an overview of the experiment lifecycle to understand the stages / ordering.
  • Use the Workflow Miro board as your source for current links and guidance.
  • To jumpstart using the Nimbus console to create and launch experiments, there's a quick 30 minute zoom onboarding training. Please reach on #ask-experimenter on Slack to schedule a Nimbus Console onboarding session.
  • Start by opening the Experiment Brief link. If it is your first Experiment Brief - there is a 5 minute "how to video" at the top of that document.
  • Fill out your experiment brief. As the Product Manager you are responsible for clearly articulating the product goals and questions that need to be answered. Capture as much as you can. It will improve and evolve as you have conversations with different subject matter experts. The experiment brief is the authoritative source to capture the shared understanding.
  • Referencing one source (the experiment brief) for the verbose experiment background enables the subject matter experts (engineering, QA, experiment tooling, data) to efficiently support the effort and respond to changing requirements. Product Managers will keep editing the experiment brief throughout the lifecycle of the experiment. Capture the decisions as they are made back in your experiment brief. Decisions left in in slack channels, verbal discussions, various notes, jira tickets, etc - increase the likelihood of errors and wasted effort.
  • There Checklist section at the end of your Experiment Brief with pre-launch, post launch, and post experiment checklist to help you through the steps of your experiment.

Key Resources for Product Managers#

Rather than duplicate links that may later break as the process improves - links to the resources and training videos are in the Workflow Miro Board.

  • There are several weekly touchpoints Office Hours for in person assistance from subject matter experts in different areas.
  • If you are confused on any aspect - that's expected the first few experiments until you've gotten the rhythm. Ask in the #ask-experimenter Slack channel - it is an open community of support from several disciplines.