Product Managers are typically going to take the role of experiment owner in the experimentation workflow. Typical responsibilities include:
- starting and completing the experiment brief
- implementing the experiment in the experimenter console
- coordinating withe the QA team for sign-off
- launching the experiment
- building a set of reviewers trained and authorized to perform review sign-offs in your feature areas
- Watch the 5 minute video onexperiment workflow for an overview of the experiment lifecycle to understand the stages / ordering.
- Use the Workflow Miro boardas your source for current links and guidance.
- Start by opening the Experiment Brief link. If it is your first Experiment Brief - there is a 5 minute "how to video" at the top of that document.
- Fill out your experiment brief. As the Product Manager you are responsible for clearly articulating the product goals and questions that need to be answered. Capture as much as you can. It will improve and evolve as you have conversations with different subject matter experts. The experiment brief is the authoritative source to capture the shared understanding.
- Referencing one source (the experiment brief) for the verbose experiment background enables the subject matter experts (engineering, QA, experiment tooling, data) to efficiently support the effort and respond to changing requirements. Product Managers will keep editing the experiment brief throughout the lifecycle of the experiment. Capture the decisions as they are made back in your experiment brief. Decisions left in in slack channels, verbal discussions, various notes, jira tickets, etc - increase the likelihood of errors and wasted effort.
- There Checklist section at the end of your Experiment Brief with pre-launch, post launch, and post experiment checklist to help you through the steps of your experiment.
Links to the resources mentioned are all in the Workflow Miro Board. Rather than duplicate links that may later break as the process improves - please refer to that visual for the current links and quick training videos.
- There are several weekly touchpoints (Office Hours) for in person assistance from subject matter experts in different areas.
- If you are confused on any aspect - that's expected the first few experiments until you've gotten the rhythm. Ask in the #ask-experimenter Slack channel - it is an open community of support from several disciplines.