Cracking Product Sense Problems

10 08 2023 2 minute read

Note: The following content is summarized from the YouTube video titled Crack Metric/Business Case/Product Sense Problems for Data Scientists | Data Science Interviews from the Emma Ding channel.

Why and When Are These Questions Asked During Interviews?
- Evaluate candidates’ ability to define metrics
- Diagnose and solve real product issues
- Assess familiarity with products (not always necessary)
What Are Companies Looking For?
- Structure: A systematic approach
- Comprehensiveness: Addressing all crucial aspects
- Feasibility: Implementable solutions
- Occurrence: During technical phone screens and onsite interviews (>= 1 round)

Given a scenario where an important metric has shifted negatively, determine the root cause.

Example question1> Creation of Facebook user groups has gone down by 20%, what will you do?
Example question2> We have a dashboard tracking our metrics, and average ETA is up by 3 minutes. How would you investigate this problem?
Example question3> How to investigate the 10% drop of the friends application?

Clarify the scenario/metric
Assess Time: Is the shift sudden or stable? Consider internal and external factors.
Examine other products/features by the same company for similar issues
Segment by user demographic and behavioral features
Decompose the metric (if necessary)
Summarize the overall approach, including plausible causes and remediation strategies

Determine how to gauge the success/health of a product/feature.

Example question1> How would you measure the success of a product?
Example question2> How would you measure the health of Mentions, Facebook’s app for celebrities?
Example question3> Instagram is launching a new feature (link to products). How do you tell it’s doing well?

Clarify the function and goal of the product/feature
Define Metrics: Choose up to three, ensuring they are contextually relevant.

Decide whether to test a product idea or launch a product/feature.

Example question1> How would you set up an experiment to understand feature change in Instagram stories?
Example question2> What would you change in the Twitter app? How would you test if the proposed change is effective?
Example question3> If a PM says they want to double the number of ads in Newsfeed, how would you determine if this is a good idea?

Clarify Goal and define metrics similarly to the “Measure Success” framework
Design Experimentation: Detail setup, user segmentation, and duration.
Make Recommendations based on experiment results, considering business impact and possible conflicting results.

You may also enjoy