Video summary

4. Product Research & Validation

Main summary

Key takeaways

Business

Business-focused summary (Product Research & Validation)

1) How to run “discovery” research to prioritize problems (example: GrabFood)

  • Use surveys (e.g., Google Forms) to gather user-reported issues, then group and interpret results.
  • Survey design can bias outcomes (lecturer’s warning):
    • Multiple-choice boxes can constrain how users think; open text can reveal additional issues.
    • Some responses may be highly varied—grouping/aggregation must be done carefully to avoid missing the true “big” problem.

Recommended survey output structure:

  • Problem list → polling results → interpretation
  • Additional comments for issues not captured in the main options
  • Optional: include non-users only if the goal is new market/persona capture

Use descriptive statistics to identify the “most significant” problems:

  • Metrics mentioned: mean, median, mode, max, min, standard deviation
  • If data has outliers/skew, prefer median over mean to reduce distortion.

Concrete prioritization example (GrabFood priority solutions):

  • Discounts/subsidies on shipping costs
    • Intended impact: increase transactions

2) Product research goals: what business outcomes it should support

Product research should support both user alignment and business execution:

  1. Understand user needs and problems
    • Get close to users’ experiences, motives, and abilities.
  2. Align user interests with business interests
    • Example: planning roadmaps; breaking down long-term targets.
  3. Create room for innovation and accuracy
    • Iterate as user desires and market/economic conditions change.

“Captain of the ship” idea: PMs should be able to defend research decisions to stakeholders (CPO/CTO/CEO/product leads).

3) Competitive & comparative analysis (market execution requirement)

  • Continuous monitoring of competitors is treated as mandatory, because market shifts can happen.
  • Examples:
    • Lazada: adapted from Chinese market to Indonesia, later incorporated live shopping
    • QR code adoption: enabled by broader wallet/bank support; Dana adopted QR to speed up payments

4) Product research lifecycle (playbook: phases + what to measure)

Product research is broken into four phases:

  1. Before product launch

    • Decide which initiatives to work on
    • Prioritize using customer needs
    • Test “product-market suitability”

  2. Testing & feedback

    • Determine what to streamline/simplify
    • Understand customer perception of iterations
    • Measure what customers like vs. dislike
    • Find what drives interest so customers align with both user and business goals

  3. Soft launch (MVP)

    • Validate whether the MVP is effective/useful
    • Identify changes before full release
    • Continue discovery after launch (monitor behavior, conversions, feature sticking)

  4. After launch

    • Analyze user satisfaction, bugs, and improvement areas
    • Emphasis: launch doesn’t end work—bug fixes and iterative discovery continue

Metric types mentioned post-launch:

  • CTR (click-through rate)
  • conversion rates

5) Methods in product research (qual + quant + product analytics)

The instructor describes three main method categories:

  • Qualitative (unstructured / in-depth)

    • Interviews to explore user perspectives and causes of problems.
    • Useful for reasons behind behavior and willingness to spend.

  • Quantitative (structured / limited choices)

    • Representative % results; good for benchmarks and comparing behaviors/habits.
    • Example benchmark logic: “character June liked ~50%, July ~30%.”

  • Data & research (self-serve analytics + external benchmarks)

    • From dashboards/admin: usage patterns, login frequency, session duration, demographics, items bought, average spend, etc.
    • External data requires credibility checks.

Bias cautions mentioned:

  • Commitment bias (sticking with an idea too long)
  • Seeking only supportive data (“data picking”)
  • Confirmation bias in user research; risk of false positives

6) MVP rationale (execution-oriented)

  • MVP is recommended early to avoid building a full product before learning whether users accept it.
  • MVP complexity cost drivers:
    • High engineering costs per update
    • Expensive infrastructure/APIs (maps, routing, translation, AI services)
  • Therefore, MVP should test the core interaction/value
    • Example: start with “driver meets user,” not full automation

7) A/B, multivariate, usability, QA, and performance testing (validation toolkit)

After MVP creation, the video lists testing types:

  • A/B testing
  • Multivariate testing
    • Test multiple variants simultaneously; identify preferred approach.
  • Performance testing
    • Example context: works across devices/OS versions; catches cases where “Android hangs / iOS breaks”
  • Quality assurance (QA) testing
    • Pass/fail against defined steps, conditions, and acceptance criteria
  • Usability testing
    • Confirms the product functions properly in real user handling

8) Hypothesis/assumption testing with Design Thinking (business translation)

Design thinking steps:

  • Empathize
    • User motivation and problems via interviews/surveys/dashboards; use the right instruments.
  • Define
    • Core problem + goals; clustering/user journeys.
  • Ideate
    • Generate multiple solution alternatives; brainstorm + user flows.
  • Prototype
    • Low/high fidelity; Figma clickable prototypes.
  • Test
    • Validate quickly with trials/iterations.

Key distinction:

  • PM research focuses on business impact and whether solutions drive goals.
  • UI/UX research focuses more on user experience/interaction design correctness.
  • Both need alignment through user goals and PM-defined outcomes.

9) Product-market fit (PMF) and market sizing (TAM/SAM/SOM)

  • PMF is described as: users don’t just use/buy—they’re willing to share information (the market “likes the product” and keeps buying).
  • Market sizing uses TAM / SAM / SOM:
    • TAM: total addressable market
    • SAM: realistic reachable segment (demographics + accessibility + ability-to-pay)
    • SOM: portion you can capture (often starts with cities due to accessibility)

Indonesia constraints mentioned:

  • “3T/2T” hard-to-reach areas, poor internet/access, economic constraints
  • Competitor alternatives include offline options (e.g., food stalls as substitutes for delivery apps)

10) Case example: LinkedIn Learning research + feature prioritization (training product)

User persona (LinkedIn Learning):

  • Early-career professional (20s–30s)
  • Goal: upskill/pivot/advance
  • Needs: reliable materials, less time to find the right course, career relevance

Two research segments (based on goals):

  • Adoption: users who haven’t tried Learning
  • Engagement: users who used Learning and courses

Methods described:

  • Quantitative survey with measurable scales (e.g., frequency ranges 1–5)
  • Qualitative follow-up interviews (e.g., 10 interviews with reward incentives)
  • Uses rewards/incentives to improve response quality
  • Avoids leading questions; emphasizes that phrasing matters

Data presentation suggestions:

  • Use P charts per 100%
  • Use bar charts/histograms for frequencies
  • Qualitative: summarize themes/challenges/needs rather than long narratives

Output: identify user problems, e.g.:

  • Hard to find courses aligned to career goals
  • Forgetting due to lack of structure (course length without guidance)
  • Less personal course recommendations
  • No tracking feature / limited progress visibility
  • Need structured, interactive support for long-term learning

Ideation directions (then prioritized via audience voting):

  1. Learning goals + paths with interactive progress tracker + personalized recommendations
  2. Gamification (tasks + points/rewards)
  3. Popular job-aligned courses (updated based on job market trends)
  4. Community & mentor support

Observed poll metric (audience engagement):

  • ~60 respondents
  • Only 3 votes for one option (~5%)
  • Used as a lesson in prioritization perspective (no “right answer” emphasized)

Takeaway:

  • Selection depends on which option best targets the core problem and desired outcomes.

Extracted KPIs / metrics & targets mentioned

  • Survey analytics
    • Mean/median/mode, max/min, standard deviation
    • Handle skew/outliers via median
  • Post-launch product analytics
    • CTR
    • conversion rates
  • PMF market sizing
    • TAM / SAM / SOM (no numeric targets provided)
  • Voting/poll participation
    • Lowest-voted option: ~5% (3 votes out of ~60)
  • No explicit revenue/CAC/LTV/churn figures were provided in the subtitles.

Concrete actionable recommendations (as stated or implied)

  • Design surveys carefully
    • Add open-text responses alongside multiple choice to avoid limiting user thinking.
  • Use descriptive statistics to identify the most significant problem
    • Prefer median when data is skewed/outlier-heavy.
  • Segment research audiences by goals
    • Example: LinkedIn Learning adoption vs engagement.
  • Avoid biased/leading questions
    • Use neutral phrasing; follow up to understand why respondents selected options.
  • Validate early with MVP
    • Don’t wait for full product completion.
  • Apply a structured validation process
    • Use qualitative + quantitative + analytics
    • Include A/B/multivariate plus usability + performance + QA testing.
  • Continue after launch
    • Monitor dashboard metrics (CTR/conversion), fix bugs, and run periodic research/cross-team reviews.

Presenters / sources

  • Sis Re (main presenter)
  • Zudi (host/instructor assistant who introduced parts and managed transitions)
  • Brother Markov (referenced in the GrabFood discovery research example)
  • Rei / Sis Re (mentioned as “our coolest tutors”; Rei appears as a greeting only)
  • LinkedIn Learning / Notion (mentioned as products/tools; not presenters)

Original video