How To Keep Your Users | Startup School

Understanding Cohort Retention

Introduction to Cohort Retention

• David Lee introduces himself as a partner at YC and emphasizes the importance of creating products that people want.

• He explains cohort retention as a quantitative method for tracking how many new users continue using a product over time.

Importance of Understanding Cohort Retention

• David shares his personal experience of misunderstanding cohort retention during his startup journey, highlighting its significance in evaluating product success.

• He recounts the moment he realized he needed to grasp this concept better after an encounter with a VC firm.

Defining Key Components of Cohort Retention

Identifying User Cohorts

• The first step is to define user cohorts based on when they first used the product, such as weekly or monthly groups.

• Advanced segmentation can include factors like country, acquisition source, or device type.

Determining Active User Actions

• The next step involves defining what constitutes an active user by selecting specific actions that indicate engagement with the product.

• Examples include opening an app or completing significant tasks relevant to the product's value proposition.

Selecting Measurement Time Period

Understanding Cohort Analysis in User Retention

Importance of Time Granularity in Tracking

• When analyzing user behavior for travel apps like Airbnb, it's crucial to select an appropriate time granularity. Users typically travel only a few times a year, suggesting that tracking should occur quarterly or semiannually.

• The chosen time period must align with the product's intended use and user engagement patterns.

Measuring User Retention: Triangle Chart Method

• A triangle chart is utilized to track cohorts by month, isolating new users from each month (e.g., January, February).

• Each user is counted once per cohort regardless of how many times they return within that month, ensuring accurate retention metrics.

• For example, if 12 new users join in January, their return rates are tracked over subsequent months to assess retention.

Visualizing Cohorts Over Time

• Users are tagged based on their joining month (e.g., "January sticker"), allowing for easy identification of returning users from specific cohorts.

• The diagonal line in the triangle chart represents total users from each cohort who engaged with the product during a specific calendar month.

Analyzing Trends and Performance

• By normalizing data against initial cohort sizes, trends can be visualized as percentages. This helps identify performance improvements or declines across different cohorts over time.

• Observing both rows and columns allows for insights into whether newer cohorts perform better or worse than previous ones.

Interpreting Cohort Retention Curves

• Line graphs illustrate cohort retention curves over time. Each line represents a different cohort's performance (e.g., December vs. January).

• Understanding these curves helps determine which products retain users effectively and highlights potential issues with declining retention rates.

Case Study: Comparing Product A and B

• An exaggerated example compares two products' retention curves—Product A shows strong initial retention while Product B appears stable despite lower numbers.

• As more data reveals trends over time, perceptions may shift; Product B’s stability could indicate long-term viability despite early losses compared to Product A’s decline.

Cohort Retention: Understanding the Key Metrics

Importance of Cohort Curves

• The primary focus for cohort retention is whether the curves flatten out over time, rather than their absolute height. A flat curve indicates user retention and potential growth.

• Flat cohort curves allow for user accumulation over time, preventing a cycle of acquiring new users while losing existing ones, which leads to stagnation.

• If cohort curves do not flatten, it may indicate that the product does not meet user needs. Early Google Photos data showed a drop-off but eventually stabilized at 20%-40% retention.

• Despite initial high drop-off rates, Google Photos' ability to retain 20% of users gave confidence in its long-term success; this led to significant user growth over four years.

Common Pitfalls in Measuring Retention

Misleading Time Period Selection

• A common mistake is selecting overly broad time periods (e.g., quarters or half-years), which can artificially inflate retention metrics by counting inactive users as active.

• The speaker shares personal experience with this mistake when measuring weekly cohorts for Bump; widening the measurement period led to misleadingly positive results.

Shallow Engagement Actions

• Choosing too easy or shallow actions (like merely opening an app) can lead to inflated metrics. Users might engage without deriving real value from the product.

• An example from Google+ illustrates how notifications could create false engagement numbers without genuine user activity.

Revenue as a Measure of Activity

Understanding User Engagement and Retention

The Importance of Active User Tracking

• Many users subscribe to streaming services like Netflix but may not actively use them. It's crucial to track both payment and active usage to gauge true engagement.

Defining Good User Actions

• Founders should visualize a customer using their product to determine what constitutes a "good user." This perspective helps in defining meaningful actions for cohort retention analysis.

Analyzing Cohort Retention Curves

• Avoid focusing on single data points; instead, analyze the entire shape of retention curves over time. This holistic view provides better insights into user behavior.

• Founders often misinterpret retention metrics by only considering one week’s performance without understanding the broader trend, leading to misleading conclusions about product success.

Caution with Analytics Tools

• While analytics tools are beneficial, they may not accurately represent user cohorts or retention metrics. Founders should verify these numbers against their own data for accuracy.

• It is advisable for founders to create their own cohort retention curves using logs or spreadsheets initially, which fosters a deeper understanding of the data before relying on analytics tools.

Frequency of Data Review

• Regularly refreshing cohort graphs (weekly or bi-weekly) is essential. Early detection of declining trends allows for timely interventions to improve user retention.

Strategies for Improving Cohort Retention

Enhancing Product Features

• Improving product functionality—such as reducing latency or simplifying workflows—can lead to better user experiences and flatter retention curves over time.

Targeting the Right Users

• Acquiring users who align with your product's purpose is vital. Misalignment can result in poor retention rates, as seen in marketing efforts targeting younger demographics that did not resonate with Google Photos' core use case.

Learning from Historical Data

Understanding Cohort Performance and Retention

Improving User Acquisition and Product Performance

• Identifying target users is crucial for product performance; understanding who will use your product can significantly enhance cohort outcomes.

• If cohort curves are not stabilizing, consider segmenting cohorts by various dimensions such as country or customer size to identify performance discrepancies.

• Some cohorts may show strong retention while others do poorly, indicating areas for improvement in user acquisition strategies.

Enhancing First User Experience

• Focus on onboarding and activation processes; many overlook the importance of guiding users on how to effectively utilize the product.

• Understanding users' previous workflows before using your product can help tailor their experience and improve engagement.

Leveraging Network Effects for Retention

• Products that benefit from network effects—where each new user enhances the experience for existing users—can see improved cohort retention over time.

• Building dense networks around your user base can lead to better overall performance of cohorts.

The Ideal Scenario: Growing Cohort Curves

• The ultimate goal is to achieve cohort curves that not only stabilize but also increase over time, indicating growing user engagement with the product.

• Consistent improvements in targeting better customers and enhancing the product can lead to increased usage among retained users.

Visualizing Cohort Data: Layer Cake Chart

• Transitioning from relative month data to absolute time allows a clearer view of user retention across different cohorts over time.

• A "layer cake" chart visually represents active users segmented by their original cohort, showcasing growth from both recent acquisitions and long-term retention.

Importance of Qualitative Feedback

• Engaging directly with users provides qualitative insights that quantitative data alone cannot offer; this feedback is essential for understanding necessary changes in the product.