Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team)

Name: Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team)
Uploaded: 2024-12-15T12:00:41.000Z
Duration: 2 h 56 min 32 s

Duolingo's Streak Feature: A Deep Dive

Overview of Duolingo and Its Growth

Duolingo is valued at $14 billion, experiencing significant growth, having doubled in value over the past six months.

The "streak" feature is highlighted as the most impactful aspect of user engagement, with over 9 million users maintaining a streak of one year or more.

The primary focus for Duolingo is to help users build consistent habits around language learning.

Insights from the Journey of Streaks

Jackson Shuttleworth, a group product manager at Duolingo, discusses the importance of user retention and how streaks contribute to this goal.

The streak feature has been pivotal in driving usage and market cap growth, suggesting that it has created billions in value for the company.

Jackson shares insights on running over 600 experiments related to streak mechanics to understand what motivates users effectively.

Key Learnings from Experiments

One major change was shifting the call-to-action (CTA) from "continue" to "commit to my goal," which resulted in significant positive outcomes.

Understanding human psychology through these experiments has revealed critical motivators and demotivators for user engagement.

Internal Discussions on Streak Features

Jackson mentions that internal discussions about the streak feature are common during team onboarding but have not been shared externally before this episode.

He emphasizes that there’s a wealth of knowledge gained over years regarding what works with streak features.

Conclusion and Future Directions

The conversation aims to provide comprehensive insights into how Duolingo developed its streak feature, referred to as "the Motherload of Learnings."

What Are Duolingo Streaks and Their Impact?

Overview of Duolingo Streaks

Duolingo streaks track the number of consecutive days a user completes a lesson on the app, starting from their first lesson.

The feature includes flexibility through "streak freezes," which act as insurance for users to maintain their streak even if they miss a day.

Initially simple, the streak feature has evolved with added challenges, goal setting, rewards, and social features that enhance user engagement.

Importance of Streaks in User Retention

Streaks are considered one of Duolingo's most significant features for user retention and growth, alongside core lessons.

The concept of streaks serves as an effective engagement hack; any app can implement similar features to retain users if executed well.

Notifications related to streak performance significantly contribute to daily active users (DAUs), making it a crucial driver for user engagement.

Statistics and Growth Insights

Over 9 million users have maintained a year-long streak on Duolingo, indicating strong user commitment and engagement.

As of recording, Duolingo is valued at $14 billion and has seen substantial stock growth over recent months, highlighting its market success linked to features like streak tracking.

Evolution of the Streak Feature

The original version of the streak was based on experience points (XP), where users set goals tied to XP accumulation rather than simply completing lessons.

This initial setup allowed for potential frustration if users set overly ambitious goals leading them to lose their streak despite engaging with the app regularly.

A pivotal change occurred when the system shifted from XP-based goals to rewarding users for completing just one lesson per day, simplifying participation.

Conclusion on User Engagement Strategies

The transition towards a more accessible model helped retain users by reducing barriers while still connecting progress with personal language learning goals.

Understanding User Engagement and Streaks in Duolingo

The Importance of Streaks

Users often care less about features that don't align with their goals; simplifying the streak feature significantly increased Daily Active Users (DAUs).

The unit of use for Duolingo is completing a lesson, which is crucial for user engagement. Transitioning from an XP-based streak to a one-lesson streak made it easier for users to understand and engage.

A simpler one-lesson streak allows more users to grasp the concept easily, enhancing overall user experience.

Experience Points (XP) System

XP, or experience points, are awarded based on user performance in lessons and drive various app features like leaderboards.

While XP drives many features, the focus on streak mechanics shifted away from this system to enhance user retention.

Balancing Learning and Engagement

Luis's perspective emphasizes prioritizing engagement over learning; without returning users, there can be no learning opportunities.

Retention is critical; if users do not return daily, they miss out on learning experiences. However, dumbing down the experience could lead to disengagement.

Experimentation with Streak Mechanics

Continuous testing ensures that changes to the streak feature do not negatively impact the learning experience.

The transition from XP-based metrics to a simplified lesson-focused approach was part of a broader experimentation strategy at Duolingo.

Insights from Testing

Further experiments aimed at making it even easier to extend streaks revealed that merely doing one exercise did not increase DAUs as expected.

Simplifying the unit of measure led to capturing less engaged users who were unlikely to remain active long-term.

Lessons Learned

Understanding user commitment levels is essential when designing features like streak systems; targeting less committed users can dilute engagement efforts.

Over-indexing on certain types of users may result in ineffective strategies that fail to retain valuable participants in the app.

Ongoing Experimentation Culture

Understanding User Retention Strategies

Experimentation and Learning from Failures

The speaker emphasizes the importance of experimentation, especially in the early stages of user engagement. Testing various approaches can yield surprising insights about user preferences.

Acknowledges that while many experiments may fail (about half), valuable lessons are still learned, highlighting a good success rate compared to industry standards.

Discusses the policy on neutral experiments; typically, if an experiment does not show positive results, it is shut down to avoid unnecessary complexity in the app.

Focus on Early User Experience

The speaker stresses focusing on enhancing the zero to seven-day user experience as critical for retention.

Data analysis shows significant retention improvements when users reach a seven-day streak due to loss aversion psychology.

Highlights that while longer streaks are beneficial, the most substantial retention gains occur within the first week.

Implementing Goal Setting Features

Introduces a feature called "streak goal," which encourages users to achieve specific streak lengths as a motivational tool.

Emphasizes starting with simple versions of features rather than complex designs, allowing for iterative improvements based on user feedback and performance.

Leveraging Psychological Insights

Draws parallels between successful monetization strategies and user engagement tactics by using compelling messaging about increased likelihood of course completion with longer streaks.

The initial messaging around achieving a 30-day streak proved effective in motivating users and enhancing their commitment.

Tailoring Goals for Different Users

After testing various goal lengths (14 days vs. 30 days), it was found that different goals appeal to different segments of users.

Introduced an opt-out option for users who feel they cannot meet their initial goal, which surprisingly led to higher engagement rates by allowing intentional decision-making regarding commitments.

Insights on Goal Setting and User Engagement in App Design

The Importance of Optionality in Goal Setting

The concept of optionality was a significant insight, leading to the development of a goal-setting feature that allows users to choose between different goals. This flexibility proved beneficial for user engagement.

Friction and User Experience

Initially, the team believed that recommending harder goals would enhance retention. However, they discovered that allowing users to select their own goals led to greater engagement.

Human Psychology in Motivation

Understanding human psychology is crucial for motivating users. The speaker feels like an amateur psychologist when it comes to understanding language learners' motivations.

Experimentation as a Learning Tool

The approach taken by Duolingo involves extensive experimentation, akin to mining for gold; once a successful strategy is identified, further testing ensues to maximize its potential.

Data-Driven Hypothesis Testing

Experiments are not random; they stem from insights gained through previous tests. A strong hypothesis guides the design of new experiments based on what has worked before.

Copy Testing and Its Impact

Effective copy testing infrastructure allows for constant experimentation with text changes. Small adjustments can lead to significant improvements in user interaction.

Case Study: Call-to-Action Changes

Changing the call-to-action from "Continue" to "Commit To My Goal" resulted in substantial user engagement increases, demonstrating how minor copy changes can have major impacts.

Review Process for Product Changes

Understanding Streaks in User Engagement

The Importance of Strategy in Feature Development

Emphasizes the necessity of having a clear strategy for feature development to avoid stagnation and local maxima.

Discusses the need for launching neutral experiments to reset features based on learnings, aiming for simplicity while maintaining effectiveness.

Highlights the challenge of optimizing features without creating overly complex or cluttered designs.

Unique Advantages of Language Learning Apps

Points out that language learning apps like Duolingo have an inherent advantage as users are motivated to return to the app for personal growth.

Compares user engagement in mobile games versus language learning, noting that streak achievements may resonate differently across contexts.

Framing Streaks Effectively

Suggests that companies must creatively frame streak concepts so users feel positively about their progress, which is easier for educational apps like Duolingo.

Notes that tracking daily progress in language learning is difficult; thus, streak metrics provide tangible motivation over long-term goals.

Lessons Learned from Iterating on Streak Features

Recaps the evolution of Duolingo's streak feature from initial XP ideas to more structured goals and user commitments.

Acknowledges common misunderstandings among users regarding how streak mechanics work, indicating a need for better explanations.

Enhancing User Understanding through Clear Communication

Shares insights on improving user retention by simplifying explanations around what a streak entails and how it functions within the app.

Understanding User Retention at Duolingo

Key Metrics and Goals

The speaker discusses the importance of providing a reference point for measuring success, specifically mentioning over 10,000 Daily Active Users (DAUs) as a significant win for Duolingo.

The first newsletter post by Jorge, former head of product, remains the most popular in their history. It emphasizes the significance of user retention in driving DAUs.

Duolingo's growth North Star is DAUs; current user retention rate (CURR) is identified as the most effective metric to drive this growth.

Focus on Retention Strategies

The retention team leads efforts focused on CURR, with the streak feature being highlighted as a key driver for improving this metric.

Incremental impact and absolute numbers are both measured to assess new daily users attributed to experiments. Day one versus day seven versus day fourteen retention metrics are particularly important.

Experimentation Insights

Absolute DAU numbers provide an easier comparison than percentage changes influenced by various biases. This method helps control for recency or novelty bias.

A discussion arises about whether some positive experiments should not be implemented into products despite initial success.

Case Study: XP Feature Experiment

An experiment involving adding XP display during lessons was initially seen as beneficial but ultimately deemed distracting from learning objectives.

The decision was made to prioritize simplicity in UI design over potential engagement wins that could detract from the core learning experience.

Long-term Considerations and Team Dynamics

Questions arose regarding future iterations of features like XP displays and how they might affect lesson experiences.

Ultimately, it was decided that maintaining a clear focus on learning was more valuable than short-term engagement boosts through added complexity.

Iconography and User Engagement

Understanding User Experience Research in Different Cultures

Cultural Context and User Engagement

The concept of commitment through visual metaphors, like a flame, may not resonate across different cultures or life stages. Insights from UXR in India highlighted this disconnect.

Conducting global user experience research (UXR) is crucial for understanding how users perceive and interact with features, leading to valuable insights that can inform design decisions.

Design Evolution: From Metaphor to Clarity

A redesign shifted focus from a flame metaphor to an odometer animation that clearly tracks user progress, emphasizing the importance of clarity in product design.

Product design should prioritize comprehensibility over cleverness; clarity enhances user understanding without sacrificing engagement or delight.

Balancing Clarity and Delight

While clarity is essential, it doesn't have to come at the expense of creating delightful experiences. Celebratory elements can still be integrated into clear designs.

Understanding user experience varies based on their engagement level; new users need straightforward guidance while experienced users can handle more complexity.

Visual Communication and Feature Understanding

Enhancements like making streak screens resemble calendars help communicate daily mechanics effectively, reinforcing the feature's purpose visually.

The Role of Flexibility in User Retention

Importance of Flexibility in Streak Mechanics

Offering flexibility within streak mechanics—like streak freezes—has proven beneficial for user retention and daily active usage (DAU).

Celebrating Progress Amidst Challenges

The "Streak Serenity Prayer" embodies the philosophy of balancing flexibility with striving for perfection while celebrating small wins regardless of setbacks.

Experimentation with Streak Freezes

Understanding Streak Flexibility in Habit Formation

Insights from Streak Freeze Experiments

The experiment revealed that having three streak freezes did not significantly improve user retention compared to two, indicating a threshold for flexibility in habit formation.

While more users returned after longer absences with an intact streak, the risk of users taking extended breaks increased, suggesting a balance is needed in offering flexibility.

The right amount of flexibility varies throughout a user's streak journey; initial stages benefit from more leniency to encourage consistency.

A significant finding was that providing two streak freezes at the start of a new streak greatly improved user retention and commitment to maintaining their habits.

Users need to reach around seven days for their commitment to solidify; thus, early support is crucial for long-term engagement.

Long-Term Streak Management

As users build longer streaks, reducing flexibility becomes important; they tend to care more about maintaining their progress and are less likely to take unnecessary breaks.

Understanding user behavior is essential; many users will have access to services and should not be conditioned to take breaks unnecessarily.

Monetization vs. Retention Strategies

Duolingo allows users to buy streak freezes using gems, raising questions about whether this feature serves monetization or retention purposes.

There’s an ongoing tension between making it easier for users to buy features while ensuring that these features do not detract from overall user engagement and satisfaction.

Earning Back Lost Streaks

The previous model required payment for restoring lost streaks but has shifted towards an "Earn Back" system where completing lessons can restore progress without cost, enhancing perceived value.

This approach fosters a sense of achievement among users as they feel they have earned back their progress rather than simply being given it back through payment.

Philosophical Considerations on Streak Integrity

Balancing the provision of flexibility with the integrity of the streak is crucial; too much leniency may lead users to undervalue their achievements over time.

Maintaining the sanctity of the streak involves careful consideration of how many supports are offered without compromising its significance in the eyes of users.

Understanding User Retention and Notification Strategies

The Importance of Streaks for User Engagement

The concept of streaks is crucial for user retention; losing interest in streaks can lead to a significant drop in engagement, described as an "extinction level event."

Having dedicated individuals, like Antonia and Luis, to oversee the integrity of these streaks is essential to prevent overstepping boundaries that could alienate users.

Managing Push Notifications Effectively

Companies often struggle with the balance of sending too many push notifications; establishing policies can help manage this load.

It's important to rely on intuition and user behavior rather than rigid policies when determining notification strategies.

Timing and Relevance of Notifications

A unique practice reminder is sent 23.5 hours after a user's last practice session, based on observed behavior rather than user-set preferences.

This timing reflects real-life schedules better than fixed reminders, acknowledging that personal circumstances can change unexpectedly.

Positive Perception of Reminder Notifications

A "streak saver" notification at 10 PM serves as a gentle reminder for users who haven't extended their streak yet; it’s perceived positively due to the emotional attachment users have towards their streak.

Users generally view this late-night reminder as helpful rather than intrusive, reinforcing their commitment to maintaining their progress.

Celebrating Achievements: The Perfect Streak Feature

Introducing features like "Perfect Streak," which visually enhances the user's progress bar without tangible rewards, encourages continued engagement by celebrating consistency.

Streak Management and User Experience Insights

Importance of Streaks

The speaker emphasizes the significance of maintaining a perfect streak, suggesting that users should avoid using the "streak freeze" feature to keep their streak intact.

A personal anecdote is shared about an employee who lost a four-month perfect streak while traveling across time zones, highlighting how much users value their streaks.

Balancing Flexibility and Perfection

The discussion revolves around finding ways to encourage users to return to perfection while still offering flexibility in managing their streaks.

There are considerations for developing a self-service feature that allows users to regain lost streaks based on valid excuses.

Enhancing User Experience through Animation and Haptics

The power of animations and user experience is discussed as a means to celebrate user achievements without relying solely on textual feedback.

Testing has been conducted on haptic feedback (phone vibrations), which can enhance the user's emotional connection with their progress.

Creating Memorable Moments

The speaker advocates for creating moments where users pause and appreciate their achievements, rather than rushing them through the app's features.

Effective use of animations and haptics can deepen user engagement with their streak, making it more meaningful.

Design Process for Haptic Feedback

Initially, product designers handled haptic design; however, this role has evolved into collaboration with specialized animation studios for better motion design integration.

Hiring challenges are noted regarding finding qualified candidates specifically skilled in haptic design due to its unique requirements.

Managing Feature Complexity

The conversation touches on the potential issue of feature bloat when introducing multiple concepts related to streak management too early in the user experience.

How Does Duolingo's Team Operate Effectively?

Key Lessons from Duolingo's Team Experience

The discussion begins with a focus on how teams can effectively manage numerous experiments, highlighting the importance of operational efficiency in achieving goals.

Duolingo emphasizes metric-based team structures, primarily focusing on metrics like CURR (current users retained) and DAUs (daily active users), which guide their work and priorities.

A strong focus on specific metrics helps teams prioritize high ROI tasks rather than getting lost in vague notions of improving features. This clarity directs efforts towards measurable outcomes.

Teams are organized around metrics or goals rather than specific features. For example, while retention focuses on streaks due to their impact on CURR, other teams can contribute to this area as needed.

Collaboration between teams is encouraged; if another team has ideas for improving streak-related features, they can pitch them to the retention team for consideration and integration into existing plans.

Operational Processes and Experimentation

The concept of "soft ownership" allows multiple teams to engage with shared features like streaks while maintaining clear accountability for overall goals.

Effective collaboration requires logistical planning; when multiple teams want to work on a feature, they must align their efforts with existing roadmaps and objectives.

Knowledge sharing is crucial when different teams interact. Understanding each other's context helps ensure that new ideas are relevant and beneficial without disrupting ongoing projects.

Antonia, the team lead, emphasizes the need for a structured process in running experiments. This includes careful planning about which experiments to run and how they connect with future initiatives.

Detailed roadmaps are essential for coordinating engineering and design efforts across various functions. This ensures that all aspects of product development are aligned toward common goals.

Importance of Planning and Coordination

Planning extends months ahead; anticipating future iterations prevents lost opportunities by ensuring that new features are ready for rollout immediately after previous ones conclude.

Understanding the Importance of Simplifying Features

The Value of a Minimal Viable Product (MVP)

Emphasizes the importance of resisting the urge to create an overly complex V1 product, advocating for a focus on core hypotheses.

Suggests stripping away unnecessary features to identify and ship the essential elements first, which facilitates faster design and approval processes.

Highlights that simplifying features not only eases shipping but also enhances clarity in design, allowing teams to iterate more effectively.

Iterative Development and Hypothesis Testing

Discusses how adopting a mindset focused on simplification leads to quicker iterations and increased confidence in hypotheses as they are tested.

Stresses the importance of dropping non-essential parts of features based on testing outcomes, ensuring that only valuable components are shipped.

Mapping Metrics for Growth

Identifying Key Growth Drivers

Describes mapping all business levers that drive growth, with a specific focus on metrics like CURR as pivotal for long-term success.

Encourages exploring various opportunities related to key metrics and developing strategies through hypothesis-driven experimentation.

Continuous Improvement Mindset

Shares an anecdote about underestimating potential improvements in existing features (like streak tracking), emphasizing ongoing optimization efforts.

Reflects on past experiences where significant strides were made over time, reinforcing the idea that there is always room for further enhancement.

Engagement Strategies: The Role of Streaks

Understanding User Retention

Argues that streak mechanics can enhance user engagement but should be implemented only if users find value in using the app regularly.

Warned against focusing too heavily on streak implementation without ensuring a solid core experience; otherwise, it may distract from critical issues.

Core Value Proposition

Reiterates that before layering engagement tactics like streak tracking, it's crucial to establish an enjoyable user experience that encourages daily use.

Understanding the Importance of Streaks in User Engagement

The Significance of Streaks

Users care about their streaks because Duolingo emphasizes them significantly, showcasing an animated streak screen after each session that captures attention.

If a streak feature is hidden within the app, users are less likely to value it; visibility is crucial for user engagement and motivation.

Making streaks top-of-mind for users through consistent reminders and notifications can enhance their importance and impact on user behavior.

Testing and Implementation Strategies

Duolingo employs a philosophy of testing various aspects of the streak feature to determine what resonates with users, advocating for experimentation over theoretical discussions.

Understanding human psychology is essential when designing features like streaks; direct user feedback through testing is more effective than prolonged ideation sessions.

Applicability Across Different Apps

The speaker believes that most apps could benefit from implementing a streak feature, although the execution must align with specific user use cases.

While some applications (like tax software) may struggle to incorporate a meaningful streak due to infrequent usage patterns, many others can successfully implement this concept.

Tailoring Streak Features

Companies should analyze their ideal usage frequency and design their streak features accordingly; Peloton's weekly workout streak serves as an example of adapting to user needs during challenging times like COVID.

A well-designed streak can significantly contribute to a company's success, as evidenced by Duolingo's growth attributed largely to its effective implementation of this feature.

Final Thoughts and Recommendations

Insights on Life and Work from a Software PM

Reflections on Reading and Entertainment

The speaker expresses enjoyment in reading books that highlight the differences between software and hardware roles, emphasizing the challenges of hardware development.

Discusses watching "Bluey" with their children, noting its joyfulness, while also mentioning enjoyment of "Emily in Paris," despite it not being highbrow television.

Shares personal growth through learning French via Duolingo, appreciating the ability to understand spoken French after investing time in the language.

Product Discoveries and Innovations

The speaker humorously reflects on a tweet questioning how a character in "Emily in Paris" can afford to stay there indefinitely.

Highlights a recent purchase of an innovative ladder from Home Depot that adjusts for uneven ground, significantly improving safety during home maintenance tasks.

Emphasizes the importance of recognizing innovations like ladders that enhance everyday life but often go unnoticed.

Life Motto and Team Traditions

Shares a favorite life motto: "You miss 100% of the shots you don't take," advocating for experimentation even when success isn't guaranteed.

Describes a quirky team tradition at Duolingo where team members synchronize clapping over Zoom during standups, which started during remote work due to the pandemic.

Mentions adding a fun element by shouting "yee-haw" together post-clap, highlighting how small traditions contribute to team culture.

Engagement with Audience and Hiring Opportunities

Reflecting on how difficult it was initially to get everyone involved in shouting “yee-haw,” but now it's embraced as part of their culture.

Invites listeners to connect via LinkedIn for more information about Duolingo's hiring opportunities, particularly for product managers interested in streak features.

Expresses interest in hearing from companies that have successfully implemented streak features differently than Duolingo has.

Closing Thoughts and Acknowledgments

The speaker appreciates insights into streak implementations from other companies as they continue to evolve this feature at Duolingo.

Concludes by acknowledging the impact their team has made within product management circles, celebrating ongoing successes.