Lecture 16 - How to Run a User Interview (Emmett Shear)

How to Conduct Effective User Interviews

Introduction to Emmett Shear and the Topic

• Emmett Shear, CEO of Twitch, discusses the importance of user interviews in startup development.

• He shares his early experiences with his first startup, Kiko Calendar, highlighting a lack of user engagement.

Lessons from Early Startups

• Shear reflects on the shortcomings of not talking to users during the development of Kiko Calendar.

• In his second startup, Justin.TV, he notes that being their own consumers allowed them to bypass some user feedback but limited broader market understanding.

The Pivot to Gaming and User Engagement

• Justin.TV's success stemmed from recognizing a larger audience interested in live broadcasting.

• The company pivoted towards gaming after realizing they needed insights beyond their initial use case.

Importance of User Interviews

• Shear emphasizes that conducting user interviews was crucial for identifying valuable features for Twitch.

• They engaged extensively with potential users to gather data that informed product decisions over three years.

Identifying Key Users

• A dedicated team was established at Twitch specifically for user engagement—something missing at Justin.TV.

• Broadcasters were identified as key users; understanding their needs shaped platform development significantly.

Strategic Insights from User Feedback

• The distinction between talking to viewers versus broadcasters highlighted different insights and priorities in product design.

Interactive Note-Taking Process

Engaging the Audience in Idea Generation

• The speaker introduces an interactive segment, encouraging participants to think about whom they would consult regarding a new note-taking app idea.

• The focus is on improving the current state of note-taking, particularly for lectures and online classes, with potential features like collaboration and multimedia enhancements.

Identifying Key Stakeholders

• Participants are prompted to take 120 seconds to identify five people or types of people who could provide valuable feedback on the app concept.

• Emphasis is placed on writing down thoughts rather than just thinking them through, reinforcing the importance of active engagement in the brainstorming process.

Understanding User Needs

• The speaker highlights that identifying users and where to find them is crucial for any startup's success. This task can be challenging but essential.

Volunteer Insights: Target Audience Exploration

• A volunteer shares their intention to speak with college students as a primary user group due to their frequent attendance at lectures.

• The volunteer considers different academic disciplines (e.g., English vs. Computer Science majors) and study techniques when selecting interview subjects.

Expanding Beyond Immediate Users

• The discussion shifts towards understanding diverse note-taking habits among students, noting that some take extensive notes while others may jot down minimal information.

• The speaker warns about targeting only college students since they often have limited financial resources; thus, it’s important to consider other stakeholders who might influence purchasing decisions.

Exploring Additional Influencers

• Suggestions include engaging with IT administrators at colleges who may facilitate software purchases for students.

• Parents are identified as another potential audience since they may be willing to invest in tools that enhance their children's educational productivity.

Broadening Perspectives for Startup Success

User Interview Insights on Note-Taking Habits

Understanding Diverse Note-Taking Approaches

• The importance of engaging with various individuals to gain a comprehensive understanding of the note-taking landscape is emphasized.

• A user interview begins with Stephanie, a college student, discussing her note-taking habits and preferences.

Current Note-Taking Methods

• Stephanie mentions using both laptop and pen-and-paper methods for taking notes, depending on the subject matter; text-based notes for History and diagrams for Physics.

• She highlights that while she uses pen and paper for some classes, she prefers digital tools due to their accessibility and ease of collaboration.

Software Preferences

• Stephanie primarily utilizes Google Docs and Evernote for her note-taking needs; Google Docs is favored for collaborative projects while Evernote is used for personal collection.

• She explains that Google Docs allows easy sharing within group projects, whereas Evernote serves as a personal repository.

Collaboration and Personalization in Notes

• Stephanie expresses a desire for better integration between different note-taking platforms, indicating that she often merges notes from various sources to suit her organizational style.

• She reveals that she tends to review her own organized notes more than those from others due to her specific formatting preferences.

Review Practices Post-Semester

• After completing a semester, Stephanie does not typically revisit class notes but does refer back to notes taken during talks or interviews to keep information fresh in her mind.

• She takes additional notes outside of class settings to summarize key points or inspirational quotes from events she attends.

Key Takeaways from User Interviews

• The interviewer acknowledges the need for deeper exploration into collaboration dynamics and volume of notes but decides against prolonging the discussion with one participant.

Understanding User Feedback and Feature Development

The Challenge of User Requests

• When engaging with users about features, it's crucial to avoid focusing solely on their requests. Real user feedback can feel compelling, making it difficult to decline feature suggestions.

• Initial interviews may not reveal significant problems or a strong need for new products. It's essential to assess whether the issues raised are substantial enough to warrant development.

• A single negative response from one user doesn't mean there isn't a broader issue; further discussions with diverse groups can uncover more insights.

Importance of Diverse Perspectives

• Engaging with a variety of users (e.g., students vs. parents) is vital as different demographics will provide varied responses and insights into their needs.

Ideation Based on User Insights

• After gathering information, consider what unique feature could enhance existing tools like Google Docs. This approach helps in brainstorming potential product ideas.

• Focus on improving current experiences by identifying one key enhancement that would significantly benefit users and encourage them to switch platforms.

Validating Ideas Before Development

• Once an idea is formed, validate its appeal by considering how it might convince users to transition from familiar tools like Google Docs to your new solution.

• An example discussed involves creating a mobile-friendly note-taking feature that complements collaborative work while addressing specific user needs for smaller notes.

Testing Concepts with Users

• The importance of user feedback is highlighted when discussing the integration of collaborative aspects into note-taking applications, ensuring they meet real-world demands.

• Before building a product, it's advisable to sketch out workflows and gather feedback without leading questions about specific features—this avoids misleading positive responses that don't translate into actual interest once developed.

What is the Minimum Viable Product for Validation?

Understanding Minimum Viable Product (MVP) and Validation

• Sam inquires about the minimum requirements for validating a product without directly asking users for feedback. The answer varies based on the feature being developed.

• Instead of creating a full-fledged application, it's suggested to develop a simple prototype or browser extension that adds incremental features to test user interest and utility.

• For products requiring monetary commitment, obtaining credit card information serves as strong validation; if users are willing to pay upfront, it indicates genuine interest in the product.

• However, with free trials or student applications, it may be challenging to gauge excitement through payment alone since users might not be inclined to spend money initially.

Insights from Twitch Feedback

• The speaker shares insights from their experience at Twitch, emphasizing the importance of gathering user feedback during development phases.

• A condensed version of extensive feedback collected from broadcasters highlights common requests and issues faced by users on Justin.tv before transitioning to Twitch.

Key User Requests and Pain Points

Detailed User Feedback Analysis

• Engaging with active broadcasters revealed specific feature requests such as clearing ban lists in chat rooms and editing highlight titles post-creation.

• Consistent themes emerged regarding interactivity features like polls and personalized chat options that competitors offered but were lacking in Twitch's initial offerings.

Evaluating User Pain Points

• Despite numerous requests for improvements, existing users' willingness to tolerate certain limitations suggests these issues may not be critical pain points.

• This observation raises an important consideration: understanding which problems truly matter requires comparing feedback from current users against those using competing services.

Competitor Insights and Market Needs

Competitor Broadcaster Feedback

• Feedback from broadcasters using other platforms highlighted desires for multi-user channel switching capabilities and revenue-sharing programs essential for monetization efforts.

Understanding User Needs in Gaming Broadcasting

Importance of User Feedback

• The focus on user feedback revealed that many potential users were deterred from using the service due to significant issues, highlighting the need for improvement.

• Engaging with an informed user base provided valuable insights, as they had already explored multiple competing services and formed opinions about them.

Engaging Non-Users

• Conversations with non-users proved crucial; understanding their perspectives helped identify barriers preventing them from adopting broadcasting services.

• Most competitors are non-users, emphasizing the importance of addressing their concerns to expand market reach effectively.

Identifying Barriers to Adoption

• Common reasons for non-use included technical limitations (e.g., computer performance), time constraints (e.g., training for tournaments), and strategic concerns (e.g., revealing strategies through practice broadcasts).

• Specific cultural factors, such as competitive practices in Korea, further complicated the adoption of broadcasting among gamers.

Strategic Outreach and Development

• The outreach program involved providing resources like computers and collaborating with software companies to enhance broadcasting capabilities.

• Integrating broadcasting features into popular gaming platforms like Xbox and PlayStation 4 was a key strategy to simplify access for users.

Focus on Core User Goals

• Rather than solely implementing requested features, the team prioritized addressing fundamental user goals: monetization, stability, quality, and universal access.

• This approach led to successful product developments that aligned more closely with actual user needs rather than just surface-level requests.

Effective User Research Practices

• Following up with interviewees after implementing changes demonstrated responsiveness and built trust; users appreciated seeing their feedback translated into actionable improvements.

• The initial adopters were often those who participated in research discussions, indicating that targeted engagement can lead to higher conversion rates.

Lessons from Previous Experiences

• Reflecting on past experiences at JustinTV highlighted the limitations of relying solely on data analytics without direct user interaction; this often resulted in misguided product development efforts.

User Interviews: The Hard Truth

The Disappointment of User Feedback

• Conducting user interviews often leads to disappointing revelations, as users may not value the features you are passionate about.

• Many startups avoid user feedback due to the fear of negative responses regarding their favorite features.

Common Mistakes in User Interviews

• A prevalent mistake is showing users your product instead of understanding their existing thoughts and needs.

• Asking leading questions about specific features can skew results; it's crucial to explore users' genuine needs instead.

• Engaging with easily accessible users rather than the target demographic can result in unrepresentative data.

Importance of Targeted User Engagement

• Investing time in identifying and reaching out to the right users yields more valuable insights than simply talking to those who are easy to access.

• Users generally appreciate being asked for their opinions, which can facilitate better engagement during interviews.

Gaining Company Buy-In for User Insights

• Presenting findings from user interviews is challenging without trust; recording interviews can help convey insights effectively.

• Sharing recorded interviews with team members enhances understanding and acceptance of user feedback within the company.

Best Practices for Conducting Interviews

• Prefer video calls (e.g., Skype) over email for conducting interviews, as they allow for interactive discussions that yield deeper insights.

• Engaging in real-time conversations encourages participants to elaborate on unexpected points, providing richer data.

Challenges with International Markets

• Conducting user interviews in non-English speaking countries presents difficulties; language barriers hinder effective communication and understanding.

Understanding User Feedback in Product Development

The Challenge of Representing Non-Fluent English Speakers

• The speaker emphasizes that fluent English speakers do not represent the entire population, highlighting the difficulty companies face when trying to engage users abroad.

• Companies often find it easier to build markets in their home countries due to challenges in communicating with international users.

Channels for User Outreach

• The primary channels used for reaching out to users included on-site messaging systems and direct emails after meeting at events.

• Engaging users involved asking them questions about their experiences via platforms like Skype, rather than compensating them financially.

• The speaker believes that if individuals are not interested enough in a problem to discuss it, pursuing them may be unproductive.

Importance of On-Site User Feedback Tools

• On-site user feedback tools are crucial for assessing new products and understanding user interactions before launch.

• While these tools help identify issues with existing products, they do not provide insights into what problems should be addressed or what users truly care about.

Focus on Competing Products

• Given limited resources, the focus was placed on users of competing products who were already engaged and could be convinced to switch services.

• This strategy aimed for quick wins necessary for sustaining growth within the startup environment.

Evolution of Engagement with Game Publishers

• Initially, game publishers were not responsive to small startups; however, as Twitch gained traction, engagement increased significantly.

User Interviews: The Importance of Continuous Engagement

Shifting User Demographics Over Time

• The initial user base that helps launch a product may not represent the long-term users; it's crucial to adapt and engage with new users as the product evolves.

• Regularly conducting user interviews is essential to understand changing needs and preferences, preventing stagnation in feature development.

• A common pitfall is ceasing communication with users after early success, which can lead to less effective future features.

Providing Effective User Feedback

• Users should be encouraged to share their thoughts freely during feedback sessions, allowing for deeper insights into their experiences and challenges.

• The goal of user interviews is to gather comprehensive narratives from users about their lives, which aids in understanding their motivations and needs.