How this PM Used Claude Code to Support 20 People

What is the Future of Product Management?

The Role of AI in Product Management

• The speaker has invested over 1,500 hours using Claude and continues to refine their setup daily.

• Hannah Stalberg, a PM at DoorDash and former Google APM, emphasizes that many PMs give up too early when utilizing AI tools like Claude for product work.

Insights on AI Tools for PMs

• Stalberg identifies "following your curiosity" as an underhyped aspect of AI in product management.

• She describes "cloud code" as misleading because it implies exclusivity to coding tasks, while its applications extend beyond that.

Evolving Responsibilities of PMs

• The discussion highlights how PM roles are expanding; one PM now often supports more engineers than before (up to 10).

• There’s a shift where PM responsibilities are merging with those of engineers and designers, leading to collaborative decision-making across teams.

Creating a Team Operating System (Team OS)

• Stalberg introduces the concept of a "team OS," which serves as a centralized knowledge base for team members.

• This system includes shared folders for agents, commands, product development documents, and team-level resources like onboarding guides.

Structuring the Team OS

• The team OS consists of three main parts: cloud folder (shared agents), product development folder (subfolders by function), and team folder (documents).

• At the root level is the Claude MD file that contains essential components such as a doc index for navigation and information about team members' roles and communication channels.

Understanding Claude MD Files and Context Management

The Purpose of Claude MD Files

• Claude MD files are designed to facilitate natural language queries within Slack channels, allowing users to direct information flow efficiently.

• A lean approach is recommended for Claude MD files, especially in team repositories, to avoid clutter and enhance performance.

Key Concepts of Context Management

• Context refers to the information available in a session with a Language Learning Model (LLM), crucial for effective interaction.

• The context window defines how much information an LLM can hold; recent advancements allow up to a million tokens, equating to several novels' worth of text.

• Compaction occurs when the context window fills up, leading to compressed summaries that may lose fidelity and usefulness.

• Thinking room is the difference between available information and the size of the context window; too much information reduces reasoning capacity.

Structure and Functionality of Repository

• The repository is structured to help Claude access relevant information at appropriate times during tasks, enhancing efficiency.

• Nested Claude MD files serve as document indexes that guide Claude in navigating through folders without unnecessary searches.

Practical Examples of Using Claude MD Files

• In practical scenarios, users can query specific customer-related data by leveraging structured folders containing relevant documents.

• Efficient navigation allows Claude to find precise answers while consuming minimal context from the overall dataset.

Optimizing Context Usage

• Effective use of Claude MD files involves minimizing unnecessary context consumption while ensuring relevance to queries posed by users.

• Users should focus on pertinent details related only to their inquiries—e.g., customer data—without delving into unrelated areas like SQL queries.

Structuring Customer Information

• Organizing customer data systematically enables efficient retrieval; key contexts about customers should be readily accessible for frequent sessions.

• Each customer's file contains essential insights that support ongoing analysis and decision-making processes.

Understanding the Efficiency of Information Structuring

The Role of Summaries in Data Processing

• The system primarily reads summary files instead of individual transcripts, which can be lengthy (over an hour). This limitation affects Claude's ability to synthesize multiple transcripts quickly and accurately.

• Prototyping with Bolt allows for the creation of real front-end code that integrates seamlessly with existing design systems, avoiding the common issue of throwaway prototypes.

Enhancing Product Management with Tools

• Users find Bolt engaging enough to work late into the night, indicating its effectiveness and user-friendliness.

• Jira Product Discovery is introduced as a tool that helps product managers manage discovery and prioritization effectively, moving away from spreadsheets.

Structuring Information for Better Access

• Adding more structured summary files aids Claude in collecting relevant information faster and more efficiently.

• Organizing information in a consistent format makes it easier for Claude to access necessary data quickly, especially when dealing with unstructured inputs.

Consistency in Customer Call Summaries

• Teams should standardize customer call summaries to follow a uniform format, facilitating easier synthesis by Claude across numerous calls.

• Implementing a shared skill file for summarizing customer calls ensures consistency even when different team members conduct interviews.

Leveraging Structured Inputs for Analysis

• By structuring summaries uniformly, teams can enhance their analytical capabilities across various customer interactions despite differing interviewing styles.

• A detailed analysis can be generated rapidly through natural language queries about past meetings or learnings.

Comprehensive Repository Organization

• A well-organized product folder may include competitive research, meeting summaries, PRDs, sales enablement materials, and strategic documents.

• Workflows are established as repeatable processes stored within cloud MD folders to streamline complex operations.

Effective Workflow Execution

• Multi-step operations are documented clearly within the repository to facilitate easy execution during meetings or document synthesis.

• Understanding how each part of a process contributes to final documentation is crucial for maintaining efficiency in workflows.

Understanding Team Organization and Data Analysis

Importance of Team Organization

• The speaker emphasizes the significance of team organization, stating that it helps everyone scale themselves and leverage their teammates' strengths.

• A well-organized analytics folder can include links to dashboards, experiment analyses, metrics playbooks, queries, and schemas to enhance data analysis across the team.

Structuring Analytics Folders

• The speaker suggests organizing folders by topic area and product area for clarity. An example is given with a fictional company called Forge Labs.

• Metrics related to billing are outlined in the repository, including relevant dashboards and SQL queries for accessing these metrics.

Accessing Data for Analysis

• The repository provides all necessary references for PMs (Product Managers) to conduct accurate data analysis without needing direct assistance from analysts.

• By querying specific metrics like generation success rate, users can access definitions, SQL queries, and table schemas directly from the repository.

Complexity of Data Tables

• As products mature beyond early stages, data tables become complex; thus having structured guidance is crucial for accurate querying.

• The repository's structure allows tools like Claude to reference appropriate files when performing queries on complex datasets.

Enhancing Funnel Analysis

• Users can inquire about user drop-off during processes like custom domain setup; the repository contains playbooks that guide funnel analysis effectively.

• This structured approach reduces concerns over inaccurate data or hallucinations since all information used has been verified by analysts or data scientists.

Collaborative Repository Development

• The entire team should contribute to building the repository; collaboration between PMs and analysts ensures comprehensive documentation of features and metrics.

• This collaborative effort empowers engineers as well as PMs to validate feature performance independently using documented queries.

Context Management in Queries

• Proper organization allows users to ask specific questions about metrics without retrieving unnecessary details about queries or schemas.

• This efficiency aids in writing PRDs (Product Requirement Documents), allowing easy access to defined metrics throughout product development.

Metrics and Repository Management in Engineering

Importance of Metrics Updates

• Discussion on which metrics to update or add, emphasizing the structured context available by feature in the repository.

• Engineers are encouraged to be independent from business managers (BMs) for data reports, with PMs acting as facilitators to ensure analytics alignment.

Feature Launch Process

• The repository must be updated before a feature launch, ensuring shared context for effective team operations as product complexity increases.

• The repository serves all functions within the team, including bug investigations and technical design documents (RFCs), promoting shared historical context.

Bug Investigation Documentation

• Storing bug investigation details helps future investigations by providing insights into past issues, including root causes and resolutions.

• Comprehensive documentation includes analysis methods and data examples, aiding engineers in efficiently addressing recurring bugs.

Ownership of Knowledge Repository

• All team members share ownership of the knowledge repository; functional leads guide specific areas while maintaining a collective agreement on information structure.

• Emphasis on updating the repository enhances team performance in an AI-driven environment through collaborative contributions.

Automation and Reporting

• Shared automations can synthesize customer research weekly, keeping teams informed about learnings without manual effort.

• Daily product work is checked into the repository using cloud code tools for collaboration across various roles beyond just technical positions.

Document Management Practices

• Team members draft documents in cloud tools before checking them into the repository for review, fostering a collaborative environment.

• Participation extends beyond technical roles; business operations also contribute to maintaining shared context within the repository.

Organizing Strategy Documents

• New strategy documents should be organized under designated folders (e.g., "vision" or "strategy docs") based on quarters for clarity.

• Pull requests (PRs) are used to submit work for review once ready, ensuring that appropriate team members evaluate specific types of documentation.

Implementing PRDs with Claude

Streamlining the Review Process

• An engineer on the team implements a Product Requirements Document (PRD) via a pull request (PR), with Claude facilitating the process by integrating GitHub commands.

• The system automatically posts updates to Slack, detailing who created the PR, its contents, and who should review it, enhancing communication within the team.

Collaborative Team Environment

• The approach fosters collaboration across various roles—analysts, designers, engineers, and go-to-market partners—creating an inclusive product development environment.

• A strategy partner collects customer feedback and documents insights in the repository for collective learning, demonstrating that non-technical members can contribute effectively.

Accessibility of Tools for All Team Members

• Emphasizes that using tools like GitHub is not limited to technical staff; anyone can learn to navigate these systems and improve teamwork.

• The speaker shares their method of interacting with GitHub through natural language queries, making it accessible even for those unfamiliar with coding.

GitHub 101: Key Takeaways

Essential Steps in Using GitHub

• Work should be organized on branches; milestones are saved through commits which document progress at stopping points.

• Once a task is complete (e.g., a PRD), users open a pull request for review. Feedback is incorporated before merging into the main branch for wider access.

Sponsorship Messages

Experimentation Platform - Chameleon

• Chameleon simplifies experimentation by allowing teams to create prototypes quickly without heavy developer involvement.

Mobile User Engagement - Amplitude

• Amplitude captures comprehensive mobile user interactions accurately without performance issues or data loss during session replays.

AI Product Management Course Recommendation

Enhancing Skills in AI Product Management

• A recommendation for an AI product management course led by OpenAI's McDad Jaffer offers practical experience and insights into AI product strategy.

Creating High-Quality Documents with Repositories

Importance of Shared Context

• A shared context repository is essential for writing high-quality documents, such as a 10x PRD or product strategy document.

• Effective planning using tools like "quad" complements the use of a shared context repository.

Utilizing Plan Mode

• Clearing the session context is crucial when switching tasks to ensure focus on the current task.

• Demonstrating plan mode, an example involving Google Stitch's recent release illustrates how to research effectively.

Prompting Techniques

• Basic prompting can yield unpredictable results, akin to working with an untrained junior employee who lacks guidance.

• A more effective approach involves asking Claude about its plan before proceeding, allowing for alignment on direction.

Transitioning to Plan Mode

• Activating plan mode (by pressing shift tab twice) helps eliminate biases towards immediate action and focuses on strategic planning.

• In plan mode, Claude researches both the release and relevant company contexts simultaneously for better strategy formulation.

Maximizing Repository Context

• The integration of repository context into documents enhances their quality and relevance during the planning phase.

• Understanding token usage in AI models highlights efficiency in processing information without unnecessary consumption.

Understanding Efficient Use of Claude

Importance of Organized Work

• The speaker emphasizes the significance of organizing work to optimize interactions with Claude, noting that disorganized queries can lead to inefficient token usage and hitting limits quickly.

Contextual Relevance in Queries

• Claude selectively accesses relevant folders based on the query, demonstrating its ability to focus on pertinent information rather than irrelevant data.

Keeping Context Updated

• Regular updates to the context repository are crucial; outdated context can lead to incorrect outputs from Claude, referred to as "context wrong."

Structuring Pull Requests (PR)

• For effective reviews, it's recommended to segment work into smaller PRs. This approach simplifies the review process for team members by reducing complexity.

High-Effort Settings for Optimal Results

• The speaker advocates for using high-effort settings when working with Claude, especially for tasks involving writing or reasoning, as it yields better results compared to lower effort settings.

Advanced Planning Techniques with Claude

Reading and Understanding Plans

• A critical step in utilizing Claude effectively is reading through generated plans thoroughly before execution, ensuring alignment and understanding of intended actions.

Expanding Research Scope

• When drafting a strategy document, expanding research parameters—such as including competitor analysis over recent months—can provide deeper insights and enhance strategic planning.

Parallelizing Tasks

• The speaker discusses creating phases within plans that allow parallelized research efforts. This method increases efficiency by enabling multiple tasks to be tackled simultaneously.

Checkpoints in Planning

• Establishing checkpoints during planning allows for review and adjustments based on preliminary findings before proceeding further in the project workflow.

Verification and Planning in Research

Importance of Verification

• Verification is crucial for ensuring that the work done by Claude is accurate and meets quality standards. The speaker emphasizes the need for a clear verification process.

• There is currently no established method for verifying how research is conducted, highlighting the importance of discussing verification methods with Claude, such as citing sources or providing URLs.

• Self-verification mechanisms are essential; for instance, using tools like Playwright MCP can help validate front-end features before completion.

Structuring Plans Effectively

• The speaker notes that while Claude proposes a structure for documents, it may not align with their preferences. Continuous feedback on document sections and narratives is necessary to achieve desired outcomes.

• Storing plan files within repositories enhances efficiency and provides historical context, allowing teams to reference previous plans when tackling similar tasks in the future.

Benefits of Documenting Plans

• Keeping plan files in repositories helps track ongoing work and serves as a resource for team members who may need to execute similar tasks later.

• OpenAI's recent article discusses making plan files first-class artifacts within shared repositories, emphasizing their significance beyond just final strategy documents.

Clarity in Agent Prompts

• It’s important to avoid summarizing plans since they should remain comprehensive for future sessions to build upon them effectively.

• The speaker suggests asking Claude to display agent prompts used during planning. This transparency ensures alignment on goals between Claude (as a junior employee) and its agents.

Contextual Understanding in Writing Tasks

• Knowing what context each agent has access to is vital; if agents lack proper information or resources, they may not produce effective outputs.

• Distributing writing tasks among multiple agents can be beneficial due to limitations in context windows and the complexity involved in long-form document creation.

Planning and Collaborating with Claude: A Comprehensive Guide

Structuring Documentation Effectively

• The speaker emphasizes the importance of being directive in documentation by outlining sections, context needed for each section, and assigning writing responsibilities to different agents.

• It is crucial for all agents to write their outputs to temporary files. This prevents crashes when multiple agents return work simultaneously, ensuring that no data is lost.

• Each agent's output should be stored in temporary files for the parent orchestrating agent to compile a final synthesis, enhancing organization and efficiency.

Engaging Claude as a Thinking Partner

• The speaker discusses inviting Claude to act as a thinking partner during the planning process, using tools like "ask user question" to refine ideas collaboratively.

• To manage multiple terminal windows effectively, the speaker names and color codes them. This practice helps maintain organization amidst numerous open terminals.

Enhancing Collaboration Through Questioning

• The speaker encourages Claude to push their thinking by asking clarifying questions about goals and potential document sections that may need inclusion.

• By allowing Claude to ask in-depth questions, the planning process becomes more robust, addressing gaps in reasoning and refining objectives.

Iterative Refinement of Plans

• The iterative approach involves spending time on initial planning rather than rushing into drafting documents. This ensures clarity before moving forward with writing.

• Continuous questioning from Claude helps identify missing elements or angles that need consideration, leading to a more comprehensive plan.

Utilizing Writing Guides for Consistency

• The speaker maintains various writing guides within their repository tailored for different document types. These guides help ensure consistency in voice when collaborating with Claude on writing tasks.

Planning and Execution of Skills in Complex Tasks

Importance of Explicit Planning

• Skills have a 70% auto-invoke rate; explicit commands should be specified in the plan to avoid uncertainty.

• Emphasizes the necessity of using specific skills, as dummy skills may not yield useful results until fully developed.

Iterative Improvement Process

• The skill can improve through iterations by analyzing outcomes and making adjustments based on feedback.

• A detailed plan is crucial for achieving desired outputs; under-planning leads to misalignment and suboptimal results.

Structuring Plans with Phases

• For long-running plans, outline phases to track progress effectively, allowing for seamless continuation after interruptions.

• Natural plan files are ephemeral; saving them ensures future reference even after completion.

When to Use Plans

• Not every task requires a formal plan; complex tasks benefit from structured planning while straightforward tasks may not need it.

• Planning is particularly useful when there’s ambiguity or complexity involved in the task at hand.

Tailoring Planning Efforts

• The level of planning should match the complexity and ambiguity of the task; more effort is needed for higher uncertainty.

• Lightweight alignment proposals can enhance clarity and ensure better outcomes even for simpler tasks.

Adopting a Beginner's Mindset

• Embracing a beginner's mindset helps alleviate feelings of overwhelm when learning new skills or concepts.

• Engaging with tools like Claude to ask questions fosters understanding and improvement in processes.

Learning and Leveraging Claude Code

Daily Learning with Claude

• The speaker emphasizes the continuous learning process while using Claude Code, indicating that they tweak their daily prompts to enhance understanding of its functionalities.

Researching Features and Influencers

• A specific prompt is shared where the speaker instructs Claude to research recent features from Enthropic and analyze top influencer posts to optimize their setup for better performance.

Importance of Understanding YAML Files

• The speaker discusses having a feature index in YAML format, explaining how they use Claude to learn about the benefits and structure of YAML files, which aids in understanding repository organization.

Emphasizing a Beginner's Mindset

• The importance of a beginner's mindset is highlighted; users should understand tools deeply rather than just copying commands. This knowledge helps in troubleshooting and improving setups.

Common Mistakes by Product Managers (PMs)

• A significant mistake identified is PMs giving up too early when learning new tools like Claude. Mastery requires time and consistent effort, as illustrated by the speaker’s extensive experience with Claude.

Choosing the Right AI Tools for PM Work

When to Use Different AI Tools

• The discussion covers various AI tools available for PM work, suggesting that coding agents are preferable for advanced tasks while chatbots can be used for quick answers without high context requirements.

Maximizing Time Efficiency

• For those with limited time, the recommendation is to automate tasks that free up more time for learning. This approach allows PMs to dedicate at least an hour daily to explore AI capabilities.

Curiosity vs. Trends in AI

Following Your Curiosity

• The speaker argues that pursuing personal interests within AI leads to better engagement and learning outcomes rather than feeling pressured to keep up with every trend or news update in the field.

Building Expertise and the Misleading Name of Claude Code

The Importance of Deep Learning

• Emphasizes the need for individuals to invest time in mastering a specific area rather than spreading themselves too thin across many topics.

• Highlights that Claude Code is not solely for coding; it serves a broader purpose, including documentation, analysis, and prototyping.

Understanding Claude Code's Functionality

• Notes that Claude Code is beneficial for both technical and non-technical users, making it an effective tool for knowledge work.

• Suggests that a more fitting name could have been "Co-work," indicating its collaborative nature.

Overcoming Fear of Technical Tools

• Advises project managers (PMs) to embrace being beginners with tools like terminals or IDEs, as familiarity grows with practice.

Integrating Software into Daily Workflows

• Recommends connecting all accessible MCP (Managed Cloud Platform) tools to Claude to enhance productivity within existing tech stacks.

Key Takeaways from the Discussion

• Reflecting on previous episodes, the conversation covered creating team operating systems and effective document writing processes using Claude Code.

• Encourages listeners to explore Hannah Stolberg’s Substack titled "In the Weeds" for further insights.

Engagement and Resources

• Invites listeners to support the podcast by following on various platforms and leaving reviews to help grow the show.

• Promotes access to a bundle of AI products available for free, aimed at assisting AI product managers.