39: Why AI Won’t Replace Humans in Healthcare (with Denise Hatzidakis)

AI Product Leadership Insights

The Challenge of Defining AI

• The speaker discusses the difficulty in establishing a clear definition of AI due to its rapid evolution and broad applications, emphasizing the need for concrete examples to distinguish genuine AI initiatives from mere buzzwords.

• It is noted that approaching AI as a purely tech-driven initiative can lead to failure; success requires integrating technology with strategic business objectives.

Introduction of the Host and Guest

• Paulie Allen introduces himself as the host, founder of AI Career Boost, and an experienced product manager who has worked on generative AI projects at Alexa.

• Denise Padzadakis is introduced as a transformational technology leader with extensive experience in healthcare and financial services, currently serving as CTO and CPO at Borie Health.

Denise's Background in Technology

• Denise shares her journey into technology, starting from childhood curiosity about electronics which led her to pursue computer science and engineering.

• She highlights her career trajectory through various roles in software development, particularly focusing on value-based healthcare over the last decade.

Innovation vs. Regulation in Healthcare

• Denise reflects on the challenges of driving innovation within heavily regulated sectors like healthcare, noting a reluctance to adopt solutions proven effective in other industries such as finance or retail.

• She emphasizes that while healthcare faces unique regulations, many underlying problems are similar across different sectors.

Role at Borie Healthcare

• In her dual role at Borie Healthcare, Denise discusses leading both product and engineering teams while focusing on AI-driven initiatives within their virtual orthopedic solutions.

• She mentions recent funding efforts aimed at scaling technologies effectively from startup phase (zero to 100 users) to larger operations (100 to 1,000 users).

Data Quality Challenges in Healthcare

• A significant challenge highlighted is ensuring data quality and accessibility; poor data quality can hinder successful implementation of AI technologies in healthcare settings.

Healthcare System Challenges and AI Integration

Issues in the US Healthcare System

• The US healthcare system is described as "arguably broken," with financial incentives that discourage data sharing, as providers profit from patients' illnesses.

• Achieving a comprehensive view of patient care is challenging due to fragmented systems outside closed environments like Mayo Clinic, making data normalization difficult.

• Privacy concerns are often cited as reasons for poor data sharing; many patients struggle to compile their health information across multiple providers, leading to inefficiencies.

• Patients frequently have to repeat their medical history at each new appointment, which can be frustrating and irrelevant to their current needs.

• Progress is being made with standards like FHIR (Fast Healthcare Interoperability Resources), but significant challenges remain until incentives align better with patient care.

Generative AI in Healthcare

• The discussion shifts towards generative AI technology's potential in healthcare; there’s uncertainty about whether the industry is ready for large-scale implementation or still exploring possibilities.

• Rapid advancements in technology create challenges for stakeholders trying to establish a stable foundation for development amidst constant changes and innovations.

• While generative AI is currently experiencing a hype cycle, its practical applications are emerging, particularly in assisting clinicians rather than replacing them entirely.

• Examples include using AI for ambient listening and scribing during consultations, helping streamline documentation processes for healthcare professionals.

• Insights gained from voice analysis can enhance customer interactions within healthcare settings by improving communication effectiveness.

Adoption of AI Technologies

• Many organizations are integrating AI into Electronic Health Records (EHR), aiming for an "AI-first" approach while emphasizing the importance of tangible outcomes over mere technological adoption.

• There’s skepticism regarding claims of new AI capabilities; many believe that previous methods were already forms of AI but lacked modern tools and frameworks.

• The focus should remain on achieving value-based outcomes rather than getting caught up in the latest trends or technologies without clear benefits.

• Companies risk losing valuable existing practices by overly focusing on adopting new technologies without considering their proven effectiveness.

• A cautionary example highlights how companies may prioritize proving what cannot be done with AI before hiring human talent, potentially stifling innovation.

The Evolving Role of AI in Healthcare and Product Management

The Current Landscape of AI in Healthcare

• There is a growing skepticism and desire for AI integration within healthcare, with boards increasingly asking about their organization's AI initiatives.

• Initially, board members focused on compliance and risk aversion regarding AI; however, the narrative has shifted to a need for more proactive engagement to avoid disruption.

• A significant mistake organizations make is treating AI as merely a tech-driven initiative rather than understanding its broader business implications.

Understanding the Value of AI

• Successful implementation of AI requires starting with questions about who it helps, what value it provides, and what problems it solves—essentially focusing on ROI.

• It's crucial to approach AI from a business perspective rather than solely a technological one to ensure effective application.

Personal Journey into AI

• The speaker's interest in AI was sparked after years in startups; they adopted a three-pronged approach to ramp up their knowledge: taking classes, applying concepts practically, and learning the language of technology.

• Enrolling in university-sponsored courses helped them grasp foundational principles but highlighted the fast-paced evolution of technology that academia struggles to keep up with.

Practical Learning Experiences

• Engaging directly with technology through hands-on projects (like building Kaggle notebooks using Python) proved essential for deeper understanding.

• Active learning is emphasized as critical for engineers transitioning into thinking differently about problem-solving beyond traditional methods.

Coding Skills for Product Managers

• While not all product managers need coding skills, having an understanding can enhance collaboration with engineering teams and improve communication regarding technical challenges.

• Governance and compliance around AI differ significantly from traditional engineering practices; thus, product managers must adapt their approaches accordingly.

Challenges in Governance and Compliance

• Managing model drift and data governance presents unique challenges in the context of AI compared to legacy systems where data flow is predictable.

• Understanding these differences is vital for ensuring effective governance while leveraging the capabilities of advanced technologies like AI.

Data Governance and Trust in Generative AI

Challenges of Data Lineage in Generative AI

• The complexity of data lineage increases significantly with generative AI, raising concerns about where data is being utilized.

• Organizations like OpenAI claim they won't use enterprise data for training, but trust issues arise regarding these assurances.

Validation and Quality Assurance

• The introduction of generative AI necessitates an additional layer of validation, quality assurance (QA), and governance beyond existing frameworks.

Concerns Over Data Usage by Major Companies

• Meta announced it would utilize public Facebook profiles for training unless users opted out, highlighting the irreversible nature of data once included in a trained model.

• There is a general lack of accountability from organizations regarding data mishandling, which can lead to significant risks.

Security Compliance and Governance Issues

• A major incident may be required to prompt serious discussions around security compliance and governance in AI technologies.

• The DeepSeek leak exemplifies negligence towards personal information security, yet it received minimal media attention despite its severity.

Thoughtful Implementation of New Technologies

• When adopting new tools like Cursor for developers, it's crucial to approach implementation thoughtfully while ensuring privacy considerations are addressed.

Cultural Shifts in Adopting AI Technologies

Navigating Low Information Environments

• Leaders must operate within environments where complete information isn't available; this requires cultural shifts within organizations.

Advice for Internal Adoption

• Emphasize thoughtful experimentation with new technologies rather than overwhelming changes; small trials can yield valuable insights without risking operational integrity.

Balancing Automation with Agility

• Organizations should assess their processes' robustness before automating tasks; automation can reduce agility if processes change frequently.

Importance of Organizational Change Management

• Implementing AI will likely alter existing processes; thus, organizational change management becomes critical to ensure smooth transitions.

AI Urban Legends and Hands-On Experience

Addressing AI Misconceptions

• The speaker discusses the impact of urban legends about AI, emphasizing the need to mitigate fears to make people comfortable with technology.

• A reference is made to "The Terminator," highlighting how popular media has contributed negatively to public perception of AI.

Value of Practical Engagement

• The importance of hands-on experience in understanding AI is stressed; product leaders should engage directly with technology rather than just theoretical learning.

• The speaker shares personal insights from coding experiences, noting that practical application helps clarify how different components fit together in AI systems.

Encouragement for Exploration

• Advice for software leaders includes experimenting with AI tools without fear, as they are not likely to cause harm if used responsibly.

• Caution is advised regarding data privacy; users should protect sensitive information while exploring various AI applications.

Understanding Different Tools

• The discussion highlights that there isn't a one-size-fits-all solution for using different AI tools; each has unique strengths (e.g., ChatGPT excels at language generation while Claude aids in Python coding).

Market Dynamics and Adoption Challenges

• Observations on the subscription model for multiple AI services likened to Netflix, indicating a shift in how companies approach vendor relationships.

• Emphasis on the rapid centralization around major players in the market and the challenges associated with shifting away from established technologies after significant adoption efforts.

Networking and Collaboration Opportunities

• The speaker invites connections via LinkedIn for discussions about innovative healthcare solutions, stressing collaboration as essential for progress in the field.