# 173 Hollywood-Grade AI Dubbing with Deepdub CEO Ofir Krakowski

Introduction to Deepdub and AI Dubbing

Building a Product, Not Just Technology

• Ofir Krakowski emphasizes the importance of creating a product that serves a clear purpose rather than focusing solely on technology. Deepdub aims to provide an accessible dubbing solution.

Overview of Deepdub

• The podcast introduces Ofir Krakowski, co-founder and CEO of Deepdub, which has gained recognition as a promising startup in the AI dubbing space.

• Ofir describes Deepdub's mission to bridge language gaps by enabling content translation into multiple languages simultaneously in an affordable manner.

Deepdub's Vision and Approach

Aiming for High Standards

• The company started with the goal of achieving theatrical-grade dubbing, believing that if they could successfully dub high-quality films like Disney movies, they could cater to broader audiences.

Personal Journey and Background

• Ofir shares his background in machine learning from his time in the Israeli Air Force, where he learned about deep learning and generative AI technologies.

• His journey into AI dubbing began after exploring various ideas with his brother Nir on how generative AI can impact global content accessibility.

Challenges in Content Accessibility

Language Barriers

• Ofir highlights that most audiovisual content is created for specific cultures and languages, making it inaccessible for non-English speakers or those who cannot read subtitles.

Importance of Dubbing for Children

• He notes that children often struggle with reading subtitles until they are older; thus, dubbed content is crucial for their education and entertainment.

Target Audience and Market Insights

Understanding User Needs

• The discussion reveals that many people worldwide lack access to content due to language barriers. This issue affects billions who do not speak English or cannot read well enough to follow subtitles.

Current Client Segments

AI Dubbing and Localization Insights

Overview of AI Dubbing Projects

• The discussion begins with the creation of new voices for dubbing content, highlighting projects that cater to both theatrical releases and smaller productions unable to afford traditional dubbing services.

Accessibility of Foreign Content

• A significant amount of foreign content previously inaccessible to U.S. audiences is now available, particularly from Latin America and Europe, thanks to advancements in technology.

Cultural Exposure Through Media

• American audiences are increasingly exposed to diverse cultures through original foreign series like "The Bridge," which showcases Danish and Swedish cultures, enhancing cultural understanding.

Challenges in Localization

• While some viewers prefer dubbed content for easier comprehension, there are challenges in making foreign media digestible for those unfamiliar with subtitles.

Deepdub's Approach to Client Support

• Deepdub integrates into the post-production process, offering a managed service that combines technology with human oversight for localization needs.

Human Oversight in AI Translation

• Different workflows exist for various clients; high-end theatrical customers require human curation alongside machine-generated translations due to current limitations in AI accuracy (70%-90%).

Achieving High Accuracy in Dubbing

• To reach 100% accuracy necessary for high-quality drama content, human intervention is essential as machines alone cannot achieve this level of precision.

Flexibility in Voice Generation Technology

• Deepdub employs various voice models allowing flexibility in generating voices from text or existing audio while capturing emotional nuances effectively.

Creative Elements Beyond Machine Capability

• The last 10% of translation often involves creative liberties that machines struggle with, such as humor or culturally specific references that require human insight.

Ranking Challenges in AI Dubbing

AI Dubbing Technology and Its Evolution

The State of Voice Generation Technology

• Three years ago, voice generation technology was predominantly owned by four to five companies, limiting its availability. Currently, it is more accessible.

• An internal research and development team focuses on advancing this technology to enhance capabilities.

Challenges in Translation

• Translation is identified as the second most challenging aspect of dubbing. The company leverages existing technologies from major firms like Google and Amazon for translation.

• Domain-specific technologies are integrated to improve translation accuracy significantly.

Development of a User-Friendly Platform

• A platform akin to Adobe Premiere for dubbing has been developed, allowing users to easily transition from 80% accuracy to 100%.

• The platform includes automatic mixing features that enable quick adjustments and interactions with voice elements.

Service Model and Accessibility

• While studios prefer end products over using the platform directly, an end-to-end solution is provided where clients can submit content for dubbing.

• The web-based platform allows remote work without needing a physical studio, catering to post-COVID preferences.

Importance of TPN Certification

• The Trusted Partner Network (TPN) certification qualifies the company as a secure vendor capable of handling sensitive pre-release materials.

• This certification is crucial for accessing content before its public release, enabling early localization efforts.

Enhancing Test Screenings with Technology

• Current test screenings are primarily conducted in English; however, understanding audience reactions in different cultures is essential.

• Early-stage adjustments facilitated by technology can save costs compared to late-stage changes.

Security Assessments and Global Reach

• TPN certification involves thorough security assessments of both technology and facilities; few companies have achieved this globally.

Engagement Trends in YouTube Content

• There’s a discussion about engagement levels between human-dubbed versus AI-dubbed content on platforms like YouTube.

Focus on Influencer Content Creation

Human vs. AI Dubbing: Quality and Accessibility

The Role of Quality in Dubbing

• Human dubbing quality can vary significantly; poor dubbing or translation can detract from the viewing experience.

• AI dubbing offers flexibility, allowing for instant text changes without needing to re-record lines in a studio.

Platform Launch and Target Audience

• A new platform is set to launch soon, aimed at influencers on YouTube for affordable content dubbing.

• The platform will offer a freemium model, allowing users to test its features before committing to paid tiers.

Features of the Dubbing Platform

• Two user views are available: a simple text-only interface and a professional view with advanced audio editing capabilities.

• Users can localize existing content or create new content directly from text, enhancing accessibility.

Market Potential and Customer Segmentation

• The platform targets both enterprise customers (e.g., banks with corporate videos) and individual creators who need localization services.

• Many potential users currently find traditional dubbing processes too complex or expensive; this platform aims to simplify that.

Democratizing Content Creation

• The platform could enable creators from non-English speaking regions to reach wider audiences by making their educational content accessible globally.

• Successful creators in their native languages often struggle with international visibility; effective dubbing could change that dynamic.

The Direct Impact of Localization on Growth

ROI from Localization Efforts

• There is a clear link between language accessibility and revenue growth for creators; successful examples like Mr. Beast illustrate this point.

Technology Integration: Buy vs. Build

Strategic Decisions in Technology Development

• The company opts to purchase basic transcription and translation technologies via API rather than developing them in-house due to data ownership concerns.

The Impact of Generative AI on Traditional Industries

The Shift in AI Awareness

• The launch of ChatGPT marked a significant shift in AI, leading to increased awareness and interest in generative AI technologies.

• Observers are overwhelmed by the rapid developments in generative AI, raising questions about how to filter relevant information for product roadmaps.

Understanding Generative AI's Role

• Generative AI has made it easier for users to understand its applications, particularly in enhancing efficiency across various processes.

• Initial skepticism from traditional industries regarding generative AI required extensive proof-of-concept demonstrations to showcase its capabilities.

Building Products vs. Technology

• Emphasis is placed on creating user-friendly products rather than merely developing advanced technology; Deepdub focuses on practical applications.

• Successful companies will prioritize building understandable and efficient products that address real user needs, as demonstrated by ChatGPT's approach.

Challenges with Dubbing and Voice Creation

• There is a distinction between voice creation and dubbing; effective dubbing requires more than just generating voices.

• Users desire complete solutions that integrate translation, adaptation, voice creation, and video mixing into one seamless process.

Industry Concerns and Collaboration

• Traditional industries express concerns over the implications of AI dubbing; there is a need for collaboration with established voice actors.

• Engaging with organizations like BFFS (the union of voice actors) aims to merge human creativity with technological advancements without displacing jobs.

Addressing Regional Limitations

• Established voices are crucial in certain regions where specific actors have become synonymous with characters; technology should complement rather than replace them.

Dubbing Talent and Market Dynamics

Current Challenges in Hiring Dubbing Talent

• The speaker discusses the competitive market for hiring dubbing directors, language adapters, and machine learning roles, highlighting the difficulty in finding talent in certain regions.

• Despite challenges, the speaker notes that there is a demand for work on their platform, which allows flexibility for workers to earn additional income outside regular hours.

Flexibility and Remote Work Opportunities

• The platform enables voice actors to work remotely from various locations, including vacation homes, thus expanding access to talent who may not be near studios.

• The ease of remote work is emphasized as a significant advantage of the platform, allowing individuals to manage their time effectively while contributing to dubbing projects.

Introduction of Deepdub GO

• A new product named "Deepdub GO" is introduced; it aims to facilitate localization efforts with a focus on global outreach.

• The speaker anticipates 2023 will be pivotal for Generative AI innovations and plans to enhance the platform's capabilities significantly.

Expansion Plans and Language Availability

• By the end of 2023, the goal is to support 60 languages on the platform, catering to diverse audiences worldwide.

• India is highlighted as a major market opportunity due to its multitude of official languages; most content there is already dubbed into one or more languages.

Future Prospects in AI Dubbing

• The potential for growth in AI dubbing startups within India indicates a burgeoning market ripe for exploration.