My AI Filmmaking Workflow: Kling, Veo & Nano Banana

How I Made My Movie Discarded Companion

Introduction to the Tutorial

• The tutorial is set on a rainy day, creating an ideal atmosphere for discussing the filmmaking process behind "Discarded Companion."

• The creator plans to walk through their video editing timeline scene by scene, highlighting both successful and unsuccessful elements of the project.

• The aim is to assist viewers in their AI filmmaking journey by sharing insights from personal experiences.

Character Development

• A character sheet was created for the Android using references from previous work, including screenshots that captured preferred aspects of the character's design.

• The creator utilized tools like Canva or Photoshop to arrange various character views (back, front, close-ups), ensuring clarity in visual representation.

• By leveraging Nano Banana's large context window, multiple angles of the same character were generated to maintain consistency across scenes.

Image Generation Process

• A prompt was used in Nano Banana to create a human version of the Android character wearing a comfy sweater, expanding on character diversity.

• Familiarity with Google Flow is emphasized as essential for effectively generating and editing images within this platform.

Scene Creation Techniques

• An initial shot inside a mining shaft was conceptualized where the Android appears curled up in a fetal position amidst trash—a unique setting choice.

• Various iterations were produced based on prompts; some results deviated from expectations but ultimately led to satisfactory visuals.

Editing and Sound Design

• Specific adjustments were made to ensure continuity between shots; for instance, changing facial expressions or positions based on feedback from earlier images.

• Environmental sounds and foley effects were prioritized over music during voiceover prompts to enhance realism without distracting soundscapes.

Sequence Construction

• A 23-second sequence was crafted by linking frames together seamlessly; each clip’s end frame became the start frame for subsequent clips.

• Attention was drawn to minor glitches caused by rain inconsistencies during editing; trimming off specific frames improved overall flow.

Character Design and Animation Techniques

Close-Up Shots and Character Reference

• The speaker discusses capturing a close-up shot of a character walking down a trash heap, emphasizing the need for specificity in prompts to avoid unwanted angles.

• A character sheet is referenced to guide the animation, with instructions to have the character gaze downward instead of directly at the camera for a more natural look.

Motion and Fluidity in Animation

• The importance of head motion is highlighted; adjustments are made to ensure upward movement rather than downward, enhancing fluidity in animation.

• The technique of cutting on motion is introduced as a method to create seamless transitions between clips, improving overall coherence.

Scene Adjustments and Happy Accidents

• The speaker describes removing the character from an initial setting and using different garbage bags as props, leading to unexpected creative outcomes.

• An adjustment was made where the character interacts with new elements (garbage bags), showcasing how prompt revisions can lead to better results.

Character Interaction with Environment

• Attempts are made to animate the character sorting through dolls and games, but some shots do not meet expectations due to inconsistencies in facial representation.

• A new scene is created where the character searches through discarded toys, demonstrating flexibility in adapting scenes based on available assets.

Emotional Expression Through Actions

• Frustration is depicted through actions like tossing aside items while searching methodically among piles of garbage.

• A sequence shows emotional depth as the character interacts with a stuffed animal, hinting at her past identity as an android.

Designing Unique Characters

• The creation process for a unique stuffed animal design begins; initial ideas evolve into something that resonates more closely with the desired aesthetic.

• Final designs reflect personal touches that align with both narrative needs and visual appeal, showcasing creativity in character development.

Creating an End Frame in Cling

Initial Setup and Concept Development

• The speaker discusses the necessity of using version 1.6 of Cling to create an end frame without a start frame, emphasizing the importance of framing in visual storytelling.

• A medium close-up shot is described where the android character looks to the right while sorting through trash, establishing a narrative context for the scene.

• The adjustment made to indicate that the character is looking away enhances realism, suggesting she has just noticed something significant.

Visual Composition and Background Adjustments

• The speaker requests a point-of-view shot from the character's perspective holding a doll, utilizing reference images for accuracy.

• Issues arise with background consistency; attempts to remove it lead to unsatisfactory results, prompting specific instructions for rearranging elements in the scene.

• Further adjustments include blurring garbage in the background and ensuring only relevant objects remain clear, enhancing focus on key elements.

Effects and Character Integration

• To maintain continuity between scenes with different weather conditions (rainy vs. dry), effects like grain and rain are added post-production.

• The absence of a character reference sheet initially hampers progress; however, once acquired, it allows for better integration of video clips featuring the character interacting with her environment.

Color Matching and Narrative Depth

• A magnetic mask technique is employed for background removal while matching colors between clips ensures visual coherence across scenes.

• The narrative deepens as viewers understand that the doll evokes memories from a past life for the character, adding emotional weight to her journey.

Scene Transitioning and Size Consistency

• Transitioning shots show changes in wardrobe as part of character development; this includes her decision-making process regarding clothing choices influenced by environmental cues.

• References are used strategically to illustrate size relationships between characters and props (e.g., stuffed animal), addressing inconsistencies observed during production.

• As scenes shift from urban settings to nature-infused environments, maintaining accurate scale becomes crucial; multiple angles help establish context within varying backgrounds.

Creating a Realistic Character Interaction

Utilizing Visual References for Character Placement

• The speaker discusses using a picture to establish the relative size of a stuffed animal compared to the character, aiding in visual consistency within the neighborhood setting.

• A bay window is highlighted as an interesting element in the shot; drapes are closed initially to create anticipation for later scenes where they will be opened.

• The prompt involves an android stepping up to ring a doorbell, emphasizing the importance of staging and removing characters from shots for clarity.

Adjusting Character Design and Dialogue

• The speaker experiments with different character designs, ultimately deciding to remove a "French tuck" from clothing for a more natural appearance.

• After establishing character placement on the porch, dialogue sequences are created, requiring careful alignment between voice performance and character positioning.

Recording Dialogue with Realism

• To enhance realism, the speaker performs dialogue while ensuring their orientation matches that of the animated character being referenced.

• Different angles and profiles are utilized during recording sessions to maintain consistent directionality in performances.

Balancing AI Voice Characteristics

• A challenge arises in delivering lines that sound like an android without mimicking poor AI voice quality; this requires navigating between human-like delivery and robotic characteristics.

• The creation of a unique voice profile using 11 Labs is discussed, showcasing how specific prompts can shape vocal characteristics effectively.

Synchronizing Audio with Visual Elements

• Once voice clips are generated, they are organized by original names for clarity when syncing audio with visuals in editing software.

• The integration process involves lowering volume levels of recorded performances while layering them over generated voices from 11 Labs for seamless interaction.

Finalizing Character Interactions

• The use of avatar libraries allows further refinement of character interactions; specific instructions ensure only designated characters speak during scenes.

• Multiple methods for dialogue generation are explored: combining 11 Labs voices with personal performance or utilizing Avatar 2.0 features for enhanced realism.

Creating Voices and Visuals in Cling 2.6

Utilizing AI for Voice Creation

• In Cling 2.6, creating a voice requires existing audio; however, 11 Labs allows users to create desired voices and export short dialogues for use in Cling.

Crafting Emotional Narratives

• The creator considered using a montage of Instagram photos to depict the character's emotional connection but opted for a more immersive AI-driven sequence instead.

Image to Video Techniques

• A sequence was designed where the character morphs into the original human sewing the doll, emphasizing emotional continuity through visual storytelling.

Scene Adjustments and Enhancements

• Initial plans included showing garbage piles in the background, but these were removed to maintain focus on the character's journey without distractions.

Technical Challenges and Solutions

• The creator faced challenges with scene transitions and adjusted prompts to achieve desired visuals, demonstrating adaptability in creative processes.

Reflections on Filmmaking Process

Learning from Experience

• The creator acknowledges that filmmaking is complex and involves learning from each project, highlighting personal growth through challenges faced during production.

Importance of Creativity Over Automation

• Emphasizes that while AI can assist with technical aspects of filmmaking, true creativity and storytelling remain uniquely human endeavors that require effort and passion.

Final Thoughts on Satisfaction in Creation

• Completion of a film brings temporary satisfaction; however, the joy lies in the creative process itself rather than just achieving an end product.