Taking AI animation to the NEXT level

Name: Taking AI animation to the NEXT level
Uploaded: 2023-04-14T21:00:09.000Z
Duration: 48 min 40 s

Creating a Fully Animated AI Character

In this tutorial, the speaker shows how to create a fully animated AI character using a realistic data set. The method does not require acting out scenes or looking like the actor/actress.

Preparing Your Training Images

Collect various images of yourself, including head and body shots from different angles, full bodies, close-ups, and far away shots.

Ensure that your hairstyle, background, and outfits change in each image to avoid confusion by the AI.

Use a website called Burme to crop your images down to 512x512 resolution.

Training Your Data Set

Use Colab with Koya Laura Dream Booth to train your data set.

Mount Google Drive and download all dependencies into your file structure.

Using Character Creator

Use Character Creator to create a puppet without needing any 3D knowledge.

Switch out data sets from one character to another easily and efficiently.

Conclusion

Creating an animated AI character is possible using realistic data sets without requiring any prior acting experience or looking like the actor/actress. By following these steps for preparing training images and training your data set using Colab with Koya Laura Dream Booth, you can use Character Creator to create puppets easily and efficiently.

Setting up the Model

In this section, the speaker explains how to set up the model and download necessary files.

Ignoring Unused Files

The model blanks will not be used.

Ignore 2.2 as it will not be used.

Downloading Stable Diffusion 1.5 V

Load in stable diffusion 1.5 V which is already there.

This will download all files into our file structure.

Hit "run" on the cell.

Locating Train Data Directory

Creates a file path to where train_data file is located.

All input data set images go into that folder automatically.

Hit "run" on the cell.

Unzipping Data Set

Use ZIP file URL to grab zip file from Google Drive.

Extract all files from zip file into new folder.

Preparing Data for Annotation

In this section, the speaker explains how to prepare data for annotation.

Using Blip Captioning for Data Annotation

Use blip captioning to tag images with a description.

This is used for realistic imagery.

Leave settings as default and run cell.

Generating Image Descriptions

In this section, the speaker explains how image descriptions are generated.

Reading Input Images

Reads input images that were put into Google Collab.

Describing Images

Describes what it sees in each image using blip captioning.

Everything outside of description is what it trains upon.

Viewing Generated Caption Files

Generated caption files can be found in train_data directory under Laura folder.

Each image has its own corresponding caption file with a detailed description of what's in the image.

Model Configuration

In this section, the speaker explains how to configure the model for training.

Configuring the Project Name and Pre-Trained Model

To configure the project name, give it a memorable name.

For pre-trained model name, copy and paste the path of your stable diffusion trained model.

Ensure that you check "Output to Drive" to save your files on Google Drive.

Saving Finalized Models

In this section, the speaker explains where finalized models will be saved.

The finalized models will be saved in a folder called "Laura" in the output file on your Google Drive.

Training Configuration

In this section, the speaker explains how to configure training settings.

Setting Train Repeats and Instance Token

Set train repeats to 10.

Keep instance token as mksks or any other random name that Stable Diffusion associates with your model.

Setting Resolution and Styles

Set resolution at 512 by 512 or 768 by 768 if using those input images.

Do not train a style; instead, train a person or woman.

Optimization Configuration

In this section, the speaker explains how to optimize configuration settings for better results.

Experimenting with Settings

Experiment with different settings to see what works best for your trading sets and resolutions.

Setting Convolution Dim and Alpha

Set convolution dim at eight.

Set convolution alpha at one.

Setting Network Dim and Alpha

Set network dim at 16.

Change network alpha to eight.

Learning Rate Configuration

In this section, the speaker explains how to configure learning rate settings.

Setting Learning Rates

Change the first learning rate to 5E-4.

Change the text encoder learning rate to 1E-4.

Change the learning rate scheduler to cosine with restarts.

Setting Warm-Up Steps

Set warm-up steps at 0.0305.

Training Configuration Continued

In this section, the speaker continues explaining how to configure training settings.

Enabling Sample Prompt and Creating Epochs

Enable sample prompt.

Create ten epochs that will save a file at every learning stage for testing purposes.

Training the Model

In this section, the speaker explains how to train a model using Google Colab.

RAM and Precision

The speaker recommends sticking to 6 or higher if you have RAM issues.

If you don't have any RAM issues, you can train on anything as low as 1.

The speaker suggests leaving mixed and save precision at fp16.

Saving the Model

The speaker saves every epoch for a total of 10 epochs.

The model is saved as a save tenses model with all default settings.

Starting the Training

To start training, run the final cell with all default settings.

This process may take around 30 to 40 minutes depending on your batch size.

Using Your Trained Model

In this section, the speaker explains how to use your trained model after it has been saved.

Finding Your Files

After training is complete, files will be automatically saved into your Google Drive.

Look for a Laura file in your drive's output folder.

There should be ten files if you followed along with this demonstration.

Additional Options

The speaker mentions that there is an option to use automatic 111 locally on your computer or through Koya Laura Notebook's new automatic 111 notebook with control net one and two.

Creating Your Puppet

In this section, the speaker explains how to create a puppet using iclone and motion live.

Loading Laura Files

Load your Laura files into iclone by selecting "Laura" under the sun icon panel.

Adjust weights of each file by adjusting figures.

Sculpting Your Character

Drag and drop a headshot into the plugin to generate a 3D model of your face.

Sculpt further if desired.

Adding Hair

Drag and drop hair from a library onto your character.

Exporting to iClone

Export your puppet to iclone for animation.

Use motion live app on phone to record facial movements and apply them in real-time.

Add lighting situation and physics on hair before rendering.

Importance of Prompt and Negative Prompt

In this section, the speaker emphasizes the importance of keeping the prompt brief and not including too many details that are not in the image. The negative prompt is also discussed.

Key Points:

The prompt should be kept as brief as possible.

Do not include too many details that are not in the image.

The negative prompt is just as important as the regular prompt.

Sampling Method, Denoising Strength, and CFG Scale

This section covers various settings related to sampling method, denoising strength, and CFG scale.

Key Points:

The speaker has set the sampling method to DPN plus plus sde but other methods can be used.

Good results have been achieved with Euler a and k m Carrabba's sampling steps.

For a width and height of 768 by 768, relatively low sampling steps are being used.

Denoising strength is set at 11 for this workflow.

CFG scale is down six to pull from the image as much as possible without compromising it.

Control Net Settings

This section discusses control net settings related to head and model enabling.

Key Points:

Head and model enabling are both enabled on control net with default settings.

A secondary model (canny) has also been enabled with default settings.

Rendering Frames

This section covers rendering frames for frame-by-frame animation using batch input/output settings.

Key Points:

Batch input/output settings are used for rendering frames for frame-by-frame animation.

Input location should be where 3D files are located and output location should be where the frames will be rendered.

Hit generate to complete the animation.

Precise Mouth Lip Syncing

This section discusses how precise mouth lip syncing can be achieved using this workflow.

Key Points:

The speaker's workflow allows for precise mouth lip syncing.

Many AI generation techniques result in blurry mouths, but this workflow avoids that issue.

The model was created on the speaker to ensure it works well.

Switching Checkpoint File

This section covers switching checkpoint files and creating a semi-anime style version of the image.

Key Points:

The checkpoint file can be switched to create different render types with a click of a button.

A semi-anime style version is created by switching the checkpoint model over without changing any settings.

It took about two minutes to render those anime frames.

After Effects and DaVinci Resolve

This section discusses using After Effects or DaVinci Resolve for dirt removal and D flicker times two for better results.

Key Points:

After Effects or DaVinci Resolve can be used for dirt removal and D flicker times two for better results.

Using these tools will give really nice move results to your animation.

Social Media and Prompt News Website

In this final section, the speaker encourages viewers to tag them on social media with their creations. They also mention their website, which has step-by-step tutorials and a weekly newsletter about AI tech.

Key Points:

Viewers are encouraged to tag the speaker on social media with their creations.

The speaker's website has step-by-step tutorials and a weekly newsletter about AI tech.

The speaker's social media handles are provided.

Conclusion of the Video

In this section, the speaker concludes the video and expresses gratitude.

The speaker thanks the audience for watching and listening to the video.