AI Pixel Art - Easy Animations for Games
TLDRIn this tutorial, the creator shares a detailed workflow for generating pixel art sprite sheets for games using AI. They guide through cloning a repository, installing necessary software, and setting up Comfy UI with custom nodes. The process involves using Stable Diffusion for generating images, converting them to 3D models with Tripo Sr, refining in Blender, rigging in Mixo, and finally, creating animations. The creator emphasizes keeping character designs simple for beginners and demonstrates how to export and compress animations into a sprite sheet, concluding with a preview of the sprite sheet in a game engine.
Takeaways
- 🎨 The video discusses a workflow for creating pixel art sprite sheets for games with a consistent style across frames.
- 🔧 The process involves using a combination of tools and scripts, including a utility script from a cloned repository.
- 💻 Comfy UI and custom nodes are required for the workflow, with tutorials available online for installation.
- 🖼️ The workflow starts with generating a T-pose image using a specific model and control net, with the image named accordingly.
- 🔄 A trip to 'tripo Sr' is necessary to convert the image into a 3D model, despite some loss of detail.
- 🎭 Blender is used to refine the 3D model, add color, and adjust material properties like roughness.
- 🔧 An .fbx file is exported from Blender with specific settings to ensure colors are retained.
- 🤖 Mixamo is used for auto-rigging and to apply animations to the character model.
- 🎞️ Blender is used again to set up the camera, lights, and render out the animation frames with a white background for better extraction.
- 🖌️ The script mentions the importance of selecting key frames and rendering them out for the sprite sheet.
- 📦 The final step involves using Comfy UI to pixelate the images and a script to compress them into a sprite sheet, which can then be imported into a game engine.
Q & A
What is the main focus of the video?
-The main focus of the video is to demonstrate a workflow for generating pixel art style sprite sheets for games with consistency between frames using stable diffusion.
What is the first step mentioned in the workflow?
-The first step is to clone a repository that contains utility scripts and UI workflows or to download the zip file and unpack it.
Why is Comfy UI needed in this workflow?
-Comfy UI is needed to run the custom nodes and workflows designed for generating the pixel art sprite sheets.
What is the purpose of the gen T pose workflow in the video?
-The gen T pose workflow is used to start the process and generate the initial character pose that will be used for creating the sprite sheets.
What is the role of the Juggernaut XL model in the workflow?
-The Juggernaut XL model is used as an sdxl model to generate good outputs for the initial character pose.
Why is it recommended to avoid complex elements like weapons in the initial stages?
-Complex elements like weapons can cause issues in the workflow and may not work well with the process, so it's advised to keep the character simple and humanoid.
What is the purpose of using Tripo Sr in the workflow?
-Tripo Sr is used to convert the generated image into a 3D model, which is then refined for better results in the animation stage.
Why is Blender used in the workflow?
-Blender is used to refine the 3D model, apply color and texture, adjust materials, and export the model in a format suitable for further processing.
What is the significance of the Mixo platform in this workflow?
-Mixo is used to upload and auto-rig the 3D model, and to provide animations that can be applied to the character for generating sprite sheets.
How does the video suggest handling animations in Blender?
-The video suggests importing the FBX file, applying color and material settings, setting up camera and lighting, and rendering out the animation frames.
What is the final step in the workflow described in the video?
-The final step is to use a script to compress the rendered images into a sprite sheet, which can then be used in games.
Outlines
🎨 Generating Pixel Art Sprite Sheets
The speaker discusses their workflow for creating pixel art style sprite sheets with consistency between frames using stable diffusion. They mention a slight jitter but overall satisfaction with the results. The process involves cloning a repository for utility scripts and UI workflows, installing Comfy UI, and setting up custom nodes. The tutorial starts with generating a T-pose using a specific model and control net, emphasizing the need for correct configuration and mentioning issues with details like weapons. The speaker advises beginners to avoid complex poses and to expect a streamlined workflow despite some manual steps.
🖥️ Refining 3D Models in Blender
After obtaining a 3D model using Tripo Sr, the speaker uses Blender to refine the model by adding color and adjusting material properties like roughness for a matte finish. They save the workspace for future use and export the model as an FBX file with color information. The tutorial then moves to Mixo for auto-rigging and selecting animations, emphasizing the importance of checking certain settings before downloading the FBX file to ensure smooth workflow in subsequent steps.
🎥 Setting Up Animations in Blender
The speaker returns to Blender to import the FBX file and set up the渲染环境, including camera and lighting, to ensure consistent lighting for rendering. They discuss the importance of setting the correct start and end frames for animations to loop properly and render the animation, adjusting the background color for better extraction in later stages. The process involves selecting key frames for the sprite sheet and renaming the frames for organization before moving to the next step.
🌐 Pixelating and Organizing Animations
The tutorial continues with importing the second animation in Blender, adjusting its length, and rendering it. The speaker selects key frames that capture the essence of the animation and renames them for clarity. They then move to Comfy UI to use a pixelation workflow, emphasizing the need for the correct checkpoint and pixel art model. The process involves running the pixelation on all images, resulting in a preview of all animations, which will be cleaned up in the next step.
🔍 Post-Processing and Creating the Sprite Sheet
The final steps involve using a command line to remove backgrounds from the pixelated images with the rembg tool, followed by manual touch-ups if necessary. The speaker then runs a compression script to create a sprite sheet from the processed images. They conclude by showing the sprite sheet in an image viewer and in a game, noting the need for further adjustments but overall satisfaction with the results. The tutorial ends with a call to action for likes and subscriptions.
Mindmap
Keywords
Pixel Art
Sprite Sheets
Stable Diffusion
Comfy UI
Juggernaut XL
Control Net
Tripo Sr
Blender
Mixo
Rembg
Sprite Sheet Compression
Highlights
Introduction to creating pixel art style Sprite sheets with consistency between frames.
Workflow involves using stable diffusion for generating pixel art.
Cloning a repository for utility scripts and UI workflows.
Instructions on downloading and unpacking the repository.
Requirement of installing Comfy UI and custom nodes.
Using a gen T pose as a starting point for the process.
Loading a checkpoint and control net for generating the initial pose.
Importance of using the correct image and prompt for the initial pose.
Recommendation to avoid complex poses like holding weapons when starting out.
Configuring a workspace and organizing generated images.
Using Tripo Sr to convert images to 3D models.
Importing the 3D model into Blender for refining and adding color.
Adjusting roughness in Blender to reduce character shininess.
Exporting the model as an FBX file from Blender.
Uploading the FBX file to Mixamo for auto-rigging and animation.
Selecting and downloading desired animations from Mixamo.
Setting up camera and lights in Blender for rendering animations.
Rendering animations at specific frames to create key poses.
Renaming and organizing rendered images for Sprite sheet creation.
Using Comfy UI's pixelated workflow to apply pixel art style.
Importing rendered images into Comfy UI for pixelation.
Running the pixelation process to convert images to pixel art.
Using a command line tool to remove backgrounds from images.
Compressing and organizing pixelated images into a Sprite sheet.
Finalizing the Sprite sheet and importing it into a game engine.