Create Consistent, Editable AI Characters & Backgrounds for your Projects! (ComfyUI Tutorial)
TLDRThis tutorial introduces a workflow for creating AI-generated characters and backgrounds with ComfyUI, compatible with Stable Diffusion 1.5 and SDXL. It demonstrates how to generate multi-view character images, integrate them into various backgrounds, and control their emotions with prompts. The guide includes a free pose sheet, a step-by-step installation guide, and tips for customizing the workflow for unique projects like children's books or AI influencers. The video also covers techniques for upscaling images, fixing facial details, and generating character expressions, concluding with a workflow for placing characters in different locations and poses.
Takeaways
- 😀 The video tutorial demonstrates how to create consistent AI characters and backgrounds using Stable Diffusion 1.5 and SDXL.
- 🎨 A post sheet is introduced, which depicts a character's bones from different angles and can be used to generate characters with control net.
- 📚 The tutorial offers a free step-by-step guide for installing and setting up workflows in ComfyUI, including where to find and place models.
- 👤 The process includes generating a character sheet, which can be used to create AI influencers, children's books, or movies.
- 🧑🎨 The video shows how to customize characters with specific traits, such as creating a 'cheese influencer'—a friendly German living in the Alps.
- 🔍 If character generation results in inconsistencies or odd poses, the process suggests adding more descriptive prompts or adjusting the sampler settings.
- ✨ The workflow includes steps for upscaling images, enhancing face details, and saving different character poses as separate images.
- 😉 The character's expressions can be controlled with simple prompts, and the face detailer can be adjusted to achieve a Pixar-like style.
- 🌟 The final part of the workflow combines different expressions, upscales them, and integrates the character into various backgrounds.
- 📌 The tutorial also explains how to use the character sheet for further applications, like training a model in Mid Journey or using it in different locations.
- 🔧 The controllable character workflow is detailed, showing how to pose characters, integrate them into backgrounds, and adjust expressions and poses for consistency.
Q & A
What is the main purpose of the video tutorial?
-The main purpose of the video tutorial is to demonstrate how to create consistent, AI-generated characters, pose them automatically, integrate them into backgrounds, and control their emotions using simple prompts with a workflow compatible with Stable Diffusion 1.5 and SDXL.
What does the post sheet depicted in the video do?
-The post sheet, which can be downloaded for free on the creator's Patreon, shows a character's bones from different angles in the open pose format. It is used in conjunction with ControlNet to generate characters based on these bones.
How can one ensure automatic generation of the character using the ControlNet in the video?
-To ensure automatic generation, one should set the pre-processor to 'None' when using the ControlNet with the open pose.
What is the significance of using the 'wildcard XL turbo' model in the workflow?
-The 'wildcard XL turbo' model is used for faster generation of the character. It is recommended to match the K sampler settings to the model's recommended settings for optimal results.
How can one create a unique AI influencer as demonstrated in the video?
-To create a unique AI influencer, one can modify the prompt to include a specific niche, such as 'cheese influencer', and add descriptive elements like a mustache to the character to make it more distinctive.
What is the role of the face detailer in the workflow?
-The face detailer automatically detects all the faces in the image and red diffuses them to improve the consistency and quality of the faces, especially when they appear small or broken.
How can one save different poses of the character as separate images?
-In the workflow, there is an option to save all the different poses as separate images by cutting them out and saving them in the next step of the process.
What is the purpose of adding 'Pixar character' as a prompt in the face detailer settings?
-Adding 'Pixar character' as a prompt helps the face detailer to generate expressions that are more in line with the style of Pixar characters, enhancing the realism and appeal of the generated faces.
How does the workflow handle the integration of the character into different backgrounds?
-The workflow includes steps to pose the character, generate a fitting background, integrate the character into the background, and adjust the expression and face details. It also allows for the character to be placed into different locations and poses using the controllable character workflow.
What is the benefit of using the openpose.ai tool in the workflow?
-Openpose.ai allows for the creation of specific poses for the character by manipulating a skeleton into the desired pose, including individual finger movements. This ensures that the generated character matches the intended pose closely.
How can one train their own model based on the images created in the workflow?
-By saving out all the different images of the character's faces using the save image node in the workflow, one can train their own model to recognize and replicate the character's likeness in various scenes.
Outlines
🎨 AI Character Creation and Posing
This paragraph introduces a workflow for creating AI-generated characters with consistent poses and emotions, compatible with Stable Diffusion 1.5 and SDXL. The creator shares a free downloadable post sheet for character bone depiction in open pose format, which is used with ControlNet to generate characters in multiple views. The video demonstrates using this workflow in Comi with a custom guide for installation and setup. The process includes importing the post sheet, selecting a model, and adjusting sampler settings. The workflow is showcased by creating an AI influencer for cheese, a friendly German character living in the Alps, by adjusting the prompt and using descriptive language to refine the character's appearance.
🖼️ Integrating Characters into Backgrounds
The second paragraph delves into integrating AI-generated characters into different backgrounds. It discusses using the character reference tool in Mid Journey to place the character in various locations, and the challenges of achieving specific poses. The creator then presents a free workflow for posing characters and integrating them into backgrounds, detailing the steps of using a model, setting up IP adapters with character images, and creating poses with openpose.ai. The workflow includes generating backgrounds, compositing the character, and adjusting the image to fix seams, match focal planes, and integrate lighting. The paragraph concludes with tips on changing character poses and expressions, and the ability to generate numerous images of the character in various poses and locations.
🧀 Customizing AI Characters for Specific Niches
In the final paragraph, the focus shifts to customizing AI characters for specific niches, using the example of creating an AI character named Hans who presents cheese. The process involves adding elements to the character's prompt for more freedom in pose and expression, and using Stable Diffusion to generate images with or without control nets for different outcomes. The creator also discusses the potential for using the character sheet and workflow for training a model in Laura, saving images for character reference, and the endless possibilities for personalizing the workflow. The paragraph ends with an invitation for support on Patreon for exclusive resources and community access, and a humorous offer for the cheese industry to book Hans for presentations.
Mindmap
Keywords
Stable Diffusion 1.5
Control Net
Post Sheet
AI Influencer
ComfyUI
WildCard XL Turbo Model
Face Detailer
Expressions
IP Adapter
Openpose AI
Latent Noise Mask
Highlights
This video tutorial demonstrates how to create consistent AI characters and backgrounds for various projects using ComfyUI and Stable Diffusion 1.5 or SDXL.
A free downloadable post sheet is introduced, which helps in generating characters from different angles using the open pose format.
The use of ControlNet is emphasized for generating characters based on the bones depicted in the post sheet.
Instructions are given on setting the pre-processor to 'None' for automatic character generation using the open pose control net.
A step-by-step guide is provided for installing and setting up workflows in ComfyUI, including model and folder structure setup.
The process begins with importing the post sheet and choosing a model, with a recommendation to match the K sampler settings to the model's recommendations.
An example prompt for generating an AI influencer is given, with a humorous twist towards a cheese influencer.
The video shows how to adjust character generation by adding descriptive prompts and tweaking the sampler settings for consistency.
Techniques for upscaling images from 1K to 2K and using the face detailer for improving facial features are explained.
The option to save different character poses as separate images is discussed, along with generating expressions using the face detailer.
Adding 'Pixar character' to the prompt is suggested for achieving a more stylized look in the face detailer.
The importance of matching the steps CFG sampler and scheduler to the model being used is highlighted.
The final part of the workflow combines different expressions and upscales them, including a single image of the face.
The video explores additional applications of the character sheet and workflow, such as training a model using the generated images.
Using Mid Journey's character reference tool to place the character in different locations is demonstrated.
A free workflow for posing characters and integrating them into backgrounds is presented, with a focus on character likeness and background composition.
The use of Openpose.ai for creating character poses and integrating them into the workflow is explained.
Techniques for fixing seams, adjusting focal planes, and matching lighting between character and background are discussed.
The ability to change character poses and expressions by adjusting control net inputs and using the Open pose editor is shown.
The video concludes with the character 'Huns' presenting cheese in various poses and locations, showcasing the workflow's flexibility.
An invitation to join the Discord community and support the creator on Patreon for additional resources and example files is extended.