Stable Diffusion Ultimate Guide. How to write better prompts, and use Image to Image, Control Net.

23 Dec 202359:54

TLDRThis comprehensive guide offers an in-depth exploration of stable diffusion, a technology that enables the creation of stunning images. The video covers the fundamentals of crafting effective prompts, utilizing advanced techniques such as prompt weightage and keyword blending, and selecting the optimal models for image generation. It also delves into tools like ControlNet for influencing image generation and various methods for enhancing and upscaling images. The host provides practical demonstrations using platforms like Playground AI and Leonardo AI, showcasing how to refine and upscale images for higher quality. The guide concludes with a discussion on the author's personal workflow and additional resources for further learning.


  • 🎨 **Image Generation with Stable Diffusion**: You can create various styles like fantasy, realistic portraits, and illustrations using stable diffusion models.
  • ✍️ **Writing Better Prompts**: Use a specific format for prompts including style, subject, details, colors, lighting, and keywords to guide the image generation process.
  • 📈 **Prompt Weightage and Blending**: Emphasize or deemphasize certain aspects of your prompt by using weightage and blend keywords for a mix of styles or features.
  • 🖼️ **Model Selection**: Choose the right model based on the desired output, such as Night Vision XL for realism or Dream Shaper for digital art.
  • 🌐 **Best Websites for Stable Diffusion**: Utilize platforms like Civit AI for a variety of models, or Leonardo AI for artistic styles with advanced features.
  • 🔍 **Advanced Techniques**: Use in-painting for editing parts of images, image-to-image for variations, and control net for influencing style while keeping the composition.
  • 🔧 **Image Enhancement**: Apply methods like high-resolution fix, separate upscaling, or use external sites for enhancing and upscaling images.
  • 🚫 **Negative Prompts**: Exclude unwanted elements by using negative prompts to guide the model away from generating undesirable features.
  • 📚 **Learning Resources**: Refer to the stable diffusion prompting guide and utilize tools like Prompto Mania for better prompt construction.
  • 💡 **Artistic Styles**: Use recognized artist names as keywords to influence the style of the generated images, but avoid copying living artists' styles.
  • 🔄 **Upscaling and Editing**: Perform final touches on images using tools like Photoshop or online editors for color and lighting adjustments.

Q & A

  • What is the main topic of the video?

    -The main topic of the video is the 'Stable Diffusion Ultimate Guide', which covers how to write better prompts and use various features like Image to Image and Control Net for generating images with stable diffusion models.

  • What are the key components of a good prompt for stable diffusion?

    -A good prompt includes specifying the style of the image, a verb to describe the action of the subject, details about the subject, colors to be used, lighting, and keywords to improve the image's contrast and detail.

  • What are some recommended keywords to enhance image quality in stable diffusion?

    -Some recommended keywords include 'Canon 50', 'DSLR' for photorealistic images, 'rendered by Octane' for 3D animation style, '4K' for increased detail, and 'hyper-realistic' to improve clothing and facial features.

  • What is the purpose of negative prompts in stable diffusion?

    -Negative prompts are used to avoid including certain elements or styles in the generated images. They help improve the image by instructing stable diffusion not to include keywords associated with undesirable outcomes.

  • How can prompt weightage help in generating images?

    -Prompt weightage allows users to emphasize or deemphasize certain keywords in a prompt. By using brackets, users can adjust the weightage of keywords, which helps stable diffusion to focus more on important aspects of the prompt.

  • What is the role of 'prompt scheduling' in image generation?

    -Prompt scheduling changes a word to another word after a specific number of steps during the image generation process. This results in a blend between two keywords, which can be useful for creating a mix of two different art styles or elements.

  • Which tool is recommended for generating prompts suitable for stable diffusion?

    -The tool 'Magic Prompt' is recommended for generating prompts suitable for stable diffusion as it understands the tag-style keywords that stable diffusion requires.

  • What is the significance of using artist names in prompts?

    -Using artist names in prompts can influence the style of the generated image. However, it's important to use names of artists recognized by stable diffusion and to blend them with other keywords for a consistent facial feature across multiple prompts.

  • What are some recommended models for different styles in stable diffusion?

    -For realism, 'Night Vision XL' is recommended. For digital art, 'Dream Shaper XL' and 'Stable Vision XL' are suggested. For fantasy style, 'Mysterious Version 4' for stable diffusion and 'Ranimated' for stable diffusion 1.5 are recommended. For anime, 'Counterfeit XL Version 1' and 'Counterfeit Version 3' for stable diffusion 1.5 are suitable.

  • What are the advantages and disadvantages of using different image generation websites like CIVIT AI, Get Image, and Leonardo AI?

    -CIVIT AI offers a wide variety of models and supports prompt weightage and scheduling but lacks image-to-image control net and painting features. Get Image provides a good variety of models with in-painting and out-painting features but has a limited monthly credit system. Leonardo AI is good for stylized and artistic images, offers advanced painting features, but has limited credits and does not include the latest version of stable diffusion XL.

  • How can one enhance or upscale an image generated with stable diffusion?

    -One can enhance or upscale an image using built-in features of the generation tools like the highest fix in Easy Diffusion or separate upscaling in Leonardo AI or playground AI. Additionally, external sites like Gigapixel (db.LOL) or Kaa can be used for upscaling, with adjustments to the AI strength depending on the content of the image.



🎨 Introduction to Stable Diffusion Guide

The video begins with a welcome to the channel and an introduction to the topic of stable diffusion, a tool for generating images. The speaker outlines the guide's content, which includes basics of prompting, keywords, tools, advanced techniques, model selection, best websites, settings, and image enhancement. The importance of choosing the style of images, such as fantasy, emo, artistic, anime, landscapes, or realistic portraits, is emphasized. The structure of a good prompt is discussed, including specifying style, subject, details, colors, lighting, and keywords.


📝 Prompting Techniques and Tools

The paragraph delves into the specifics of crafting better prompts, using tools like Prompto Mania and Magic Prompt to enhance the process. It also touches on advanced prompting techniques such as prompt weightage, keyword blending, and the limitations of stable diffusion in understanding natural sentences. The use of negative prompts to refine image generation and the concept of prompt scheduling to blend styles are introduced.


🎭 Advanced Prompting and Consistent Facial Features

This section discusses advanced techniques like keyword blending and generating consistent facial features across multiple prompts using celebrity names. The importance of using recognized artist names for style influence and the recommendation of specific artists for stable diffusion are covered. The paragraph also provides a cheat sheet for artists in the description below.


🖼️ Model Recommendations and Comparisons

The speaker provides recommendations for different models in stable diffusion based on the desired style, such as realism, digital art, fantasy, or anime. A comparison of various models is shown, highlighting their unique styles and suitability for different types of images. The use of specific models for generating high-quality images in different styles is emphasized.


🌐 Best Websites for Stable Diffusion

The paragraph discusses various websites for using stable diffusion, including Civit AI, Get Image, Leonardo AI, Playground AI, Stable UI, and Easy Diffusion. Each platform's pros and cons are outlined, such as the variety of models, user interface, credit systems, and specific features like in-painting and image-to-image control.


⚙️ Exploring Stable Diffusion Settings

The video explains important settings in stable diffusion, including seed, CFG (prompt guidance), sampler, and steps. The impact of each setting on image composition, quality, and speed is detailed. The paragraph also introduces in-painting, a feature for modifying parts of images, and demonstrates its use with an example on Playground AI.


🖌️ Image Editing with In-Painting and Image-to-Image

The speaker demonstrates how to use in-painting for editing images by adding sunglasses and changing a shirt color. The process of using the canvas for image editing, including tools for erasing, masking, and segmenting objects, is shown. Additionally, the image-to-image feature is introduced, which uses an existing image to guide the creation of variations.


🛠️ Control Net and Image Enhancement Techniques

The paragraph covers the use of control net for influencing image generation through edge, pose, and depth mapping. The process of enhancing images through various methods, including high-resolution fixes, separate upscaling, and using external sites like Gigapixel and Kaa, is discussed. The video concludes with the presenter's personal workflow for generating and enhancing images.



💡Stable Diffusion

Stable Diffusion is an AI model that generates images from textual descriptions. It is a core concept in the video as it is the technology used to create the images being discussed. The video provides a guide on how to use Stable Diffusion effectively to produce high-quality images.


A prompt is the textual description entered into the Stable Diffusion text box to generate an image. It is a fundamental part of the image generation process and the video emphasizes the importance of crafting effective prompts to guide the AI in creating desired images.

💡Image to Image

Image to Image is a feature that allows the user to upload an existing image and use it as a reference to guide the creation of a new image. This tool is highlighted in the video as a way to generate variations or modifications of a previously created image.

💡Control Net

Control Net is a tool within Stable Diffusion that enables users to influence the image generation process by controlling aspects like edges, pose, and depth. The video explains how Control Net can be used to modify the style or details of an image while preserving its composition.


Keywords are specific words or phrases included in the prompt that help refine the AI's output to match the user's vision. The video discusses the strategic use of keywords to enhance image quality, such as '4K' for detail and 'DSLR' for photorealism.

💡Prompt Weightage

Prompt weightage is a technique used to emphasize or de-emphasize certain elements within the prompt by assigning weights to keywords. This advanced technique is mentioned in the video as a way to control the prominence of specific features in the generated images.

💡Negative Prompts

Negative prompts are keywords that are used to specify what should be avoided in the generated image. The video explains how negative prompts can improve image quality by instructing the AI to exclude unwanted elements or styles.

💡Artist Styles

Artist styles refer to the distinctive styles of known artists that can be referenced in the prompt to influence the aesthetic of the generated image. The video provides a cheat sheet for recognized artist names that work well with Stable Diffusion.


Upscaling is the process of increasing the resolution of an image, often to enhance its detail and quality. The video discusses various methods of upscaling, including using built-in features of certain AI tools or external websites like Kaa.


In-painting is a feature that allows users to modify specific parts of an image using Stable Diffusion. The video demonstrates how in-painting can be used to make edits like adding sunglasses to a subject or changing the color of their shirt.


In the context of the video, models refer to different versions or iterations of the Stable Diffusion technology, each with its own strengths and ideal use cases. The video provides recommendations on which models to use for various image styles, such as realism, digital art, or fantasy.


Stable Diffusion Ultimate Guide provides a comprehensive understanding of generating high-quality images using AI.

Learn how to write better prompts for stable diffusion to achieve desired image outcomes.

Discover the best keywords for prompts to enhance the image generation process.

Explore advanced prompting techniques such as prompt weightage and keyword blending.

Understand which model to choose for generating images with stable diffusion.

Get insights on the best stable diffusion websites and settings for image generation.

Master the use of Image to Image and Control Net for more control over the generated images.

Enhance your images post-generation for a better look and feel.

Create a wide variety of image styles including fantasy, artistic, anime, landscapes, and realistic portraits.

Use negative prompts to avoid undesired elements in the generated images.

Learn how to use prompt weightage to emphasize or deemphasize certain keywords in your prompts.

Explore the use of prompt scheduling to blend keywords and create a mix of art styles or elements.

Generate consistent facial features across multiple prompts using keyword blending.

Incorporate the styles of recognized artists to influence your image generation.

Compare different models like Dream Shaper, Night Vision, and Counterfeit for various styles.

Utilize the in-painting feature to modify parts of images with stable diffusion.

Experiment with image to image control to create variations of existing images.

Control net allows for the manipulation of image generation to match specific styles or poses.

Enhance and upscale your images using various methods like high-resolution fixes and external sites.