An AI artist explains his workflow

Vox
2 May 202308:18

TLDRThe transcript describes the creative process of an AI artist using Stable Diffusion technology to craft a scene featuring his alter ego, Stelfie, in a boxing match with Muhammad Ali. The artist emphasizes the importance of starting with a sketch and maintaining control over the AI's output, using various techniques and tools such as Photoshop and Procreate to refine the image. He discusses the significance of different parameters like samplers, steps, inpaint, and outpaint, and the challenges of accurately depicting faces and body shapes. The artist highlights the collaborative nature of working with AI, viewing it as an opportunity for artists to explore new creative avenues. He candidly shares the iterative process, from initial sketches to the final artwork, and the manual adjustments made to achieve a realistic portrayal of the characters.

Takeaways

  • 🎨 **Artistic Vision**: The AI artist uses the character Stelfie as an alter ego to explore adventures and showcase the potential of Stable Diffusion combined with artistic skills.
  • ✍️ **Sketching First**: The creative process begins with a hand-drawn sketch to maintain control over the original idea.
  • 🤖 **AI's Role**: Stable Diffusion and similar models are used for generating initial poses, but they can also lead the artist away from their initial vision.
  • 🖥️ **Photoshop for Refinement**: When the AI can't find the right pose, the artist manually recreates it in Photoshop, highlighting the importance of human control in the process.
  • 🔄 **Different Samplers**: The choice of sampler (like Euler or DPM) is crucial for achieving realism and detail in different aspects of the artwork.
  • 🔢 **Parameters Matter**: Parameters such as steps, inpaint, and outpaint significantly affect the output, allowing the artist to guide the AI's creativity.
  • 🤝 **Collaboration with AI**: The artist views the process as a joint effort with AI, not as a replacement for human creativity.
  • 👤 **Model Training**: A model trained on Stelfie's face is used for consistency, demonstrating the use of specific training for character representation.
  • 🔧 **Manual Adjustments**: The artist manually adjusts facial features in Photoshop to achieve a desired likeness, such as Muhammad Ali's face in the artwork.
  • 🎭 **Character Design**: Stelfie is intentionally designed to be less fit to contrast with typical athletic representations, showing a thoughtful approach to character portrayal.
  • 🖌️ **Digital Art Tools**: A mix of tools is used throughout the process, with Stable Diffusion, Photoshop, and Procreate each contributing to different stages of creation.
  • 👐 **Challenges with Hands**: The artist acknowledges the difficulty in reproducing hands and uses personal photographs as a reference, emphasizing the ongoing challenge in digital art.

Q & A

  • What is the main character in the artist's project?

    -The main character is Stelfie, a funny and clumsy individual who time travels and has incredible adventures.

  • What was the artist's goal when starting the project?

    -The artist aimed to capture a scene where Stelfie engages in a boxing match with Muhammad Ali, showcasing the potential of Stable Diffusion combined with good artist skills.

  • How does the artist begin the creation process?

    -The artist starts with drawing a sketch, which serves as the foundation for the artwork.

  • What role do samplers play in the artist's workflow?

    -Samplers are crucial for achieving realism and details in the artwork, such as replicating skin texture effectively.

  • What are the two important parts of the Stable Diffusion process mentioned by the artist?

    -The two important parts are inpaint and outpaint. Inpaint allows the machine to modify specific parts of the image, while outpaint asks the machine to imagine what's outside the image based on the existing content.

  • How does the artist use Stable Diffusion and other tools to create the artwork?

    -The artist uses a combination of Stable Diffusion for generating initial poses and ideas, Photoshop for refining poses and details, and Procreate for final touches. The process involves going back and forth between these tools.

  • What specific model does the artist use for Stelfie's face?

    -The artist uses a model that has been trained specifically on Stelfie's face using snapshots taken from different angles of a 3D model of Stelfie.

  • What is the significance of noise strength in Stable Diffusion?

    -Noise strength is important as it provides more or less control over the image itself, affecting the level of detail and randomness in the generated artwork.

  • How does the artist approach creating a realistic face of a well-known person like Muhammad Ali?

    -The artist asks Stable Diffusion to generate a face that resembles Muhammad Ali and then manually adjusts features like the nose, jaw, and eyes in Photoshop to achieve a more accurate representation.

  • What does the artist mean by wanting Stelfie to be 'a bit fluffy'?

    -The artist wants Stelfie to appear not super fit, without a six-pack, to reflect a more realistic and relatable physique rather than an idealized athletic one.

  • How does the artist describe the overall process of creating art with AI?

    -The artist views the process as a joint effort with the AI, where they drive the machine rather than the other way around, and sees it as an opportunity for new artists to explore a new branch of art.

  • Why does the artist use their own hand for the hands in Stelfie's artwork?

    -The artist uses their own hand because reproducing hands has always been challenging. They take a picture of their hand in the needed position, clean it up, and paste it onto the artwork.

Outlines

00:00

🎨 Stelfie's Artistic Creation and AI Collaboration

The first paragraph introduces Stelfie, a character who is portrayed as both humorous and clumsy, with a penchant for time travel and extraordinary adventures. The creator views Stelfie as an alter ego and utilizes Stable Diffusion, an AI model, to bring the character to life, particularly in a boxing match scenario with Muhammad Ali. The process involves initial sketching, experimenting with random prompts for poses, and manual adjustments in Photoshop when AI-generated poses are unsatisfactory. The importance of using different samplers for realism and detail, such as DPM for skin replication, is emphasized. Parameters like steps, inpaint, and outpaint are discussed for their roles in refining the AI's output. The creator also highlights the balance between AI and manual work, with Stable Diffusion contributing 50%, Photoshop 40%, and Procreate 10% to the final artwork. Stelfie's face is created using a model trained on 3D snapshots, and the noise strength parameter in Stable Diffusion is crucial for controlling the final image. The paragraph concludes with the creator's manual adjustments to Stelfie's physique to achieve a more realistic and non-athletic look.

05:02

🖌️ Refining the Artwork and the Role of the Artist

The second paragraph delves into the challenges and refinement process of the artwork. The creator discusses the need for manual adjustments to achieve a realistic portrayal of Muhammad Ali's physique, distinguishing the historical athlete's build from that of modern, highly muscular athletes. The iterative process involves going back to Stable Diffusion for assistance with specific details, such as edges, lighting, and skin tones. The creator emphasizes the importance of driving the AI process rather than being driven by it, viewing the collaboration between artist and AI as a joint effort. With a background in traditional and digital art, the creator sees AI as an opportunity for new artists to explore a different branch of art, opening up novel creative avenues. The paragraph also touches on the manual creation of hands in Stelfie's artwork, a task that has historically been challenging to replicate digitally, and the creator's personal contribution to this aspect of the art.

Mindmap

Keywords

Stelfie

Stelfie is a fictional character created by the AI artist, serving as an alter ego. He is depicted as a humorous and clumsy character who embarks on time-traveling adventures. In the context of the video, Stelfie is used as a subject for an artistic project that combines AI technology with traditional artistry. The artist aims to capture Stelfie in a boxing match with Muhammad Ali, showcasing the character's unique personality and the potential of AI in art creation.

Stable Diffusion

Stable Diffusion is an AI model mentioned in the transcript that is used for generating images from textual descriptions. It is described as being 'extremely good, but also extremely cheeky,' indicating that while it is powerful, it can also be unpredictable and may divert from the original artistic vision. The artist uses Stable Diffusion as a tool to explore initial poses and ideas for the artwork, emphasizing the importance of combining it with good artist skills.

Photoshop

Photoshop is a widely used digital image editing software that the artist employs to refine and manipulate the images generated by Stable Diffusion. In the video, the artist discusses using Photoshop to recreate poses that were difficult to achieve with AI, and to manually adjust facial features to resemble Muhammad Ali. Photoshop is integral to the artist's workflow, accounting for about 40% of the work process described.

ControlNet

ControlNet is mentioned as an extension that the artist would use if they were to reproduce a pose today. It is suggested that with ControlNet, the process would be significantly faster, taking only about 15 minutes. This highlights the evolving nature of AI tools and their potential to streamline parts of the creative process.

Samplers

Samplers in the context of the video refer to different algorithms within the Stable Diffusion model that determine how the AI generates images. The artist emphasizes the importance of choosing the right sampler for achieving the desired level of realism and detail in the artwork, such as using DPM for replicating skin texture more effectively.

Steps

Steps refer to the number of iterations the AI performs on a given prompt. The artist can choose a low or high number of steps, which affects the level of detail and refinement in the generated image. This parameter is crucial for controlling the output of Stable Diffusion and aligning it closer to the artist's vision.

Inpaint and Outpaint

Inpaint and Outpaint are techniques used in the editing process. Inpaint involves instructing the AI to modify specific parts of an image, while Outpaint asks the AI to imagine and create content beyond the existing image boundaries. These techniques are important for the artist's ability to direct the AI in creating the desired composition.

Procreate

Procreate is a digital illustration app used by the artist for a portion of the workflow. While it is used less frequently than Stable Diffusion and Photoshop, it still plays a role in the final touches of the artwork, contributing to about 10% of the work process as described by the artist.

Noise Strength

Noise Strength is a parameter in the Stable Diffusion web UI that allows the artist to control the level of detail and randomness in the generated image. A higher noise strength provides more control over the image, but it can also make achieving good results, especially with faces, more challenging.

Model Training

The artist discusses training a specific model on Stelfie's face using 3D snapshots from different angles. This process allows the AI to better recognize and replicate the character's facial features. Model training is a key aspect of customizing the AI to the artist's needs and ensuring the generated images are more accurate representations of the intended subject.

Artist's Part

The artist emphasizes the importance of the artist's role in the creative process, stating that they view the process as a 'joint effort with the AI.' Despite being a traditional artist for two decades, the artist sees AI as an opportunity rather than a threat, opening up new avenues for creativity and artistic expression.

Hands Replication

Reproducing hands is highlighted as an especially challenging aspect of the artistic process. The artist uses a practical solution by taking photographs of their own hand in the required position and then digitally integrating it into the artwork. This approach demonstrates a combination of traditional and digital techniques to overcome the limitations of AI in certain areas.

Highlights

Stelfie, the AI artist's alter ego, is a character involved in time-traveling adventures.

The project began to demonstrate the synergy between Stable Diffusion and artistic skills.

Capturing a scene of Stelfie boxing with Muhammad Ali was the initial goal.

The creative process starts with a sketch and involves using random prompts for initial poses.

Photoshop is used to recreate poses when Stable Diffusion fails to generate them.

ControlNet, an extension, can reproduce poses quickly with practice.

Different samplers are used throughout the process for varying levels of realism and detail.

Parameters such as steps, inpaint, and outpaint are crucial for guiding Stable Diffusion.

Stable Diffusion and Photoshop are used in a 50/40/10% ratio for the workflow.

A model trained on Stelfie's face is used for facial features, leveraging 3D snapshots.

Noise strength in Stable Diffusion provides control over the final image.

Replicating popular faces like Muhammad Ali's is challenging and requires manual adjustments.

The artist aims for a realistic, non-buff portrayal of Muhammad Ali's physique.

Photoshop is used for fine-tuning, including adjusting body shape, exposure, and skin tone.

The artist emphasizes the importance of driving the AI rather than being driven by it.

The artist views AI as an opportunity for new talent in the field of digital art.

Traditional art skills are not threatened but seen as complementary to AI in the creative process.

Hands are often recreated from the artist's own, highlighting the challenge of reproducing them digitally.