Midjourney Video Updates + A Deeper Look at Sora

Curious Refuge
23 Feb 202413:21

TLDRThis week's AI news covers significant developments in the field, with a focus on Sora's capabilities compared to Runway and the challenges it faces for cinematic film making due to long rendering times. Despite Sora's potential for detail shots, image-to-video workflows remain key for AI video production. The community is abuzz with an AI-generated parody of Terminator 2 by a team of 50 AI artists. Sunno's AI music generation model impresses with its new features, and an AI film-making course opens for enrollment. Exciting advancements are announced for sound effects and AI models, including 11 Labs' text-to-sound effects model and Stability AI's stable diffusion version 3. Google Gemini's 1.5 pro model is highlighted for its ability to process vast amounts of text, audio, or video, hinting at future AI-generated films. Mid Journey's development continues with character consistency and faster rendering times, and rumors of AI video capabilities in version 7. The segment concludes with a humorous note on Will Smith's spaghetti meme and features of the week in AI films, showcasing creativity and technological advancements in the industry.

Takeaways

  • πŸ“£ The Hollywood Professional Association invited Shelby and the speaker to share their vision for a democratized filmmaking future with influential Hollywood figures.
  • πŸ” A comparison between Sora and Runway was discussed, highlighting the extreme differences in realism that Sora can achieve, although it may not be ideal for a back-and-forth filmmaking process due to long rendering times.
  • 🚧 Information about Sora's limitations was revealed, such as the challenge of achieving character and scene consistency, which is crucial for cinematic filmmaking.
  • πŸ˜‚ Online examples of 'Sora fails' were mentioned, suggesting that the tool is not without its issues and can produce unexpected results.
  • 🎬 An upcoming event in Los Angeles was announced, where a team of 50 AI artists will showcase a feature-length parody of Terminator 2, indicating the growing presence of AI in the film industry.
  • 🎡 Sunno's AI music generation model has been updated to version 3, offering faster generations, a dedicated instrumental button, and expanded language support.
  • 🎢 The music playing under the video was created using Sunno, demonstrating the practical application of AI-generated music.
  • πŸ“š An AI filmmaking and advertising course is opening for enrollment, targeting individuals interested in enhancing their storytelling skills with AI.
  • πŸ”Š 11 Labs announced a new text-to-sound effects model, which is a significant step towards automating sound effects in films.
  • πŸŽ‰ Congratulations were extended to 11 Labs for joining the Disney accelerator program, showcasing the collaboration between innovative AI companies and major studios.
  • πŸ“ˆ Stability AI's stable diffusion version 3 model is upcoming, promising better quality and more textual control over inputs, potentially rivaling other image generators on the market.
  • πŸ“± Elon Musk's interest in integrating Midjourney directly into Twitter was mentioned, indicating a future where AI art generation could be seamlessly integrated into social media platforms.

Q & A

  • What significant event was mentioned at the beginning of the transcript?

    -The significant event mentioned at the beginning of the transcript is the invitation to Shelby and the speaker by the Hollywood Professional Association to their annual Tech Retreat, where they shared their vision for a democratized filmmaking future with over 800 influential people in Hollywood.

  • What is the main issue with Sora as a cinematic film making tool according to the transcript?

    -The main issue with Sora as a cinematic film making tool is that it requires about an hour of rendering time to create a one-minute clip. This latency might not be suitable for a back-and-forth filmmaking process and could make it challenging to achieve maximum control over the generated scenes and maintain consistency.

  • What is the name of the AI music generation model that was updated and is said to have impressive results?

    -The name of the AI music generation model that was updated with impressive results is Sunno.

  • What is the feature of Sunno's version 3 that allows for faster generations and more accessibility?

    -Sunno's version 3 has a dedicated instrumental button and expanded language support, which makes it more accessible to a wider audience and allows for faster music generation.

  • What is the special announcement made by Tim from theoretically media?

    -The special announcement made by Tim from theoretically media is the congratulatory message to bck reels for winning $500 from AOL in a competition.

  • What is the significance of the Gemini 1.5 Pro model's ability to input up to 1 million tokens of information?

    -The significance of the Gemini 1.5 Pro model's ability to input up to 1 million tokens of information is that it allows for the uploading and processing of large amounts of text, audio, or video data. This capability could potentially be used to build AI-generated films in the future by reverse engineering the information from a video.

  • What is the current challenge with character consistency in AI projects that Mid Journey 6 aims to address?

    -The current challenge with character consistency in AI projects is maintaining consistency with characters, backgrounds, and style across different outputs. Mid Journey 6 aims to address this by providing a feature for character consistency.

  • What is the rumored feature of Mid Journey 7?

    -The rumored feature of Mid Journey 7 is the inclusion of AI video capabilities.

  • What is the significance of the AI film 'The Pomegranate Spell'?

    -The significance of the AI film 'The Pomegranate Spell' is that it is an entry into the Runway 48 Hour film competition, retelling the myth of Pany with beautiful romantic shots and realistic animation.

  • What is the theme of the AI film 'I Want to Be Happy'?

    -The theme of the AI film 'I Want to Be Happy' is about a robot that steals a microchip allowing it to experience emotions, and it falls in love with a puppy.

  • What is the current status of the integration of Mid Journey into Twitter?

    -The current status of the integration of Mid Journey into Twitter is that it is in conversations, but there are many details to be worked out before it can be fully integrated. Elon Musk has expressed interest in having an AI art generator directly on Twitter or X, regardless of the deal's outcome.

  • What is the purpose of the AI film news and how can one receive it directly?

    -The purpose of AI film news is to provide updates and insights into the world of AI filmmaking. One can receive it directly by signing up at Curious Refuge to have the news delivered to their inbox.

Outlines

00:00

πŸ“š AI News and Hollywood Tech Retreat Recap

The channel host expresses gratitude to the Hollywood Professional Association for inviting them to share their vision for democratized filmmaking. They discuss the potential and limitations of Sora, an AI tool for creating realistic visuals, noting its long rendering times and current lack of fine control over character and scene generation. The host also mentions upcoming events, such as a feature-length parody of Terminator 2 created by a team of AI artists, and introduces a new AI music generation model by Sunno, which offers faster song generations and more language support.

05:02

🎡 AI Music and Sound Effects Innovations

The host highlights the advancements in AI music generation with Sunno's new model, demonstrating its ease of use and sharing a personal anecdote about their mother using the tool. They also discuss 11 Labs' new text-to-sound effects model, which is set to revolutionize sound design with the ability to generate sound effects from text prompts. Additionally, the host congratulates a winner from a previous competition and mentions the integration of AI art generators into social media platforms, specifically Twitter, and the development of character consistency in Mid Journey 6.

10:04

πŸŽ₯ AI Filmmaking and Character Consistency

The host provides updates on the development of Mid Journey 6, emphasizing its upcoming features like character consistency and improved aesthetics. They also speculate about the potential inclusion of AI video in Mid Journey 7. The segment includes a humorous anecdote about a Will Smith meme and a parody video, followed by a discussion of AI films of the week, including 'The Pomegranate Spell,' 'The File,' and 'I Want to Be Happy,' which showcase the capabilities and creativity in AI filmmaking. The host concludes by inviting viewers to join their upcoming session and to subscribe for more AI film news.

Mindmap

Keywords

πŸ’‘AI

AI, or Artificial Intelligence, refers to the simulation of human intelligence in machines that are programmed to think like humans and mimic their actions. In the video, AI is central to the discussion as it explores various AI tools and their impact on the film industry.

πŸ’‘Sora

Sora is mentioned as a tool for creating realistic images, potentially for use in films. It is compared with Runway, another tool, and discussed in terms of its rendering time and current limitations in film making.

πŸ’‘Rendering Time

Rendering time refers to the duration it takes for a computer to process and generate a visual output, such as an image or video. In the context of the video, it is mentioned that Sora requires an hour of rendering time for a one-minute clip, which affects its usability in a dynamic film-making process.

πŸ’‘Control Over Generations

This concept relates to the level of detail and direction a filmmaker can exert over the creation of characters, scenes, and other elements in a film. The video discusses the challenges of achieving this with Sora, suggesting that it might not be ideal for all aspects of film production.

πŸ’‘Image to Video Workflows

This term refers to the process of converting still images into video format, often involving AI. The video suggests that, for now, this method might continue to be a staple in AI video production, especially for creative directing.

πŸ’‘Sunno

Sunno is an AI music generation model that has been updated to version 3, as mentioned in the video. It allows users to input a prompt and generate music in a specific style, making it accessible and fun for a wide audience.

πŸ’‘11 Labs

11 Labs is a company that has developed a text-to-sound effects model, which is expected to revolutionize the way sound effects are created for films. The video highlights the potential for AI to automate this process in the future.

πŸ’‘Disney Accelerator Program

The Disney Accelerator Program is a mentorship program that supports innovative companies, including those in the AI field. In the video, it is mentioned that 11 Labs has been accepted into this program, indicating a partnership that could push the boundaries of storytelling.

πŸ’‘Stable Diffusion

Stable Diffusion is an AI model for generating images from textual descriptions. The video discusses the upcoming version 3 of this model, which promises better quality and more control over the inputs.

πŸ’‘Mid Journey

Mid Journey is an AI tool for creating images and is mentioned several times in the video. It is discussed in the context of its development, with mentions of Mid Journey 6 and rumors about Mid Journey 7, including potential AI video capabilities.

πŸ’‘Character Consistency

Character consistency is the ability to maintain uniformity in the appearance and attributes of characters throughout a film or series. The video notes that Mid Journey 6 is testing a feature for character consistency, which is crucial for coherent storytelling in AI-generated content.

Highlights

AI developments have seen significant advancements with new tools and updates being covered in the AI news of the week.

The Hollywood Professional Association invited Shelby and the speaker to share their vision for a democratized filmmaking future at their annual Tech Retreat.

Sora, a tool for creating realism in film, was compared to Runway, showing a stark difference in capabilities.

Sora's rendering time for a one-minute clip is about an hour, which may not be suitable for a back-and-forth filmmaking process.

Current limitations of Sora suggest it might be challenging to achieve maximum control and consistency in film production.

Image-to-video workflows with creative direction are likely to remain a staple in AI video production.

Sora may be more useful for detail shots or establishing shots rather than for the bulk of a film.

A feature-length parody of Terminator 2, created by a team of 50 AI artists, is set to premiere in Los Angeles.

Sunno's AI music generation model has been updated to version 3, offering faster generations and more language support.

The music under the video was created using Sunno, showcasing the tool's capabilities.

Enrollment for an AI filmmaking and advertising course is opening on February 28th.

11 Labs announced a new text-to-sound effects model, which is set to be released soon.

11 Labs has been accepted into the Disney accelerator program for 2024, highlighting a partnership to push storytelling boundaries.

Stability AI's stable diffusion version 3 model is upcoming, promising better quality and more textual control.

Mid Journey 6 is in development, with features like character consistency and faster processing times.

Rumors suggest that Mid Journey 7 will include AI video capabilities.

Google's Gemini 1.5 pro model can input up to 1 million tokens of information, potentially impacting the film industry with AI-generated films.

Twitter is in conversations with Mid Journey for potential integration, as stated by Elon Musk.

The AI film 'Pomegranate Spell' was an entry in the Runway 48 Hour film competition, featuring beautiful romantic shots and realistic animation.

Jamie Roas Cassetti's film 'The File' explores biological experiments with advanced 3D models and interesting movement.

The film 'I Want to Be Happy' tells the story of a robot experiencing emotions after stealing a microchip.