These AI Filmmaking Tools are Incredible!

Curious Refuge
10 May 202421:45

TLDRThis AI film news episode introduces an app called Simulan that uses AR and AI to create high-quality composites from simple phone recordings. It also covers the public beta release of Mid Journey, a tool for changing character appearances, and a lighting transfer tool for dynamic image re-lighting. The video discusses AI's impact on fashion and entertainment, with OpenAI's Natalie Potter chatbot and Udio's audio in-painting feature. It mentions AI film events, including the Cannes International Film Festival and AI on the Lot, and highlights the use of AI in music production and podcasts. The episode concludes with stories of AI's positive impact, such as enabling Randy Travis to sing again after a stroke, showcasing AI's potential to enhance storytelling and creativity in various industries.


  • ๐Ÿ“ฑ There's a new app called Simulan that uses AR and AI to enhance phone-taken videos into high-quality composites with 3D assets.
  • ๐Ÿš€ The compositing process is streamlined with tools like Simulan, which could revolutionize the way characters are added to scenes in film making.
  • ๐Ÿค– Mid Journey released a public beta for users who have generated over 100 images, and introduced a feature called Rooms for community interaction.
  • ๐ŸŽฅ Sora, an AI tool, is demonstrating the ability to alter characters in scenes based on text prompts, showcasing the flexibility of AI in content creation.
  • ๐Ÿงฅ An AI tool can now change the clothes of actors in images, which could significantly impact the future of advertising and film making.
  • ๐ŸŽจ AI is being used to change the lighting of images, suggesting a future where relighting photos or videos is as simple as a few clicks.
  • ๐ŸŒŸ The Cannes International Film Festival is hosting an AI film making Meetup, indicating the growing interest and acceptance of AI in the film industry.
  • ๐ŸŽ‰ AI film making events are gaining popularity, with large-scale events like AI on the Lot in Los Angeles attracting hundreds of attendees.
  • ๐ŸŽผ AI is being used to extend and inpaint audio tracks, hinting at a future where music production is heavily augmented by AI tools.
  • ๐Ÿ“š Open AI is reportedly working on a Google search competitor, which could offer a new way to search the internet using AI.
  • ๐Ÿ” AI tools are improving in their ability to identify AI-generated content, although challenges remain in distinguishing between real and synthetic media.

Q & A

  • What is the name of the app that creates high-quality VFX shots using just a phone?

    -The app is called Simulan, which uses AR technology to scan a 360 scene and composite characters in a scene, then upres the image using AI for maximum quality.

  • How does the Simulan app utilize cloud technology in its workflow?

    -After capturing the scene with a phone, the user sends the data to the cloud, where it is processed and returned with a highly realistic composite of the 3D asset.

  • What is the significance of the shadowing in the Mech suit robot scene?

    -The shadowing in the Mech suit robot scene is significant because it demonstrates the app's ability to simulate realistic lighting conditions, such as direct sunlight and the effect of walking under a tree's shadow.

  • How does the AI film news mention the use of AI in the future of advertising and film making?

    -AI film news discusses a tool that can change the clothes of actors in images, which will have huge implications for the future of advertising and film making, allowing for easy changes to costumes and outfits.

  • What is the name of the tool that allows changing the clothes of actors in images?

    -The tool is not explicitly named in the transcript, but it is described as allowing users to upload images to change the outfits of subjects in photos.

  • What is the AI film making event that is happening at the Cannes International Film Festival?

    -The event is an AI film making Meetup at the Cannes Innovation Studio, sponsored by Curious Refuge and Leonardo, taking place on Friday, May 17th.

  • What is the name of the AI tool that can change the lighting of any image?

    -The tool is not explicitly named in the transcript, but it is described as allowing users to input prompts and change the direction of lighting to achieve dynamic results.

  • What is the name of the AI film making competition mentioned in the script?

    -The specific name of the competition is not mentioned in the transcript, but it is noted that the winning film trailer will receive an Apple Vision Pro.

  • How does the AI spokesperson from the government of Ukraine help in the current crisis?

    -The AI spokesperson, named Victoria Shei, communicates information written and verified by humans in the Ukrainian government, which helps save resources and fight misinformation during the crisis.

  • What is the new feature released by the team at Mid Journey in their public beta?

    -Mid Journey released a feature called Rooms, which is similar to a Discord thread, allowing users to see others generating prompts in a specific style and enter into a voice chat.

  • What is the significance of the AI-generated music video created by Paul Trillo for the band Washed Out?

    -The significance is that it demonstrates the potential of AI in creating complex and visually appealing content that would have been cost-prohibitive and challenging to produce using previous technology.

  • How does the AI tool that changes the clothes of actors in images work?

    -The tool allows users to upload a reference image of clothing and then apply it to a different person's image. The system generates a new image with the uploaded clothing item composited onto the person in the photo.



๐Ÿ“ฑ Simulan App: Phone-to-Cloud VFX Magic

The script introduces an app called Simulan that enables users to create high-quality visual effects (VFX) shots using just their smartphones. The process involves scanning a 360-degree scene with AR technology to capture reflection maps and lighting data, then shooting a simple representation of a character in the scene with a phone. The composited, low-resolution scene is sent to the cloud and returned with a highly realistic composite. The script provides examples of impressive demos, including a robot car jumping over a bridge, robots fighting in a garage, a Mech suit robot in a park with realistic shadowing, and a detailed shot of a Porsche with accurate reflections. The tool's potential for ease in character compositing and the mention of Mid Journey's public beta and its 'Rooms' feature for community interaction are also highlighted.


๐ŸŽญ AI-Powered Clothing Changer and Relighting Tool

The script discusses a tool that allows users to change the clothes of actors in images based on uploaded reference images, which could revolutionize advertising and film production. It demonstrates the process of swapping a sweater onto a different person and mentions the potential for quick, drag-and-drop clothing changes in photos or videos. Additionally, the script covers an AI chatbot created by OpenAI in collaboration with the Met Gala, a lighting transfer tool that can dynamically change the lighting of an image, and the various AI film events and updates in the industry, emphasizing the growing integration of AI in creative fields.


๐ŸŽ‰ Upcoming AI Film Events and Innovations

The script provides information about upcoming AI film events, including the CAN International Film Festival and AI on the Lot in Los Angeles. It mentions the AI film making meetup at the CAN Innovation Studio and various talks and workshops. It also discusses the use of AI by the government of Ukraine to disseminate official messages through a virtual spokesperson and updates to the Uido tool that extend track duration and allow audio inpainting. The script highlights the potential for AI to identify fake content with high accuracy and the testing of AI for translating podcasts by Spotify, emphasizing the growing role of AI in content creation and accessibility.


๐ŸŽผ AI in Music Production and Style Presets

The script explores the future of music production with AI, mentioning a demo that showcases adding a piano solo to a track. It also discusses the OpenAI team's work on a Google search competitor and the release of a tool called Chat GPT Classifier, designed to identify AI-generated content. The script touches on Spotify's use of AI for podcast translation, enabling content to be accessible in multiple languages. It also covers the addition of style presets in P laab's tool, allowing users to change the style of generated videos with various options like claymation, 3D render, and anime style.


๐ŸŽฌ AI Films of the Week and Randy Travis' AI-Powered Song

The script highlights three AI films of the week: Paul Trillo's music video, which received a staff pick from Vimeo, an advertisement for a 1980s-inspired Diet Coke commercial by Paul Montreal, and 'The Rainforest,' an adventure film. It concludes with a story about country music artist Randy Travis, who, with the help of AI, was able to sing again after suffering a stroke. The script emphasizes the positive impact of AI, encouraging viewers to engage with the Curious Refuge community for more AI news and tutorials.



๐Ÿ’กAI Filmmaking Tools

AI filmmaking tools refer to software or applications that utilize artificial intelligence to assist in the creation and editing of films. These tools can automate complex tasks such as visual effects (VFX), character animation, and scene composition, making the filmmaking process more efficient and accessible. In the video, various AI tools are discussed, including Simulan for AR scene shooting and upscaling, and a tool for changing actor's clothing in images, which showcase how AI is revolutionizing the film industry.


Simulan is a specific AI tool mentioned in the video that allows users to shoot an augmented reality version of a scene and then enhance the quality using AI. It works by scanning a 360 scene to capture reflection maps and lighting data, then composites a character shot with a phone into the scene. The result is a high-quality, realistic composite that is processed in the cloud. This tool exemplifies the potential of AI to enhance the quality and realism of visual content created with minimal equipment.

๐Ÿ’กAR (Augmented Reality)

Augmented Reality (AR) is a technology that overlays digital information or images onto the real world, enhancing the user's perception of their surroundings. In the context of the video, AR is used to composite characters into real-world scenes for film production. The use of AR in filmmaking allows for more interactive and immersive storytelling, and when combined with AI, it can create highly realistic and detailed scenes as demonstrated by the Simulan tool.

๐Ÿ’กVFX (Visual Effects)

Visual Effects (VFX) are the manipulation or creation of digital imagery to produce scenes that cannot be achieved with conventional filming techniques. The video discusses how AI is being used to generate impressive VFX shots using just a smartphone, which would traditionally require extensive post-production work. AI-driven VFX tools are making it possible to create cinematic-quality visuals with less effort and at a lower cost.

๐Ÿ’กCloud Compositing

Cloud compositing refers to the process of combining different visual elements of a film or image in a cloud-based environment. The video mentions that after shooting a scene with AR technology, the data is sent to the cloud where it is processed and returned with a highly realistic composite. This method leverages the power of cloud computing to deliver high-quality results without the need for extensive local hardware resources.

๐Ÿ’กMid Journey

Mid Journey is mentioned as a tool that has released a public beta for users who have generated over 100 images. It also features a 'Rooms' functionality, which is similar to a Discord thread, allowing users to interact, share prompts, and engage in voice chat. This tool seems to be part of a broader ecosystem of AI-driven creativity, facilitating collaboration and idea sharing within the community of users.

๐Ÿ’กAI Chatbot

An AI chatbot is an artificial intelligence application designed to simulate conversation with human users. In the video, it is mentioned that Open AI and the Met have created an AI chatbot of Natalie Potter, a New York socialite. This indicates the integration of AI into various sectors, including fashion and art, to enhance user interaction and provide novel experiences.

๐Ÿ’กAI Relighting

AI relighting is the process of using artificial intelligence to change the lighting conditions of an image or video. The video demonstrates a tool that allows users to input a prompt and relights a photo with more dynamic and realistic lighting. This technology could greatly simplify the post-production process for filmmakers and photographers, enabling them to achieve desired lighting effects with ease.

๐Ÿ’กAI Filmmaking Meetup

The AI Filmmaking Meetup is an event that brings together professionals and enthusiasts in the field of AI filmmaking. The video mentions a specific meetup at the Cannes International Film Festival, organized by Curious Refuge in collaboration with Leonardo. These meetups serve as a platform for discussing the latest advancements in AI filmmaking technology and exploring its potential in the industry.

๐Ÿ’กAI Generated Content

AI generated content refers to any material, such as text, images, or videos, that are created using artificial intelligence algorithms. The video discusses the use of AI for creating music, images, and films. It also touches on the ethical considerations and challenges of identifying AI-generated content, such as with the Open AI's chat GPT classifier, which can detect AI-created content with high accuracy.

๐Ÿ’กAI in Music Production

AI in music production involves using AI algorithms to assist in the creation, editing, and enhancement of music tracks. The video highlights updates to a tool that allows for audio inpainting, where specific parts of a track can be altered or modified without affecting the rest of the song. This capability showcases how AI can revolutionize the way music is produced, offering new levels of creativity and precision.


A new app allows creation of incredible VFX shots using just a phone.

Simulan is an innovative tool that uses AR to shoot and upres scenes with AI for maximum quality.

The technology enables realistic composites with 3D assets, as demonstrated in online demos.

A robot car character example showcases character replacement and cinematic compositing.

A scene of two robots fighting in a garage demonstrates realistic camera shake and tracking.

A Mech suit robot walking through a park displays impressive shadowing and lighting effects.

A Porsche scene highlights realistic reflections in a phone-to-cloud compositing workflow.

The potential for AI tools to revolutionize character compositing in scenes with motion tracking and actor tracking.

Mid Journey releases a public beta and introduces a feature called Rooms for community interaction.

Sora demo showcases character and style changes based on prompts, indicating upcoming functionality.

Paul Trillo creates a music video for the band Washed Out using Sora, blending scenes experimentally.

An AI tool allows changing the clothes of actors in images based on uploaded images, with implications for advertising and film.

Met Gala features an AI chatbot of Natalie Potter, indicating a blend of traditional art and fashion with AI.

A lighting transfer tool can change the overall lighting of an image to be more dynamic.

AI film events like the CAN International Film Festival and AI on the Lot are bringing the industry together.

Ukraine's government uses an AI spokesperson named Victoria Shei for official messaging during crisis.

Udio updates its tool to extend track duration and introduce audio inpainting for music production.

Open AI is developing a Google search competitor and a tool to identify AI-generated content.

Spotify tests AI for translating podcasts, allowing content in the native tongue of the listener.

Plaab introduces style presets for video generation, allowing users to change the style with a few clicks.

AI film news highlights include a music video by Paul Trillo, an AI-generated Diet Coke ad, and a short film called 'The Rainforest'.

AI technology enables Randy Travis to sing again after a stroke, aiming to educate on the positive uses of AI.