Jake Seal is the founder of BlackVFX, an award-winning company that leverages the significant potential of AI advancements in the visual effects (VFX) industry. Projects that BlackVFX has worked on include Spider-Man 2 (2004), The Amazing Spider-Man (2012), Point Break (2015), The Shallows (2016), Back Roads (2018) and Jeepers Creepers: Reborn (2022). This article will look at the ever-increasing role of AI in VFX creation and how it helps to create more realistic and immersive film and television experiences for audiences.
The VFX industry has always been at the forefront of technological innovation, consistently pushing the boundaries of what is possible in digital imagery. Latterly, one of the most disruptive elements in the field has been integration of AI and machine learning techniques, revolutionising numerous aspects of VFX production. From streamlining mundane, repetitive tasks to generating complex simulations, AI has the potential to completely reshape how visual effects are conceived, developed, and integrated into TV and film productions.
One of the most immediate impacts of increased adoption of AI in VFX creation has been the ability to automate routine tasks. Take for instance rotoscoping, which was traditionally a extremely labour-intensive process. Today, AI-powered tools are automatically tracing and separating elements in footage, leading to a significant reduction in the time required for rotoscoping.
Besides rotoscoping, realistic facial animation is a major sticking point for the believability of any film. Today’s filmmakers are increasingly relying on AI to generate highly lifelike and accurate facial animations for digital characters, with these technologies analysing the expressions and facial movements of human subjects to produce increasingly realistic digital representations.
Machine learning algorithms have improved the speed and accuracy of motion tracking, making it easier to integrate CGI into live-action footage. Meanwhile, AI makes cleanups faster and more efficient by assisting in the removal of unwanted elements from footage such as rigs, wires and modern objects in period productions.
One aspect where AI algorithms have particular potential is in the enhancement of image quality. AI denoising tools can be employed to clean up noisy or grainy footage while preserving important details. In addition, machine learning models can be relied on to intelligently upscale low-resolution footage, adding realistic texture and detail. Machine learning algorithms can also generate intermediate frames, paving the way for smoother slow-motion effects or the conversion of low frame rate footage to higher frame rates.
Today, much of the time-consuming and laborious hand work traditionally associated with VFX can be at least partially automated. In doing so, film studios can increase cost effectiveness, reducing the need for large teams and driving down production costs. Automating VFX ultimately enhances creativity by reducing time spent on repetitive tasks.
Pixar released the animated film Elemental in 2023, with the movie’s characters more abstract than ever before. While the prospect of making characters composed of water, air or fire is no easy task, volumetric neural style transfer made this a reality. Taking the raw voxels of ‘pyro simulation’ and passing it throughout a style transfer network trained on an artist’s impression of what they wanted, the creative team produced voxels with a natural, unpredictable look but also the unmistakable cast of the artist’s styling. Using a machine learning technique to shape the silhouette’s edges, Pixar avoided the need for hand sculpting and animating every detail, which in this case was simply not an option.
About BlackVFX
BlackVFX provides a full spectrum of services to television and film production companies, including virtual sets that enable filmmakers to create 3D environments with realistic textures, lighting and props. BlackVFX’s photogrammetry service facilitates the capture of high-quality 3D models of real-world environments and objects, while its pre-visualisation service helps clients to take those all-important first steps in producing VFX shots.
With more than 15 years of in-depth industry experience, the BlackVFX team has worked with renowned filmmakers, helping visions come to life and delivering high-quality postproduction services. The organisation has been recognised with several coveted industry awards, including earning recognition at the Grammy Awards, Clio Awards and MTV Video Music Awards.