In Short:
Meta has launched a new AI model called Movie Gen, designed to create realistic videos and audioclips. It can generate 16-second high-def videos and make edits to existing clips, such as altering objects or appearances. While it’s not yet available, it aims to enhance media experiences on platforms like Facebook and Instagram. Movie Gen has 30 billion video and 13 billion audio parameters, promising high-quality outputs.
Meta has recently unveiled its new media-centric AI model, named Movie Gen, designed for the generation of realistic video and audio clips.
Capabilities Demonstrated
The company showcased several 10-second clips produced using Movie Gen, featuring various scenarios, including a baby hippo swimming, designed to illustrate its capabilities. Although the tool is not immediately available, the introduction of Movie Gen follows shortly after Meta Connect, which highlighted both new and updated hardware as well as the latest version of its large language model, Llama 3.2.
Advanced Editing Features
Movie Gen goes beyond simple text-to-video generation by allowing targeted edits to existing clips. This includes functions such as inserting objects into scenes or altering the appearance of surfaces. One example demonstrated a transformation where a woman with a VR headset was creatively altered to appear as if she were wearing steampunk binoculars.
Audio Integration
The model also generates audio to accompany the video clips. Sample clips included an AI figure standing by a waterfall, where sounds of splashing water mingled with orchestral music, as well as the purring engine of a sports car combined with screeching tires and a snake moving through a jungle, all set to suspenseful audio.
Technical Specifications
In a research paper released by Meta, further details about Movie Gen were disclosed. The Movie Gen Video model comprises 30 billion parameters, while the Movie Gen Audio model contains 13 billion parameters. The parameter count generally reflects a model’s capability; for context, the largest version of Llama 3.1 has 405 billion parameters. Capable of producing high-definition videos lasting up to 16 seconds, Meta asserts that the Movie Gen outperforms competing models in terms of video quality.
Potential Applications
Earlier this year, CEO Mark Zuckerberg showcased the Meta AI feature called “Imagine Me,” which allows users to upload a personal photo and role-play their image into various scenarios. This initiative hints at the potential for a similar video feature with the Movie Gen model, which could significantly enhance user engagement across platforms like Facebook, Instagram, and WhatsApp. Meanwhile, in September, competing company Google disclosed plans to integrate aspects of its Veo video model into YouTube Shorts next year.
Future Considerations
It remains to be seen how quickly Meta will make Movie Gen widely accessible. The announcement blog only briefly addressed a “potential future release.” For comparison, OpenAI introduced its AI video model, Sora, earlier this year, but has yet to unveil it for public use or provide a release timeline.
Exploration of Existing Tools
While major tech firms are still hesitating to fully release video models to the public, users can currently explore AI video tools from smaller startups, such as Runway and Pika. For instance, users curious about creative transformations can experiment with Pikaffects to visualize themselves in imaginative scenarios.
