Stability AI, for instance, has introduced Stable Video Diffusion
In the midst of the recent upheaval at OpenAI, other AI startups persist in advancing their product offerings. Stability AI, for instance, has introduced Stable Video Diffusion, an AI model designed to generate videos by animating existing images. This development follows the trajectory of Stability’s existing Stable Diffusion text-to-image model, presenting one of the few open-source and commercially available video-generating models.
However, Stability is taking a cautious approach with Stable Video Diffusion, currently labeling it as a “research preview.” Access to the model is subject to specific terms of use, delineating its intended applications, such as educational and creative tools, design, and artistic processes, while expressly excluding applications involving factual or true representations of people or events.
The potential misuse of the model looms as a concern, considering the historical patterns of AI research previews, which have sometimes found their way onto the dark web. This is particularly pertinent given the absence of an apparent built-in content filter for Stable Video Diffusion, raising concerns about potential abuse.
Stable Video Diffusion is presented in two iterations: SVD and SVD-XT. SVD transforms still images into 576×1024 videos with 14 frames, while SVD-XT, sharing the same architecture, increases the frame count to 24. Both models can generate videos at speeds ranging from three to 30 frames per second.
The models underwent initial training on a dataset comprising millions of videos, followed by fine-tuning on a smaller set of hundreds of thousands to around a million clips. The origin of these videos remains unclear, posing potential legal and ethical challenges related to usage rights, especially if copyrighted material was included.
Despite its limitations, Stability transparently communicates that Stable Video Diffusion can produce high-quality four-second clips. While unable to generate videos without motion or slow camera pans, control by text, render text legibly, or consistently generate faces and people accurately, Stability notes the models’ extensibility, suggesting potential adaptations for generating 360-degree views of objects.
Stability AI envisions a future where Stable Video Diffusion evolves into a range of models that build on and extend SVD and SVD-XT. The startup also plans to introduce a “text-to-video” tool, enabling text prompts for the models on the web. The overarching goal is commercialization, with Stability recognizing the model’s potential applications in advertising, education, entertainment, and beyond.
Amidst financial challenges and recent executive departures, Stability AI remains ambitious. Despite cash burn concerns and delayed payments, the company secured $25 million through a convertible note, bringing its total funding to over $125 million. Seeking a valuation quadruple its current status, Stability faces the pressures of low revenues and a high burn rate.
The departure of Ed Newton-Rex, former VP of audio, added to Stability AI recent setbacks. Newton-Rex cited a disagreement over copyright issues and the ethical use of copyrighted data in AI model training as the reason for his departure in a public letter. Stability AI navigates a complex landscape, balancing technological innovation with financial sustainability amid industry challenges
