Stability AI, known for its AI-powered art generator Stable Diffusion, has launched a new open AI model called Stable Audio Open, designed for generating sounds and songs. The model was trained exclusively on royalty-free recordings, ensuring no copyright issues arise from its use.
Key Features of Stable Audio Open
Stable Audio Open operates by taking text descriptions, such as “Rock beat played in a treated studio, session drumming on an acoustic kit,” and generating recordings up to 47 seconds long. The model was trained on approximately 486,000 samples sourced from FreeSound and the Free Music Archive.
The model’s capabilities include creating drum beats, instrument riffs, ambient noises, and production elements suitable for videos, films, and TV shows. Additionally, it can edit existing songs or apply the style of one song to another.
“A key benefit of this open source release is that users can fine-tune the model on their own custom audio data,” Stability AI stated on its corporate blog. For instance, a drummer could use their own drum recordings to generate new beats.
Limitations and Commercial Use
However, Stable Audio Open has certain limitations. It cannot generate full songs, melodies, or vocals with high quality. Stability AI suggests that users seeking these features consider the company’s premium Stable Audio service.
Moreover, Stable Audio Open is restricted from commercial use as per its terms of service. It also exhibits performance biases across different musical styles and languages, which Stability AI attributes to the diversity limitations in the training data.
“The source of data is potentially lacking diversity and all cultures are not equally represented in the data set,” Stability AI explained. “The generated samples from the model will reflect the biases from the training data.”
Challenges and Controveries
Stability AI has faced challenges in stabilizing its business, and recently, it encountered controversy when Ed Newton-Rex, the VP of generative audio, resigned due to disagreements over the company’s stance on using copyrighted works for training generative AI models under “fair use.” Stable Audio Open seems to be part of an effort to shift this narrative while promoting Stability AI’s paid services.
As AI music generators like Stability AI’s gain traction, copyright issues have become a focal point. In May, Sony Music, representing artists like Billy Joel, Doja Cat, and Lil Nas X, issued warnings to 700 AI companies against unauthorized use of its content for training audio generators. Additionally, Tennessee signed the U.S.’s first law aimed at curbing abuses of AI in music in March.
Related News:
Featured Image courtesy of Stability AI