ElevenLabs, the AI voice startup known for its voice cloning, text-to-speech, and speech-to-speech models, has just added another tool to its product portfolio: an AI Voice Isolator. This innovative tool is available on the ElevenLabs platform starting today, allowing creators to remove unwanted ambient noise and sounds from any piece of content, ranging from films to podcasts or YouTube videos.
The introduction of the AI Voice Isolator comes just days after the launch of ElevenLabs’ Reader app. While the new tool is free to use with some limitations, it’s important to note that the concept of noise isolation is not entirely new to the market. Several other creative solution providers, including Adobe, already offer tools designed to enhance the quality of speech in content. The real test for ElevenLabs will be how effective their Voice Isolator is in comparison to these established solutions.
Comparison of Noise Isolation Tools
| Feature | ElevenLabs Voice Isolator | Adobe Enhance Speech | Other Competitors |
|---|---|---|---|
| Availability | ElevenLabs Platform | Adobe Creative Cloud | Various Platforms |
| Cost | Free with limits | Subscription-based | Varies |
| API Access | Coming Soon | Available | Varies |
| Effectiveness | High (initial tests) | High | Moderate to High |
| Unique Selling Point | AI-based post-production | Integrated with CC | Tool-specific features |
When recording content like films, podcasts, or interviews, creators often encounter background noise, which can interfere with the clarity of the main audio. These unwanted sounds can range from people talking, wind blowing, or vehicles passing by. Such noises might not be noticeable during the recording but can significantly affect the final output, often suppressing the speaker’s voice.
Traditionally, creators have used microphones with ambient noise cancellation to address this issue during the recording phase. While effective, these microphones can be costly and may not be accessible to early-stage creators with limited resources. This is where AI-based tools like the new Voice Isolator from ElevenLabs come into play.
At its core, the Voice Isolator functions during the post-production stage. Users upload the content they wish to enhance, and the underlying AI models process it to detect and remove unwanted noise, leaving clear dialogue as the output. ElevenLabs claims that the product can extract speech with a quality comparable to studio recordings. Ammaar Reshi, the company’s head of design, demonstrated the tool’s effectiveness by removing the noise of a leaf blower, resulting in crystal-clear speech.
- Ambient Noise Removal: Eliminates unwanted sounds like people talking, wind blowing, or vehicles passing.
- Post-Production Tool: Enhances audio quality after recording, making it accessible for creators with limited resources.
- High-Quality Output: Extracts speech with clarity comparable to studio recordings.
- User-Friendly: Easy to use; upload the content and let the AI models process it.
- Free with Limits: Available for free with certain usage limits, with paid plans for extended use.
To evaluate the real-world applicability of the Voice Isolator, we conducted three tests. In the first test, we recorded three separate sentences, each with different background noises. The other two tests involved recording sentences with a mix of random noises occurring irregularly. In all cases, the tool processed the audio within seconds, effectively removing noises associated with opening/closing doors, banging on tables, clapping, and moving household items. The only sounds it struggled to remove were banging on walls and finger snapping.
While the Voice Isolator’s ability to handle irregularly occurring background noise is impressive, there is still room for improvement. As with any AI-based tool, continuous development and refinement are essential. ElevenLabs is expected to further enhance the tool’s performance over time.
One area that remains somewhat opaque is the underlying models powering the Voice Isolator. The company has not disclosed specifics about these models or whether the recordings processed by the tool are used for training purposes. However, Sam Sklar, who handles growth at ElevenLabs, emphasized that users could opt out of the use of personal data for training through a form linked in the company’s privacy policy.
Currently, the Voice Isolator is available only through the ElevenLabs platform, with plans to open API access in the coming weeks. Although the exact timeline for this is unclear, it will likely broaden the tool’s accessibility and integration with other applications.
For those interested in trying out the tool, ElevenLabs offers free access with certain usage limits. According to Sklar, “The Voice Isolator model costs 1000 characters per minute of audio. We have a free plan on our site that comes with 10k characters/month, so it’s possible to use it with 10 minutes of audio per month for free.” Users requiring more extensive use will need to switch to paid plans, starting at $5 per month, billed monthly.
- Free Plan: 10k characters/month, equivalent to 10 minutes of audio per month.
- Paid Plans: Starting at $5/month for more extensive use.
- API Access: Planned for the coming weeks, no exact timeline yet.
The introduction of the AI Voice Isolator marks another significant milestone for ElevenLabs. By providing a tool that enhances audio quality in post-production, the company is addressing a common pain point for creators. While the tool shows promise, especially with its ability to handle irregularly occurring background noise, further improvements and transparency regarding its underlying models will be crucial for its success. As ElevenLabs continues to innovate and expand its product offerings, the AI Voice Isolator is set to become an invaluable tool for creators seeking to deliver high-quality audio content.
Featured Image courtesy of DALL-E by ChatGPT