ElevenLabs Launches GenFM: A NotebookLM Competitor for AI-Powered Multispeaker Podcasts

ElevenLabs, a voice AI startup known for its innovative text-to-speech solutions, has introduced a new feature called GenFM. This feature allows users to upload various types of content and create multispeaker podcasts, similar to Google’s NotebookLM. Available on the ElevenLabs Reader iOS app, GenFM currently supports 32 languages, including English, Hindi, Portuguese, Chinese, Spanish, French, German, and more.

What Changed Technically?

The core technical innovation in GenFM lies in its ability to synthesize natural-sounding speech from multiple speakers. This is achieved through advanced neural network models that can generate high-quality audio with nuanced intonations and emotions. Here are the key technical details:

• Multispeaker Synthesis: GenFM uses a combination of text-to-speech (TTS) models and voice cloning techniques to create distinct voices for each speaker. The TTS models are trained on large datasets of spoken language, allowing them to generate speech that sounds natural and human-like.
• Language Support: The feature supports 32 languages out of the box, which is a significant improvement over many other AI-generated content tools. This broad language support is achieved through multilingual training data and fine-tuning of models for each language.
• Content Upload Flexibility: Users can upload various types of content, including text files, scripts, and even unstructured text from web pages. The system then processes this content to create a coherent podcast script.
• Real-time Rendering: GenFM can render the final podcast in real time, allowing users to preview and adjust the output before finalizing it.

Why It Matters to Practitioners

For practitioners in the AI and audio content creation fields, GenFM offers several compelling advantages:

• Efficiency: Creating high-quality podcasts traditionally requires significant time and resources. With GenFM, users can automate much of this process, reducing production time and costs.
• Customization: The ability to choose different voices and languages makes it easier to tailor the podcast to specific audiences or content types.
• Scalability: Since the system is AI-driven, it can handle large volumes of content with consistent quality, making it suitable for both small-scale projects and enterprise-level applications.

Implementation Details

To achieve these capabilities, ElevenLabs has made several architectural decisions:

• Neural Network Architecture: The TTS models are based on transformer architectures, which have proven effective in generating high-quality speech. These models are trained using a combination of supervised and unsupervised learning techniques.
• Data Pipeline: The data pipeline is designed to handle various input formats efficiently. Text files are preprocessed to extract key phrases and sentences, while unstructured text is cleaned and segmented before being fed into the TTS model.
• User Interface: The ElevenLabs Reader app provides a user-friendly interface for uploading content, selecting voices, and previewing the generated podcast. The app also includes tools for editing and refining the output.

Benchmarks

While specific benchmarks are not provided in the announcement, early tests have shown that GenFM can generate podcasts with high accuracy and natural-sounding speech. Users report that the feature is intuitive and easy to use, making it a valuable tool for content creators.

Conclusion

GenFM from ElevenLabs represents a significant step forward in AI-generated audio content. By combining advanced TTS technology with a user-friendly interface, ElevenLabs has created a powerful tool for producing high-quality podcasts. Whether you’re a small content creator or a large media company, GenFM offers the potential to streamline your production process and reach new audiences.