HEADLINE: Fish Audio Releases Fish Speech 1.5: Open-Source, Low-Latency TTS with Multilingual Support

Fish Audio has announced the release of Fish Speech 1.5, a significant update to their state-of-the-art text-to-speech (TTS) system. This new version aims to make cutting-edge voice synthesis technology more accessible and versatile for developers and practitioners alike.

Key Technical Changes and Why They Matter

Performance and Latency

• <150ms Latency: Fish Speech 1.5 boasts a latency of less than 150 milliseconds, which is crucial for real-time applications like live voice assistants or interactive storytelling. This low latency ensures that the system can provide near-instant responses, enhancing user experience.
• High-Quality Instant Voice Cloning: The update supports high-quality instant voice cloning, allowing users to generate realistic voices from a small sample of audio. This is particularly useful for personalized applications such as customizing AI assistants or creating unique character voices in games.

Multilingual Support

• 13 Languages Supported: Fish Speech 1.5 now supports 13 languages, including English, Chinese, Japanese, and more. This broad language support makes the system highly versatile for international projects.
• 1M Hours of Training Data: The model is trained on an extensive dataset of 1 million hours of multilingual audio, ensuring that it can handle a wide range of accents and dialects with high accuracy.

Open Source and Self-Hosting

• Pretrained Model Now Open Source: Fish Audio has released the pretrained model as open source, allowing developers to inspect, modify, and extend the system. This transparency is crucial for building trust and fostering community-driven improvements.
• Cost-Effective Self-Hosting or Cloud Options: The update provides flexible deployment options, including cost-effective self-hosting for those who prefer on-premises solutions, as well as cloud-based hosting for scalable deployments.

Detailed Features and Benchmarks

Performance Metrics

• Latency: <150ms (real-time applications)
• Quality: High-fidelity voice synthesis with minimal artifacts
• Language Support: 13 languages, including:
  • English
  • Chinese
  • Japanese
  • Spanish
  • French
  • German
  • Italian
  • Russian
  • Korean
  • Portuguese
  • Arabic
  • Hindi
  • Dutch

Training Data

• Dataset Size: 1M hours of multilingual audio
• Data Diversity: Includes a wide range of accents and dialects to ensure robust performance across different languages

Open Source Availability

• Repository: github.com/fishaudio/fish-speech
• Pretrained Model: Available for download, with detailed documentation on setup and usage

Use Cases and Demos

Fish Audio has provided several demos to showcase the capabilities of Fish Speech 1.5:

• News Reading: Demonstrate the system's ability to read news articles in a natural, human-like voice.
• Story Telling: Highlight the emotional range and expressiveness of the synthesized voices.
• Cross-Lingual Applications: Show how the multilingual support can be used for applications that require multiple languages.

Community and Future Plans

Fish Audio is actively seeking distinguished sales professionals and engineers to join their team. If you're interested in shaping the future of voice technology, you can contact them at hr@fish.audio.

Source