OpenVoice, a groundbreaking open-source AI, is setting new benchmarks in voice cloning technology with its remarkable speed and precision.
This innovative AI tool is a collaborative effort by specialists at MIT, Tsinghua University, and the Canadian startup MyShell. OpenVoice is designed to clone voices using only a few seconds of audio input, offering detailed control over various aspects such as tone, emotion, accent, rhythm, and more.
MyShell, the driving force behind OpenVoice, recently introduced this technology through a blog post. The post includes a link to a research paper that has yet to undergo peer review, detailing the technology behind OpenVoice. Additionally, MyShell and HuggingFace provide demo sites where interested users can test the technology firsthand.
The announcement tweet from MyShell underscores their commitment to making AI accessible to all:
“Today, we proudly open source our OpenVoice algorithm, embracing our core ethos – AI for all. Experience it now: [link]. Clone voices with unparalleled precision, with granular control of tone, from emotion to accent, rhythm, pauses, and intonation, using just a… [image link]”
— MyShell (@myshell_ai) January 2, 2024
OpenVoice utilizes dual AI models that work in tandem for effective text-to-speech conversion and voice tone cloning. The first model is trained on a diverse set of 30,000 audio samples, including various emotions from English, Chinese, and Japanese speakers. This model handles language styles, accents, emotions, and other speech patterns. The second model, a “tone converter,” has learned from a massive dataset of over 300,000 samples across 20,000 voices. This dual-model approach allows OpenVoice to replicate voices with minimal data input, making it much faster than competing technologies like Meta’s Voicebox.
MyShell, the California-based startup behind OpenVoice, was founded in 2023. With an impressive early investment of $5.6 million and a rapidly growing user base of over 400,000, MyShell positions itself as a decentralized hub for AI application development and discovery. Besides leading the way in instant voice cloning, the company offers a range of products, including unique text-based chatbots, meme creation tools, text RPGs crafted by users, and more. While some of its content is subscription-based, MyShell also monetizes by allowing bot creators to promote their bots on the platform.
Through the strategic move of open-sourcing its voice cloning technology on platforms like HuggingFace and monetizing its wider app ecosystem, MyShell aims to expand its user base and champion a collaborative, open approach to AI development.
