Meta, the parent company of Facebook, has launched an impressive new translation model called SeamlessM4T (Multilingual & Multimodal Translation Model). This foundational speech/text translation and transcription model is an all-in-one system that performs a range of tasks including speech-to-speech and speech-to-text translation, as well as text-to-text translation and speech recognition. The model supports input and output in 100 languages, with speech output available in 35 languages.
What sets SeamlessM4T apart from existing translator models is its comprehensive offering and superior performance. While tech giants like OpenAI and Google have already developed their own speech-to-text models (Whisper and AudioPaLM-2, respectively), SeamlessM4T surpasses them in terms of quality. To evaluate the performance of various Speech-to-Text (S2T) and Speech-to-Speech translation (S2ST) models, an Automatic Speech Recognition Bilingual Evaluation Understudy (ASR BLEU) metric was used, and SeamlessM4T demonstrated higher scores compared to its competitors.
The demand for multi-language translation is on the rise as companies recognize the importance of catering to diverse vernacular markets worldwide. Indian IT giant Tech Mahindra is also working on a Large Language Model (LLM) that will allow speech in numerous Indic languages, including Hindi. Meanwhile, Eleven Labs has introduced Eleven Multilingual v2, an AI speech model that supports 28 languages and offers enhanced conversational capabilities.
Meta’s decision to make SeamlessM4T publicly available under the CC BY-NC 4.0 license, which restricts the use of the model for commercial purposes, has sparked debate among users. Some argue that this move limits adoption and deviates from the conventional Apache licensing model. However, Meta’s recent release of Llama 2, a freely available multimodal model, suggests that concerns about limiting open-source access may not be warranted.
In the ever-evolving field of language translation and transcription, multimodality has become a sought-after feature. While OpenAI’s GPT-4 was expected to offer a multimodal platform allowing inputs via images, voice, and text, it hasn’t fully delivered on these promises. In contrast, Meta has consistently released multimodal models, including CM3leon, which generates both text-to-image and image-to-text translations.
While SeamlessM4T appears to be a promising addition to the translation model landscape, its adoption and impact remain to be seen. The non-commercial license may influence its widespread use, but its comprehensive offering of text and speech translation makes it an attractive option for users seeking multilingual and multimodal capabilities.
In conclusion, Meta’s SeamlessM4T is an innovative translation model that offers a wide range of features and outperforms existing models in terms of quality. As companies increasingly focus on multi-language translation, offerings like SeamlessM4T will play a crucial role in addressing diverse vernacular markets globally. However, the licensing approach and limitations on commercial use may impact its adoption and development within the open-source community. Nonetheless, this latest release underscores Meta’s commitment to advancing multimodal capabilities in the language translation space.
