CMU Researchers Propose XEUS: A Cross-lingual Encoder for Universal Speech trained in 4000+ Languages
What unique features does XEUS incorporate during training?
XEUS incorporates unique features like accented speech and code-switching during training, enhancing its robustness3. It includes new objectives such as dereverberation and noise reduction, and is based on the HuBERT architecture with enhancements like E-Branchformer layers and a simplified loss function3. Pre-trained on 1.081 million hours of data across 4,057 languages, XEUS uses advanced augmentation techniques and significantly more data than previous models.
What is Self-supervised learning (SSL) in speech technologies?
Self-supervised learning (SSL) in speech technologies refers to a machine learning approach where models learn meaningful speech representations from unlabeled data by solving pretext tasks that exploit statistical regularities in the data4. This reduces the need for labeled data and enables the models to be applied to a wider range of languages and tasks.
How many languages do current SSL models support?
Current SSL models support approximately 100-150 of the world's 7,000+ languages1. This limitation is primarily due to the scarcity of transcribed speech and the lack of resources to generate extensive annotated data needed for training1. However, researchers have developed XEUS, a Cross-lingual Encoder for Universal Speech, which is trained on over 1 million hours of data from 4,057 languages, significantly increasing language coverage for SSL models.