Roblox Builds In-House LLM to Bring Real-Time Chat Translation to Metaverse

Roblox, an online gaming platform, has developed an AI-enabled real-time translation system that allows gamers to communicate via text with one another despite speaking different languages.

In a February 5 blog post by Chief Technology Officer Daniel Sturman, the company revealed that it had to develop an in-house large language model (LLM) in order to achieve this. The “unified translation model” is able to translate text messages at a base latency of 100 milliseconds, making for a seemingly seamless, instantaneous messaging process.

“Imagine discovering that your new Roblox friend, a person you’ve been chatting and joking with in a new experience, is actually in Korea – and has been typing in Korean the entire time, while you’ve been typing in English, without either of you noticing,” said Sturman. “We’ve made possible on Roblox something that isn’t even possible in the physical world – enabling people who speak different languages to communicate seamlessly with one another in our immersive 3D experiences.”

The model can translate a total of 16 languages, including English, Chinese, French, German, Italian, Korean, Portuguese, Russian, and Turkish.

AI translation is not new to Roblox, which has been using the technology to translate its static, in-experience content. Real-time chat translation, however, is the platform’s way of removing language barriers and bringing its over 70 million daily active users together.

The context-aware model recognizes Roblox-specific language, including slang and abbreviations. It is also able to translate between any pair of languages. For example, a Turkish-speaking player can seamlessly chat with another who speaks only German or Korean. According to Sturman, two of the biggest challenges were creating a model that could independently translate between all language pairs while being fast enough to support real-time chats.

“To achieve this, we could have built out a unique model for each language pair (i.e., Japanese and Spanish), but that would have required 16×16, or 256 different models. Instead, we built a unified, transformer-based translation LLM to handle all language pairs in a single model,” Sturman explained. “Given a source sentence and target language, we can activate the relevant “expert” to generate the translations.”

For “less common” translation pairs like French and Thai, there was a lack of high-quality data to draw from, prompting the platform to use “back translation”, a system where the text is translated into the original language and compared to the source text for accuracy.

Iterative back translation was used during the training process, with “back-translated” data being combined with existing labeled data, thereby creating a richer dataset for the model to learn from. After this, the translations were run through a quality estimation model that focuses on improving the clarity of the translated text.