GTE Multilingual Base

gte-multilingual-base is an encoder-only multilingual text embedding model that generates dense and sparse vector representations for input text, supporting up to 8,192 tokens and over 70 languages.

Architecture and Capabilities

Trained with an encoder-only transformer architecture, the model delivers a 10x increase in inference speed compared to decode-only LLM-based embedding models of similar size. It supports elastic dense embeddings, allowing the output dimension to be adjusted between 128 and 768 without sacrificing downstream effectiveness, and can also produce sparse token vectors for hybrid retrieval. With 305 million parameters and a native 8,192-token context, it is designed for long-document and multilingual retrieval tasks.

Benchmark Performance

According to the accompanying paper, the text encoder outperforms the same-sized XLM-R and matches the performance of the large-sized BGE-M3 model, achieving better results on long-context retrieval benchmarks. The model achieves state-of-the-art results on multilingual retrieval datasets including MIRACL and MLDR, cross-lingual retrieval on MKQA, and English retrieval on BEIR and LoCo. On the MTEB leaderboard it demonstrates strong performance across English, Chinese, French, and Polish tasks.

Research Background

The model is introduced in the paper mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval (EMNLP 2024 Industry Track). It is part of the GTE family and has been downloaded over 1.2 million times, reflecting its adoption in production and research environments.