Bilingual Embedding Base

Lajavaness/bilingual-embedding-base is an embedding model that encodes French and English sentences into a shared 1024-dimensional vector space using a custom BilingualModel architecture built on XLM-RoBERTa from SentenceTransformers. The model applies mean pooling over token embeddings followed by L2 normalization, producing fixed-size vectors suitable for semantic similarity, clustering, and retrieval tasks.

Architecture and Training

The model is based on XLM-RoBERTa with a maximum sequence length of 512 tokens. Training proceeds in three stages:

• Stage 1 – NLI training: Multi-Negative Ranking Loss on combined English (SNLI) and French (XNLI) datasets to learn sentence semantics.
• Stage 3 – STS fine-tuning: Siamese BERT networks with cosine-similarity loss on English and French STS Benchmark data.
• Stage 4 – Augmented SBERT: Data augmentation using cross-encoder labels (see Augmented SBERT), improving bi-encoder performance on pairwise scoring tasks.

Benchmark Performance (MTEB French tasks)

These results demonstrate effective cross-lingual understanding for both English and French in retrieval and clustering contexts.