Japanese Reranker Cross-Encoder XSmall V1
hotchpotch/japanese-reranker-cross-encoder-xsmall-v1
published Mar 2024 · updated May 2025
Japanese Reranker Cross-Encoder XSmall V1 is a rerank model that scores the relevance between a query and passages, optimized for Japanese text retrieval.
specs
| Task | Reranking (Cross-Encoder) |
| Architecture | 6-layer MiniLMv2 with 384 hidden size |
| Parameters | Not specified |
| License | MIT License |
about this model
Model Overview
hotchpotch/japanese-reranker-cross-encoder-xsmall-v1 is a Japanese cross-encoder reranker model that scores the relevance of a query–document pair. It uses 6 transformer layers with a hidden size of 384 and accepts sequences up to 512 tokens.
Benchmark Performance
The model achieves the following scores on four Japanese retrieval evaluation datasets:
| Dataset | Score |
|---|---|
| JQaRA | 0.6136 |
| JaCWIR | 0.9376 |
| MIRACL | 0.7411 |
| JSQuAD | 0.9602 |
These results place the xsmall variant competitively among Japanese rerankers, outperforming many larger bilingual models while requiring significantly less compute.
Training and Efficiency
The model was trained on six Japanese datasets (JQaRA, JSQuAD, MIRACL, mMARCO, Mr.TyDi, and Wikipedia lead paragraphs) using 15 hard negatives per positive sample. Knowledge distillation from the large variant was applied to improve score quality. On an RTX 3090, evaluating the JaCWIR set took 196 seconds—approximately 6× faster than the large variant—making it suitable for latency-sensitive reranking pipelines.
best for
- ·Improving search result relevance for Japanese queries by re-ranking initial retrieval results
- ·Building high-accuracy Japanese question-answering pipelines
FAQ
The model takes a query and a list of passages as input, and outputs a relevance score (0 to 1) for each passage.
On an RTX3090, the xsmall variant is about 6x faster than the large variant (196s vs 1253s on JaCWIR evaluation).
The model is released under the MIT License.
Use the gigarouter OpenAI-compatible endpoint with your API key, sending the query and passages in the request.
We're benchmarking and onboarding Japanese Reranker Cross-Encoder XSmall V1 as a hosted, OpenAI-compatible API. Sign in for free credit and be ready when it lands, or tell us you want it and we'll prioritize it.