Hosted reranker models

Reranker models are specialized neural networks that take a query and a set of candidate documents and output relevance scores, reordering the list with the most relevant results first. They solve the problem of improving accuracy after an initial, cheap retrieval step—common in search engines, retrieval-augmented generation (RAG) pipelines, and enterprise question-answering. For example, a legal research platform might first retrieve 100 possibly relevant cases by keyword, then use a reranker to surface the top 5 that best match the user’s intent.

In production, rerankers are typically placed after a fast bi-encoder or keyword-based index. The retriever passes a limited number of candidates (e.g., 50–200) to the reranker, which re-scores them. This two-stage design balances recall and latency. When choosing between models, the primary trade-off is accuracy versus speed and memory. Smaller models like cross-encoder/ms-marco-MiniLM-L2-v2 offer higher throughput and lower latency, while larger ones like Qwen/Qwen3-Reranker-4B or cross-encoder/ms-marco-MiniLM-L12-v2 (and the multilingual jinaai/jina-reranker-v2-base-multilingual) provide stronger relevance signals at the cost of slower inference. On GigaRouter, 4 reranker models are live now, with 38 total being onboarded.

For most production call volumes, calling a hosted API eliminates the operational burden of managing GPU infrastructure, scaling, and model updates, making it simpler to integrate reranking without dedicating engineering resources to self-hosting.