Qdrant’s blog (Clelia Bertelli, 2026-09-25) covers cross-encode-rs: Rust ONNX Runtime cross-encoder inference for reranking (query+doc scored together), aimed at AI rerank services without a Python runtime/GIL (blog).
On crates.io the owner is AstraBert (Clelia Bertelli), so this is not a Qdrant-org release; the announcement is on the Qdrant blog. The README notes Linux and macOS only.
What shipped
- Tokenize → length-sorted batches →
ortONNX session → sigmoid/softmax bynum_labels - Compared vs
fastembedandsentence-transformers(ONNX/CPU) on MiniLM + Jinajina-reranker-v2-base-multilingual(mteb/scidocs-reranking) - Ops angle: single binary, no GIL, controlled batching worker for concurrent load
Benchmarks (vendor claims)
All speedups are vendor-reported on a MacBook M4 Max—not independently verified (blog):
- MiniLM: ~1.1× faster at the median vs the Python stacks
- Larger Jina model: about 1.4× ahead (blog: ~1.3–1.5× across percentiles)
All three libraries hand work to onnxruntime, so this is not a claim of language superiority or Rust>Python FLOPs. Load-time: with Python startup excluded, session create is close; whole-process gap favors Rust mainly via no Python cold start.
Who should care
Teams building a dedicated rerank microservice who want an ONNX-backed binary without Python should read the Qdrant blog and the crates.io page—attribute benches, keep the AstraBert ownership note, and stay on Linux/macOS.

The Campfire
CommentsThanks. Your comment is waiting for the desk. Once it is approved it appears here within about a minute.
Add a comment