hypencoder

Vespa sample applications - Hypencoder: query-dependent neural ranking

This sample application reproduces Hypencoder (SIGIR '25) in a Vespa rank profile. Hypencoder replaces cosine similarity with a hypernetwork: the query encoder generates the weights of a small query-specific neural network at query time, and that q-net is applied to each document's stored embedding to produce the relevance score.

The app demonstrates running Hypencoder entirely inside Vespa with no custom code: a hugging-face-embedder for the passage encoder, an onnx-model for the query encoder, and a rank profile that expresses the q-net forward pass as tensor expressions.

A forthcoming post on the Vespa blog covers the design and performance characteristics in detail.

Requires at least Vespa 8.338.38.

Prerequisites

• Docker or Podman.
• Vespa CLI (brew install vespa-cli on macOS).
• Python 3.10+ with pip install -r requirements.txt (just numpy and tokenizers).

Download the models

Pre-built ONNX exports live in huggingface.co/andreer/hypencoder-onnx. The passage encoder is fetched by Vespa at deploy time, so you only need the query encoder and the tokenizer locally:

mkdir -p app/models
BASE=https://huggingface.co/andreer/hypencoder-onnx/resolve/main/2_layer
curl -L -o app/models/query_encoder.onnx $BASE/query_encoder.onnx
curl -L -o app/models/tokenizer.json     $BASE/tokenizer.json

app/schemas/doc.sd references models/query_encoder.onnx; app/services.xml points the hugging-face-embedder at the passage encoder's URL. encode_query.py uses the tokenizer to build query tensors.

See EXPORT.md if you want to build the ONNX files yourself.

Start Vespa

docker run --detach --name vespa --hostname vespa \
  --publish 8080:8080 --publish 19071:19071 \
  --memory 12g \
  vespaengine/vespa:latest

vespa config set target local
vespa status deploy --wait 300

Deploy

vespa deploy app --wait 300

Feed

vespa feed dataset/sample.json

The passage_embedder runs server-side, producing the 768-d CLS embedding for each document.

Query

The app exposes four rank profiles:

profile	what it does
cosine_baseline	Plain cosine similarity. A reference point for the cost of bi-encoder scoring on this corpus.
hypencoder_onnx	Full Hypencoder rank-all: the q-net scores every matched document.
hypencoder_rerank	Cosine first-phase, Hypencoder q-net on the top 100.
hypencoder_lexical_rerank	BM25 first-phase, Hypencoder q-net on the top 100.

encode_query.py produces the query JSON for each profile. The flag selects the profile:

# cosine_baseline
python encode_query.py --cosine "tallest mountain in the world" > /tmp/q.json
vespa query --file /tmp/q.json

# hypencoder_onnx (default)
python encode_query.py "tallest mountain in the world" > /tmp/q.json
vespa query --file /tmp/q.json

# hypencoder_rerank
python encode_query.py --rerank "tallest mountain in the world" > /tmp/q.json
vespa query --file /tmp/q.json

# hypencoder_lexical_rerank
python encode_query.py --lexical "tallest mountain in the world" > /tmp/q.json
vespa query --file /tmp/q.json

References

• Hypencoder paper (arXiv 2502.05364)
• jfkback/hypencoder-paper - the original Hypencoder repo
• Vespa documentation: ONNX models in rank profiles
• Vespa documentation: hugging-face-embedder