How would you choose top-k, relevance thresholds, and abstention behavior for RAG?

Tunes evidence quantity and no-answer behavior from measured relevance rather than treating similarity scores as universal confidence.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

I tune top-k on a labeled query set by measuring whether the needed evidence appears and how much irrelevant context reaches generation. Similarity scores are ranking signals, not universal probabilities, so a threshold must be calibrated for the chosen model, index, distance metric, and query type. When evidence is missing or weak, the application asks for clarification, falls back to search, or says it cannot answer.

Full answer below

Practice focus

Candidate-set sizing, score calibration, insufficient evidence, and risk-aware fallback.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
More retrieved chunks can add evidence, but they can also add noise, contradiction, latency, and cost.
I tune top-k on a labeled query set by measuring whether the needed evidence appears and how much irrelevant context reaches generation. Similarity scores are ranking signals, not universal probabilities, so a threshold must be calibrated for the chosen model, index, distance metric, and query type. When evidence is missing or weak, the application asks for clarification, falls back to search, or says it cannot answer.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
I examine recall at k, precision at k, answer support, and the effect of duplicate chunks. Easy fact lookups may need very few passages, while multi-part questions may need several sources or query decomposition. A reranker can let first-stage retrieval favor recall and then send fewer, better candidates to the model.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Setting top-k to the context window maximum.
Treating a cosine score as a calibrated confidence probability.
Forcing an answer when no evidence passes the policy.

How would you choose top-k, relevance thresholds, and abstention behavior for RAG?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue practice

Next in path

Tech stacks

Skill areas

Human editorial direction

Was this drill useful?

Found this answer useful?