Architecture scenarioHard~15 minAI Application Architecture / 5 of 18

Interview question

How would you route requests across models or deployments?

Uses explicit capability and risk policy to route by task while preserving evaluation, version attribution, budgets, and fallback semantics.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

I route from trusted application context such as task type, required capability, risk, region, latency target, and tenant policy. Each route names an evaluated model and configuration that is known to satisfy that task. The selected route is recorded on the trace so quality, cost, and failures can be compared accurately.

Full answer below

Practice focus

Model routing, capability policy, tenant controls, health, and measurable outcomes.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Routing is a product and reliability policy, not a random choice of the cheapest available model.
I route from trusted application context such as task type, required capability, risk, region, latency target, and tenant policy. Each route names an evaluated model and configuration that is known to satisfy that task. The selected route is recorded on the trace so quality, cost, and failures can be compared accurately.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
A simple classifier may use a smaller model, while a complex grounded explanation can use a more capable one. Health-aware routing can move between equivalent deployments, but switching to a materially different model is a product change with its own evaluation and output behavior. Tenant budgets, data residency, provider approval, and context limits constrain the eligible set before any optimization.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
Routing rules are versioned, canaried, and protected from self-reinforcing feedback where a weak router sends hard cases to the wrong path and then learns from poor outcomes. Confidence-based escalation uses calibrated evidence and a bounded number of attempts rather than unlimited model chains. Fallback routes preserve safety and contract guarantees even when quality is reduced. I monitor route distribution, override rate, task success, tail latency, and cost by slice so an efficiency gain cannot hide a capability regression.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Routing solely by model price.
Treating all healthy deployments as behaviorally equivalent.
Allowing the model to choose an unauthorized provider.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

No trace records why a route was selected.
Fallback changes output contracts silently.
Routing rules ship without route-specific evaluation.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

When is health-based failover safe?
How would you calibrate escalation?
Which constraints filter eligible models first?

Continue the section

Next in AI Application Architecture: Manage AI conversation state

Step 6 of 18

How would you manage conversation state in an AI application?

Practice route

Read core answer Review checklist Follow-up prompts Track progress

Finished this drill?

Practice freely. Sign in only when you want to save completion, confidence, notes, or bookmarks.

New to Aporeon? Create account

Drill details

Human editorial direction

Aporeon is shaped by Aleksandar Tomovski, a software developer with experience on both sides of technical interviews. Content is reviewed for accuracy, natural spoken delivery, useful depth, and honest trade-offs.

About the creator Editorial standards

Was this drill useful?

Help improve the interview library.

Found this answer useful?

Say thanks with a standalone, one-time $5 contribution. No account required.

No subscription or perks.Choose another amount

Opening drill

Architecture scenarioHard~15 minAI Application Architecture / 5 of 18

Interview question

How would you route requests across models or deployments?

Uses explicit capability and risk policy to route by task while preserving evaluation, version attribution, budgets, and fallback semantics.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

Full answer below

Practice focus

Model routing, capability policy, tenant controls, health, and measurable outcomes.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Routing is a product and reliability policy, not a random choice of the cheapest available model.
I route from trusted application context such as task type, required capability, risk, region, latency target, and tenant policy. Each route names an evaluated model and configuration that is known to satisfy that task. The selected route is recorded on the trace so quality, cost, and failures can be compared accurately.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
A simple classifier may use a smaller model, while a complex grounded explanation can use a more capable one. Health-aware routing can move between equivalent deployments, but switching to a materially different model is a product change with its own evaluation and output behavior. Tenant budgets, data residency, provider approval, and context limits constrain the eligible set before any optimization.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
Routing rules are versioned, canaried, and protected from self-reinforcing feedback where a weak router sends hard cases to the wrong path and then learns from poor outcomes. Confidence-based escalation uses calibrated evidence and a bounded number of attempts rather than unlimited model chains. Fallback routes preserve safety and contract guarantees even when quality is reduced. I monitor route distribution, override rate, task success, tail latency, and cost by slice so an efficiency gain cannot hide a capability regression.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Routing solely by model price.
Treating all healthy deployments as behaviorally equivalent.
Allowing the model to choose an unauthorized provider.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

No trace records why a route was selected.
Fallback changes output contracts silently.
Routing rules ship without route-specific evaluation.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

When is health-based failover safe?
How would you calibrate escalation?
Which constraints filter eligible models first?

Continue the section

Next in AI Application Architecture: Manage AI conversation state

Step 6 of 18

How would you manage conversation state in an AI application?

How would you route requests across models or deployments?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section

How would you route requests across models or deployments?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section