Comparison questionMedium~10 minAI Fundamentals / 10 of 20

Interview question

How would you choose a model for a product feature?

Chooses a model from task-specific evaluation, operating constraints, safety, latency, cost, context, modality, and provider requirements.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

I start with the feature's real inputs, expected outputs, failure cost, and latency and budget constraints. Then I build a representative evaluation set and compare candidate models on the qualities the feature needs: correctness, instruction following, structured output, context size, language or modality support, refusal behavior, latency, and cost. I choose the smallest or simplest model that meets the requirement with acceptable margin rather than automatically choosing the most capable model.

Full answer below

Practice focus

Evidence-driven model selection that resists leaderboards, brand preference, and choosing the most capable model by default.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Model selection is a product and operating decision supported by a task-specific evaluation, not a one-dimensional benchmark contest.
I start with the feature's real inputs, expected outputs, failure cost, and latency and budget constraints. Then I build a representative evaluation set and compare candidate models on the qualities the feature needs: correctness, instruction following, structured output, context size, language or modality support, refusal behavior, latency, and cost. I choose the smallest or simplest model that meets the requirement with acceptable margin rather than automatically choosing the most capable model.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
The model also has to fit the system around it. I consider regional availability, data-use and retention terms, rate limits, version stability, observability, tool or schema support, and fallback options. Average benchmark scores are discovery signals, not acceptance tests. A coding benchmark says little about customer-support policy grounding, so I run our examples and inspect important failure slices instead of optimizing one aggregate score.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
I make the choice reversible. The application uses a versioned model configuration or gateway rather than spreading provider IDs through business code. Release evidence includes quality distributions, tail latency, token use, invalid-output and refusal rates, and cost per successful task. I test model changes like dependency changes, with shadow or canary traffic and rollback. If different task classes have different needs, routing can be justified, but only after the evaluation shows that the added complexity beats one well-chosen default.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Selecting from a public leaderboard alone.
Choosing the largest model without a task or cost threshold.
Ignoring provider data handling and regional constraints.
Changing the model and prompt together without isolating the effect.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

The production model cannot be identified or rolled back.
No representative evaluation set exists.
A vendor demo substitutes for testing the product's own inputs.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

What belongs in a model evaluation set?
When is a smaller model sufficient?
How would you compare refusal and invalid-output rates?
What makes model selection reversible?

Continue the section

Next in AI Fundamentals: Hosted vs self-hosted models

Step 11 of 20

Compare a hosted model API with a self-hosted or open-weight model.

Practice route

Read core answer Review checklist Follow-up prompts Track progress

Finished this drill?

Practice freely. Sign in only when you want to save completion, confidence, notes, or bookmarks.

New to Aporeon? Create account

Drill details

Human editorial direction

Aporeon is shaped by Aleksandar Tomovski, a software developer with experience on both sides of technical interviews. Content is reviewed for accuracy, natural spoken delivery, useful depth, and honest trade-offs.

About the creator Editorial standards

Was this drill useful?

Help improve the interview library.

Found this answer useful?

Say thanks with a standalone, one-time $5 contribution. No account required.

No subscription or perks.Choose another amount

Opening drill

Comparison questionMedium~10 minAI Fundamentals / 10 of 20

Interview question

How would you choose a model for a product feature?

Chooses a model from task-specific evaluation, operating constraints, safety, latency, cost, context, modality, and provider requirements.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

Full answer below

Practice focus

Evidence-driven model selection that resists leaderboards, brand preference, and choosing the most capable model by default.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Model selection is a product and operating decision supported by a task-specific evaluation, not a one-dimensional benchmark contest.
I start with the feature's real inputs, expected outputs, failure cost, and latency and budget constraints. Then I build a representative evaluation set and compare candidate models on the qualities the feature needs: correctness, instruction following, structured output, context size, language or modality support, refusal behavior, latency, and cost. I choose the smallest or simplest model that meets the requirement with acceptable margin rather than automatically choosing the most capable model.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
The model also has to fit the system around it. I consider regional availability, data-use and retention terms, rate limits, version stability, observability, tool or schema support, and fallback options. Average benchmark scores are discovery signals, not acceptance tests. A coding benchmark says little about customer-support policy grounding, so I run our examples and inspect important failure slices instead of optimizing one aggregate score.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
I make the choice reversible. The application uses a versioned model configuration or gateway rather than spreading provider IDs through business code. Release evidence includes quality distributions, tail latency, token use, invalid-output and refusal rates, and cost per successful task. I test model changes like dependency changes, with shadow or canary traffic and rollback. If different task classes have different needs, routing can be justified, but only after the evaluation shows that the added complexity beats one well-chosen default.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Selecting from a public leaderboard alone.
Choosing the largest model without a task or cost threshold.
Ignoring provider data handling and regional constraints.
Changing the model and prompt together without isolating the effect.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

The production model cannot be identified or rolled back.
No representative evaluation set exists.
A vendor demo substitutes for testing the product's own inputs.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

What belongs in a model evaluation set?
When is a smaller model sufficient?
How would you compare refusal and invalid-output rates?
What makes model selection reversible?

Continue the section

Next in AI Fundamentals: Hosted vs self-hosted models

Step 11 of 20

Compare a hosted model API with a self-hosted or open-weight model.

How would you choose a model for a product feature?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section

How would you choose a model for a product feature?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section