Debugging scenarioHard~15 minAI Fundamentals / 19 of 20

Interview question

An AI feature became slower and more expensive. How would you diagnose it?

Diagnoses an AI regression by decomposing request stages, configuration changes, token growth, retries, routing, tools, retrieval, caching, and provider behavior.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

I confirm the time window and affected feature, tenant, model, and request class, then split latency and cost by stage: context construction, retrieval, model calls, tool calls, validation, retries, and queueing. I compare the deployed configuration and input distributions with the previous baseline. Common causes include longer history or retrieved text, a model or routing change, more generated tokens, retry storms, cache misses, slower tools, or provider throttling.

Full answer below

Practice focus

Evidence-led production diagnosis across both conventional service signals and AI-specific usage and workflow dimensions.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
The investigation should locate which stage and request slice changed before attempting prompt or model optimizations.
I confirm the time window and affected feature, tenant, model, and request class, then split latency and cost by stage: context construction, retrieval, model calls, tool calls, validation, retries, and queueing. I compare the deployed configuration and input distributions with the previous baseline. Common causes include longer history or retrieved text, a model or routing change, more generated tokens, retry storms, cache misses, slower tools, or provider throttling.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
I use traces to follow representative slow and expensive requests and correlate provider request IDs, token counts, finish reasons, model version, retry and fallback paths, and dependency timings. p95 and p99 matter because averages can hide one costly slice. I check whether quality also changed: an optimization that shortened output may have triggered more repairs, while a retrieval change may have increased both context size and unsupported answers.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
I contain the incident before perfect diagnosis by enforcing spend and iteration limits, reducing admission, disabling a faulty route, or rolling back the configuration. Then I quantify cost per successful task rather than per model call and separate traffic growth from unit-cost regression. Sensitive prompts are not dumped into logs; redacted metadata and approved sampled traces provide evidence. The fix receives a regression evaluation and a canary because token and latency improvements that damage task completion simply move the cost elsewhere.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Blaming the provider before decomposing the workflow.
Looking only at average latency or aggregate spend.
Ignoring retries, repairs, tool loops, and cache misses.
Logging complete sensitive prompts to gain visibility.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

No trace links model, retrieval, and tool stages.
A loop can make an unbounded number of calls.
The team cannot compare cost per successful task before and after the change.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

What would you contain immediately?
Which dimensions would you use to slice the regression?
How can retrieval increase model cost?
What evidence can be retained without storing sensitive prompts?

Continue the section

Next in AI Fundamentals: Design a first LLM feature

Step 20 of 20

How would you design a safe first LLM feature for an existing product?

Practice route

Read core answer Review checklist Follow-up prompts Track progress

Finished this drill?

Practice freely. Sign in only when you want to save completion, confidence, notes, or bookmarks.

New to Aporeon? Create account

Drill details

Human editorial direction

Aporeon is shaped by Aleksandar Tomovski, a software developer with experience on both sides of technical interviews. Content is reviewed for accuracy, natural spoken delivery, useful depth, and honest trade-offs.

About the creator Editorial standards

Was this drill useful?