Architecture scenarioHard~15 minAI Application Architecture / 8 of 18

Interview question

How would you design streaming responses for an AI feature?

Treats streams as event protocols with partial-output UX, cancellation, validation, moderation, and terminal outcome semantics.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

I stream typed events such as started, text delta, tool status, completed, refused, and failed rather than forwarding raw provider chunks directly. The client renders partial output as provisional and commits the final result only after the terminal event and required validation. Disconnects propagate cancellation where possible, while the server still records the actual provider outcome.

Full answer below

Practice focus

Streaming protocol design, partial failure, safety, backpressure, and reconnect behavior.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Streaming reduces time to first visible output but exposes partial content before the complete result is known.
I stream typed events such as started, text delta, tool status, completed, refused, and failed rather than forwarding raw provider chunks directly. The client renders partial output as provisional and commits the final result only after the terminal event and required validation. Disconnects propagate cancellation where possible, while the server still records the actual provider outcome.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
The protocol carries an operation ID, sequence numbers, heartbeats, and a clear end state so the UI can distinguish a pause from completion. Backpressure and output limits prevent slow clients from consuming unbounded memory. If moderation or semantic validation requires a complete answer, I buffer the relevant unit or use a safer non-streaming path instead of pretending post-display removal prevents exposure.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
I define replay and resume semantics deliberately: either the client reconnects to persisted events or starts a new idempotent operation, but it never guesses where text ended. Tool calls and structured output are not executed from partial fragments. Metrics separate time to first event, generation duration, client delivery time, cancellation, and incomplete streams. Provider-specific stream formats stay behind the adapter so a model upgrade cannot silently change the public event contract.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Sending provider chunks directly to the browser.
Executing a tool before its streamed arguments are complete.
Treating a socket close as successful completion.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

Unsafe content can be displayed before checks run.
The stream has no explicit terminal event.
A slow client creates unbounded server buffering.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

When would you buffer instead of stream?
How would reconnect work?
Which latency measurements matter?

Continue the section

Next in AI Application Architecture: Control model call lifetime

Step 9 of 18

How would you apply timeouts, retries, and cancellation to model calls?

Practice route

Read core answer Review checklist Follow-up prompts Track progress

Finished this drill?

Practice freely. Sign in only when you want to save completion, confidence, notes, or bookmarks.

New to Aporeon? Create account

Drill details

Human editorial direction

Aporeon is shaped by Aleksandar Tomovski, a software developer with experience on both sides of technical interviews. Content is reviewed for accuracy, natural spoken delivery, useful depth, and honest trade-offs.

About the creator Editorial standards

Was this drill useful?

Help improve the interview library.

Found this answer useful?

Say thanks with a standalone, one-time $5 contribution. No account required.

No subscription or perks.Choose another amount

Opening drill

Architecture scenarioHard~15 minAI Application Architecture / 8 of 18

Interview question

How would you design streaming responses for an AI feature?

Treats streams as event protocols with partial-output UX, cancellation, validation, moderation, and terminal outcome semantics.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

Full answer below

Practice focus

Streaming protocol design, partial failure, safety, backpressure, and reconnect behavior.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Streaming reduces time to first visible output but exposes partial content before the complete result is known.
I stream typed events such as started, text delta, tool status, completed, refused, and failed rather than forwarding raw provider chunks directly. The client renders partial output as provisional and commits the final result only after the terminal event and required validation. Disconnects propagate cancellation where possible, while the server still records the actual provider outcome.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
The protocol carries an operation ID, sequence numbers, heartbeats, and a clear end state so the UI can distinguish a pause from completion. Backpressure and output limits prevent slow clients from consuming unbounded memory. If moderation or semantic validation requires a complete answer, I buffer the relevant unit or use a safer non-streaming path instead of pretending post-display removal prevents exposure.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
I define replay and resume semantics deliberately: either the client reconnects to persisted events or starts a new idempotent operation, but it never guesses where text ended. Tool calls and structured output are not executed from partial fragments. Metrics separate time to first event, generation duration, client delivery time, cancellation, and incomplete streams. Provider-specific stream formats stay behind the adapter so a model upgrade cannot silently change the public event contract.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Sending provider chunks directly to the browser.
Executing a tool before its streamed arguments are complete.
Treating a socket close as successful completion.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

Unsafe content can be displayed before checks run.
The stream has no explicit terminal event.
A slow client creates unbounded server buffering.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

When would you buffer instead of stream?
How would reconnect work?
Which latency measurements matter?

Continue the section

Next in AI Application Architecture: Control model call lifetime

Step 9 of 18

How would you apply timeouts, retries, and cancellation to model calls?

How would you design streaming responses for an AI feature?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section

How would you design streaming responses for an AI feature?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section