How should an application handle refusals, timeouts, rate limits, and malformed model responses?

Separates model and provider outcomes, applies bounded recovery, validates outputs, and exposes useful status and telemetry.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

I classify the outcome first. A refusal is a model decision and may require safer context, a narrower task, or a user explanation; it is not automatically a transient error. A timeout may be retried only when the operation is safe and the deadline allows it. A rate limit uses bounded backoff, provider guidance, capacity controls, or a fallback. Malformed output fails schema and semantic validation and may get one constrained repair attempt. Every path has a user-visible result instead of returning invented content.

Full answer below

Practice focus

Failure-aware AI integration design that does not collapse every non-answer into a blind retry or generic server error.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
These outcomes have different causes and recovery paths, so the integration needs an explicit result contract rather than a single exception bucket.
I classify the outcome first. A refusal is a model decision and may require safer context, a narrower task, or a user explanation; it is not automatically a transient error. A timeout may be retried only when the operation is safe and the deadline allows it. A rate limit uses bounded backoff, provider guidance, capacity controls, or a fallback. Malformed output fails schema and semantic validation and may get one constrained repair attempt. Every path has a user-visible result instead of returning invented content.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
The adapter normalizes provider-specific responses into application outcomes such as completed, refused, invalid, throttled, timed out, and unavailable. Retries use jitter and a total attempt and time budget. A circuit breaker or admission limit can protect the provider and our own queue. If streaming has already exposed text, a retry cannot pretend the response never started; the UI needs a partial or interrupted state.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Retrying refusals as if they were network errors.
Parsing free-form text without validation.
Using unbounded retries or exponential backoff without a total deadline.
Assuming a fallback model behaves identically.

How should an application handle refusals, timeouts, rate limits, and malformed model responses?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue practice

Next in path

Tech stacks

Skill areas

Human editorial direction

Was this drill useful?

Found this answer useful?