AI Evaluation

An AI feature became slower and more expensive. How would you diagnose it?

Diagnoses an AI regression by decomposing request stages, configuration changes, token growth, retries, routing, tools, retrieval, caching, and provider behavior.

How do temperature and sampling settings affect model output?

Explains how decoding settings reshape token selection and why they must be tuned against task-specific evaluation rather than folklore.

How do you balance quality, latency, and cost in an AI feature?

Balances AI quality, tail latency, and cost through explicit product thresholds, model choice, context control, caching, routing, and measurement.

How would you choose a model for a product feature?

Chooses a model from task-specific evaluation, operating constraints, safety, latency, cost, context, modality, and provider requirements.

How would you design a safe first LLM feature for an existing product?

Designs a bounded and reversible first AI feature with measurable value, controlled data, validation, evaluation, observability, fallback, and human oversight.

How would you roll out a model or model-version change safely?

Treats model changes as controlled releases using versioned configuration, fixed evaluations, shadow or canary traffic, monitoring, and rollback.

AI Engineering

Applied AI interview preparation grounded in real product and production decisions.

An applied AI engineering path covering model behavior, tokens, context, output controls, embeddings, model selection, reliability boundaries, and production diagnosis.