AI Evaluation

A model dependency is causing cascading failures. How would you diagnose and contain it?

Uses traces and saturation evidence to locate retry amplification, queue growth, exhausted workers, and failing fallback paths.

An agent keeps repeating tools without making progress. How would you diagnose it?

Uses traces to separate unclear goals, bad tool interfaces, missing state, contradictory results, poor termination, and retry amplification.

An AI feature became slower and more expensive. How would you diagnose it?

Diagnoses an AI regression by decomposing request stages, configuration changes, token growth, retries, routing, tools, retrieval, caching, and provider behavior.

An AI feature's quality regressed in production. How would you diagnose it?

Reproduces failing cases, compares versioned traces, isolates the first changed component, mitigates impact, and adds regression coverage.

A RAG system missed the right document. How would you diagnose it?

Traces a missed answer through source ingestion, parsing, chunking, filtering, query construction, retrieval, fusion, and ranking.

Build an offline AI evaluation harness with slice-based release gates

Builds a reproducible evaluation harness that records immutable cases, versioned outputs, deterministic checks, calibrated graders, slices, and release decisions.

AI Engineering

Applied AI interview preparation grounded in real product and production decisions.

A practical AI engineering path covering model fundamentals, structured output, retrieval, evaluation, application architecture, managed cloud platforms, agents, security, and production diagnosis.