How would you define success for an AI feature before choosing metrics?

I create a small measurement contract that maps each product claim to an observable result, evaluator, dataset slice, and owner. Leading indicators such as tool-call validity are distinguished from outcomes such as successful resolution, because optimizing the proxy alone can make the experience worse. Thresholds account for uncertainty, baseline performance, and the cost of false positives versus false negatives. I revisit the contract when users, policy, models, or workflows change so yesterday's metric does not silently govern a different product.