What are tokens, and why do they matter in an AI application?

I define separate budgets for fixed instructions, user input, retrieval, tool evidence, history, and completion, then observe their actual distributions. Hard truncation is dangerous because it can remove an instruction, citation, or recent turn without producing an obvious error. A robust service applies explicit priorities, rejects inputs it cannot handle safely, and records token counts by model and feature. I also avoid persisting sensitive raw prompts merely to measure usage; aggregate counts and appropriately redacted traces are often sufficient.