Concept questionEasy~5 minAI Fundamentals / 2 of 20

Interview question

What are tokens, and why do they matter in an AI application?

Explains tokens as model-specific text units and connects tokenization to context limits, output limits, latency, cost, and text behavior.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

A token is a unit produced by a model's tokenizer. It may be a whole word, part of a word, punctuation, whitespace, or another text fragment. The model reads and generates token IDs, not raw sentences. Tokens matter because the request and response must fit the model's limits, and many providers price and rate-limit usage by token count. More tokens also usually mean more computation and latency. The exact count depends on the tokenizer, so character or word counts are only rough estimates.

Full answer below

Practice focus

The ability to reason about model input and output limits without pretending tokens are always words or universal across models.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Tokens are an application concern because models meter, limit, and process token sequences rather than the characters or words users see.
A token is a unit produced by a model's tokenizer. It may be a whole word, part of a word, punctuation, whitespace, or another text fragment. The model reads and generates token IDs, not raw sentences. Tokens matter because the request and response must fit the model's limits, and many providers price and rate-limit usage by token count. More tokens also usually mean more computation and latency. The exact count depends on the tokenizer, so character or word counts are only rough estimates.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
A short-looking input can tokenize poorly because of code, uncommon names, numbers, or a different language, while familiar English text may use fewer tokens. I count with the tokenizer or provider usage data when a limit matters. The budget includes more than the user's message: system instructions, conversation history, retrieved passages, tool results, and reserved output all compete for space. That makes token usage a product and architecture decision, not just a billing detail.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
I define separate budgets for fixed instructions, user input, retrieval, tool evidence, history, and completion, then observe their actual distributions. Hard truncation is dangerous because it can remove an instruction, citation, or recent turn without producing an obvious error. A robust service applies explicit priorities, rejects inputs it cannot handle safely, and records token counts by model and feature. I also avoid persisting sensitive raw prompts merely to measure usage; aggregate counts and appropriately redacted traces are often sufficient.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Saying one token always equals one word.
Counting only the user's visible message.
Assuming every model uses the same tokenizer.
Treating token cost as the only reason to control context size.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

The design silently truncates whatever exceeds the limit.
No output budget is reserved.
Token usage cannot be attributed to a feature, model, or request stage.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

What competes for space in a context window?
Why can code or multilingual text use surprising token counts?
How would you enforce a token budget?
What should happen when an input cannot fit safely?

Continue the section

Next in AI Fundamentals: The context window

Step 3 of 20

What goes into an LLM context window?

Practice route

Read core answer Review checklist Follow-up prompts Track progress

Finished this drill?

Practice freely. Sign in only when you want to save completion, confidence, notes, or bookmarks.

New to Aporeon? Create account

Drill details

Human editorial direction

Aporeon is shaped by Aleksandar Tomovski, a software developer with experience on both sides of technical interviews. Content is reviewed for accuracy, natural spoken delivery, useful depth, and honest trade-offs.

About the creator Editorial standards

Was this drill useful?

Help improve the interview library.

Found this answer useful?

Say thanks with a standalone, one-time $5 contribution. No account required.

No subscription or perks.Choose another amount

Opening drill

Concept questionEasy~5 minAI Fundamentals / 2 of 20

Interview question

What are tokens, and why do they matter in an AI application?

Explains tokens as model-specific text units and connects tokenization to context limits, output limits, latency, cost, and text behavior.

1Say itAnswer out loud first

2CompareCompare with Core answer

3RefineRefine with Stronger and Advanced

TL;DR

Full answer below

Practice focus

The ability to reason about model input and output limits without pretending tokens are always words or universal across models.

Answer first

Answer path

Core answer first, then checkpoints toward a stronger answer.

Start
Core answer
Start here. This is the direct spoken answer to practice first.
Overview
Tokens are an application concern because models meter, limit, and process token sequences rather than the characters or words users see.
A token is a unit produced by a model's tokenizer. It may be a whole word, part of a word, punctuation, whitespace, or another text fragment. The model reads and generates token IDs, not raw sentences. Tokens matter because the request and response must fit the model's limits, and many providers price and rate-limit usage by token count. More tokens also usually mean more computation and latency. The exact count depends on the tokenizer, so character or word counts are only rough estimates.
Checkpoint
Stronger answer
Optional upgrade with examples, trade-offs, and sharper phrasing.
A short-looking input can tokenize poorly because of code, uncommon names, numbers, or a different language, while familiar English text may use fewer tokens. I count with the tokenizer or provider usage data when a limit matters. The budget includes more than the user's message: system instructions, conversation history, retrieved passages, tool results, and reserved output all compete for space. That makes token usage a product and architecture decision, not just a billing detail.
Finish
Advanced answer
Optional deeper angle for production judgment and nuance.
I define separate budgets for fixed instructions, user input, retrieval, tool evidence, history, and completion, then observe their actual distributions. Hard truncation is dangerous because it can remove an instruction, citation, or recent turn without producing an obvious error. A robust service applies explicit priorities, rejects inputs it cannot handle safely, and records token counts by model and feature. I also avoid persisting sensitive raw prompts merely to measure usage; aggregate counts and appropriately redacted traces are often sufficient.

Review signals

Use these to catch weak phrasing, shallow reasoning, and risky claims.

Common mistakesCandidateCommon mistakesMistakes that weaken the answer or signal shallow understanding.Open details

Details

Saying one token always equals one word.
Counting only the user's visible message.
Assuming every model uses the same tokenizer.
Treating token cost as the only reason to control context size.

Red flagsCandidateRed flagsSignals that an interviewer may treat as risky.Open details

Details

The design silently truncates whatever exceeds the limit.
No output budget is reserved.
Token usage cannot be attributed to a feature, model, or request stage.

Follow-up prompts

Quick interviewer prompts inside this drill.

Follow-up questionsCandidateFollow-up questionsLikely next questions after the main answer.Open details

Details

Treat these as the next interviewer turn. Answer one out loud, then save the gap in your practice notes.

What competes for space in a context window?
Why can code or multilingual text use surprising token counts?
How would you enforce a token budget?
What should happen when an input cannot fit safely?

Continue the section

Next in AI Fundamentals: The context window

Step 3 of 20

What goes into an LLM context window?

What are tokens, and why do they matter in an AI application?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section

What are tokens, and why do they matter in an AI application?

Practice focus

Answer path

Overview

Review signals

Follow-up prompts

Continue the section