LLM API Pricing Explained: Input, Output & Reasoning Tokens
API pricing pages display rates "per 1 million tokens", but why is output generation so much more expensive than input ingestion? And why do reasoning models like o3 generate unexpectedly high invoices?
1. Why Output Tokens Cost 3x to 5x More Than Input Tokens
The asymmetry between input and output pricing reflects fundamental GPU hardware mechanics:
- Input Prefill is Parallel: When you provide a 4,000-token prompt, modern GPUs process all 4,000 tokens concurrently in parallel matrix multiplications across thousands of Tensor Cores. Prefill is compute-bound and highly hardware-efficient.
- Output Generation is Sequential: Generation is autoregressive. To emit token #101, the model must wait for token #100. Each output token requires loading all model weights from GPU memory to the compute units, making generation strictly memory-bandwidth bound.
2. The Reasoning Token Trap
Reasoning models (like OpenAI o3, o3-pro, and Claude extended thinking) do not just generate an answer. They generate an invisible chain-of-thought deliberation log that can span 1,000 to 16,000 tokens before emitting the first user-facing word.
These hidden reasoning tokens are billed as output tokens at full output rates ($60.00/1M on o3). A single query with 5,000 hidden reasoning tokens costs $0.30 just for the internal thinking phase!
3. The Blended 3:1 Industry Standard
To compare models fairly, engineers use a blended cost metric assuming an average prompt-to-completion ratio of 3:1 (3 input tokens for every 1 output token):
For Claude Sonnet 5 ($3.00 in / $15.00 out), the blended rate is $6.00 / 1M tokens.