1M context — what actually changes
Anthropic's new Claude Opus 4.7 accepts up to one million tokens in a single API call. A whole book, an entire long conversation, a mid-sized codebase — all of it fits in context now. You can skip the retrieval step and "just ask everything."
Cost moves. The per-token input price didn't change, but pushing 1M tokens every call can hit double-digit dollars per request. The lever is prompt caching: send the same prefix within five minutes and the cached portion costs 90% less. Cache hit rate is now an operational metric on par with latency.
Is RAG dead? No. Workloads that need a different large context every call still win with RAG. 1M context shines when (1) you re-use a context many times (codebase analysis) or (2) you bundle several round-trips inside the 5-minute TTL. Recompute the break-even.
Take-aways
- 1M context doesn't kill RAG. The break-even differs by workload pattern.
- If you don't measure prompt-cache hit rate, you don't know your cost. Add it as a KPI.
- The 5-minute TTL is a new design constraint — group your round-trips inside it.
→ 한국어 버전 →
