What does the model actually remember?
AI memory, context windows and long conversations
A long conversation is not a perfect archive. Separate what the product stores, what the model can see for this response and what the next stage genuinely needs.
Four things commonly called memory
| Type | Meaning | Failure mode |
|---|---|---|
| Chat history | messages and results in the current thread | not every part necessarily receives equal attention |
| Context window | the tokens the model can process in one call | large enough does not mean well organised |
| Saved memory | preferences or facts a product may retrieve across chats | product-specific, editable and not ground truth |
| Project knowledge | files and standing instructions attached to a workspace | may become stale or conflict with the current task |
When should you continue?
Stay in the same conversation when the goal, material and decisions belong to one task. This preserves local decisions and avoids needless repetition.
When should you start fresh?
- The subject or audience changes substantially.
- Old instructions keep influencing new answers.
- You want an independent comparison without earlier framing.
- The thread has become expensive, slow or full of obsolete intermediate work.
Create a controlled handoff
Create a handoff for a new conversation.
Include only:
- goal and completion criteria
- confirmed decisions
- sources or files that still apply
- open questions and known errors
- the next concrete step
Separate facts from assumptions. Exclude discarded ideas and the full transcript.Continue with context engineering and the cost of long conversations.
Sources
- OpenAI: Conversation state — conversation state, context windows and compaction
- OpenAI: Compaction — continuing long-running work
- Anthropic: Effective context engineering — selecting and maintaining context