Week 1: Foundations
Day 4: Tokens and Context Windows
Goal
Learn what text models actually read and why long conversations can lose details.
Learn
Language models process text as tokens: small chunks of words or characters. A context window is the amount of information the model can consider at one time. If a conversation, document, or codebase is larger than the window, the system must summarize, retrieve, or leave something out. Long context helps, but it is not the same as perfect memory.
Behind the scenes
AI tools are products wrapped around models, data, prompts, retrieval, safety systems, and user interfaces. The better you understand the wrapper, the better your results get.Example
If a conversation gets too long, an AI may miss an early detail unless the app summarizes or retrieves it again.
Practice
Paste a short paragraph into a chatbot and ask it to estimate token count, then ask what information would matter if the paragraph were ten times longer.
Checkpoint
Checkpoint
You can explain tokens, context, and why concise context improves answers.