Skip to content
Learn AI
বাংলা
Part 3 13 min read

Context Engineering

the new "prompt engineering"

A model can read only so much at once. Learn seven ways to use that space well: context budgeting, summarization, caching, memory, compression, trimming tool results, and avoiding the "lost in the middle" problem.

Go to quiz
On this page

Section 1 of 7

Context window budgeting

A model has a fixed-size working memory. Every byte you allocate competes with every other byte. Context engineering is the discipline of deciding what lives where — system prompt, history, tool definitions, retrieved docs, prior outputs, the user's actual request.

The budget model

Treat the context window as a finite pie. Five claimants, each with a minimum allocation and a cost-per-token that varies:

budgeting a 200K-token window — illustrative allocations SYSTEM TOOLS RETRIEVAL HISTORY USER+OUTPUT SYSTEM — persona, rules, schema ≈ 8% · 16K tokens TOOLS — schemas, definitions ≈ 7% · 14K tokens RETRIEVAL — RAG chunks ≈ 40% · 80K tokens HISTORY — prior turns ≈ 30% · 60K tokens USER + OUTPUT — current turn ≈ 15% · 30K tokens retrieval usually dominates; reduce it first. system + tools are the floor.
A 200K-token window broken into system + tools + retrieval + history + current turn. Percentages are typical for an agentic RAG app, not universal.
এই ডায়াগ্রামে একটি 200K-টোকেন কনটেক্সট উইন্ডোকে SYSTEM (≈৮%), TOOLS (≈৭%), RETRIEVAL (≈৪০%), HISTORY (≈৩০%) এবং USER+OUTPUT (≈১৫%) অংশে ভাগ করে দেখানো হয়েছে। প্রতিটি অংশের প্রস্থ তার টোকেন বাজেটের সমান; RETRIEVAL সবচেয়ে বড়, তাই আগে সেটাই কমানো উচিত।

Allocation principles

  • → System prompt: floor of ~1–2K tokens. Persona + schema + hard constraints. Larger prompts cost more per call AND dilute attention.[14]
  • → Tool definitions: floor of ~1–3K tokens. Only include tools the current step might call; load specialised tools lazily.
  • → Retrieval: budget = window − system − tools − current turn − reserved output. Cap retrieval chunks; prefer quality over quantity.
  • → History: shrink aggressively. Use summarization (§2) or selective replay to keep it bounded.
  • → Output reservation: always reserve enough tokens for the response. Forcing truncation produces silent failures.
# A budget calculator
def budget(window: int, tools: list[dict], retrieved: str,
          system: str, history: str, max_out: int) -> dict:
    return {
        "system":      len(enc.encode(system)),
        "tools":       sum(len(enc.encode(json.dumps(t))) for t in tools),
        "retrieval":   len(enc.encode(retrieved)),
        "history":     len(enc.encode(history)),
        "max_output":  max_out,
        "total":       sum((len(enc.encode(x)) for x in [system, history, retrieved])) + max_out,
        "headroom":    window - "total",
    }

The "Lost in the Middle" effect. Liu et al. (2023) showed that models reliably recall information at the beginning and end of long contexts but perform close to random in the middle.[1] This is the single most important budget heuristic: put the things you most need retrieved in the start or the end of your prompt, not buried in the middle. (§7 expands this.)

Practice · 1 task

Open any free AI chat: ChatGPT · Claude · Gemini

How practice works
  • A new chat is a fresh conversation with no history. In ChatGPT, Claude, or Gemini, click New chat. It is usually at the top left.
  • Same chat means: keep typing in the chat that is already open. The AI remembers what you sent before.
  • To use a prompt: click Copy, click in the message box of the chat, paste it (Ctrl+V, or press and hold on a phone), and press Enter.
  • AI answers are a little different every time. Your answer does not need to match the example word for word. Look for the same idea.
  • Use made-up data only. Do not paste private information into a chat.

Task 1

Fit a plan into an 8,000-token window

Why this matters

When you build a chat app that searches documents, the AI can read only a fixed amount of text at once. You must check that everything fits, or the answer gets cut off.

  1. Step 1
    New chat

    Copy this prompt, paste it, and press Enter. The AI will add up the numbers.

    Prompt
    A chat app has a context window of 8,000 tokens.
Fixed costs:
- System prompt: 1,500 tokens
- Tool definitions: 2,000 tokens
- Current user turn: 300 tokens
- Reserved for the answer: 1,000 tokens
The team wants to add:
- Chat history: 2,500 tokens
- Retrieved chunks (small pieces of documents found by search): 6 chunks of 700 tokens each

Task:
1. Show a table with every item and the total.
2. Say if the plan fits. Say how many tokens it is over or under.
3. If history stays at 2,500 tokens, how many 700-token chunks fit?
Show your arithmetic.

    You should see: A table with a total of 11,500 tokens. The plan does not fit: it is 3,500 tokens over. Only 1 chunk fits.

    Example answer
    Item | Tokens
System prompt | 1,500
Tool definitions | 2,000
Current user turn | 300
Reserved for answer | 1,000
Chat history | 2,500
6 chunks x 700 | 4,200
Total | 11,500

The plan does not fit. It is 11,500 - 8,000 = 3,500 tokens over.
Without chunks the total is 7,300, so 700 tokens are free. Only 1 chunk of 700 tokens fits.

    Your answer will be worded differently.

  2. Step 2
    Same chat

    On paper, add the six numbers: 1,500 + 2,000 + 300 + 1,000 + 2,500 + 4,200. Compare your total with the total of the AI.

    You should see: Your total is 11,500. If the AI gave a different total, the AI made a mistake.

  3. Step 3
    Same chat

    Copy this prompt, paste it, and press Enter. The AI will make a smaller plan that fits.

    Prompt
    Now make a plan that fits in 8,000 tokens. Rules: keep the 1,000-token answer reserve, keep 3 chunks of 700 tokens each, and load only 1,000 tokens of tool definitions now (the rest load later, only when needed). Keep the system prompt (1,500) and the current user turn (300). Shrink the history by summarizing it. Give a new table with the largest history that fits, and how many tokens are left over.

    You should see: A new table. The largest history is 2,100 tokens, and about 0 tokens are left over.

    Example answer
    Item | Tokens
System prompt | 1,500
Tool definitions (loaded now) | 1,000
Current user turn | 300
Reserved for answer | 1,000
3 chunks x 700 | 2,100
Summarized history | 2,100
Total | 8,000

The largest history that fits is 2,100 tokens. Tokens left over: 0.

    Your answer will be worded differently.

  4. Step 4
    Same chat

    Check the new plan on paper: 1,500 + 1,000 + 300 + 1,000 + 2,100. Then add the history size of the AI. The total must be 8,000 or less.

    You should see: The first sum is 5,900. With a history of 2,100 the total is 8,000. If the AI chose another history size, the total must still be 8,000 or less.

What just happened

The context window is the most text the AI can read at one time, and the answer must fit inside it too. In the first plan, the fixed parts plus the documents were bigger than the window. To fix it, you had to cut several parts at once: a smaller history, fewer tools, and fewer chunks. Always keep room for the answer, or it gets cut off.

You are done when You have the total of the first plan (11,500) and a new plan that adds up to 8,000 or less.

Check yourselfWhy must you keep room for the answer when you plan what fits?Show answer

The window holds the input and the answer together. If you keep no room for the answer, the answer gets cut off, and nothing warns you.

Try a harder version

Change the window in the first prompt to 32,000 tokens and send it again. Does the plan fit now? Which part would you still limit first, and why?

Watch · 2 videos