Agents and Agentic Systems

The minimal agent

Strip away the marketing and every agent is the same shape: a loop that takes the current state, asks an LLM what to do next, executes the answer (a tool call, a structured response, or a final answer), and feeds the result back in. Where agents diverge from each other is how the loop is structured, what the state contains, and how failure is handled.

# The minimal agent in 12 lines
def agent_loop(goal, llm, tools, max_steps=20):
    history = [{"role": "user", "content": goal}]
    for step in range(max_steps):
        response = llm.complete(history, tools=tools)
        if response.finish_reason == "stop":
            return response.text
        result = tools.invoke(response.tool_call)
        history.append({"role": "tool", "content": str(result)})
    raise AgentTimeout("loop did not terminate")

Agent vs workflow

A workflow is a deterministic graph of LLM calls and tool calls: every step is authored, every branch is explicit. An agent is a workflow where at least one node's behaviour is decided at runtime by the LLM itself. The line is blurry in practice — most production systems are "agentic workflows" that combine both.

What every production agent needs

• A bounded loop with a hard step cap and a wall-clock timeout. Unbounded loops are how agents burn $10k overnight.
• A tool surface that's narrow, well-typed, and namespaced (so the model can't accidentally call a destructive tool without going through a guardrail).
• A scratchpad — some persistent state across turns that survives the LLM's context window. Either the model writes to a file, the framework keeps it, or you stuff it into the next prompt.
• An exit predicate: at minimum, "I think I have enough; here is the answer." Without one the agent will keep acting forever.
• Observability: every step recorded with prompt, tool call, result, cost, and latency. Without it you cannot debug.

ReAct — Reason + Act (interleaved)

Introduced by Yao et al. in 2022,[1] ReAct is the simplest agent loop: the model writes a Thought, picks an Action, observes the result, repeats. The thought is in the model's private scratchpad and never shown to the user; only the final answer is. ReAct is the default loop in LangChain's AgentExecutor and most early agent frameworks.

# ReAct scratchpad format (text version)
Thought: I need to find the population of Tokyo in 2024.
Action:   search(query="Tokyo population 2024 official")
Observation: As of January 2024, the estimated population...
Thought: Now I can answer.
Action:   Finish("Tokyo's population is ~13.96 million.")

Plan-and-Execute

Plan-and-Execute separates planning from execution: a planner (usually a strong LLM) produces a sequence of steps, then a worker (often a smaller model or a deterministic tool) carries out each step. The worker can be ReAct itself, a single-shot LLM call, or a pure function. If a step fails, the planner re-runs only that step (cheap) rather than the whole loop.[2]

Reflexion — verbal self-criticism

Reflexion adds a second pass: after a ReAct attempt, the model is asked to reflect on what went wrong, and that reflection is appended to the memory for the next attempt.[3] In benchmarks (HumanEval, AlfWorld) Reflexion beats vanilla ReAct by 10–20% absolute.

ReWOO — Reasoning Without Observation

ReWOO's insight: in many retrieval-heavy tasks, the model doesn't need to see the result of tool #1 to plan tool #2. It plans the entire tool graph upfront, the worker executes everything in parallel, then the solver reads the evidence and answers.[4] ReWOO uses 4–6× fewer tokens than ReAct on retrieval benchmarks.

Which loop to pick

What a single agent gives you

A single agent is one LLM in one control loop with one tool surface. Every decision flows through one prompt, one model call, one trace. Debugging is "read the trace."

What a multi-agent setup gives you

Multi-agent systems have an orchestrator that delegates to specialised workers — researcher, writer, critic, coder — each with its own prompt, its own tool surface, and (often) its own model.[5]

Communication patterns between agents

• Shared scratchpad (blackboard): all agents read/write to a shared file or in-memory dict.
• Message passing: orchestrator sends typed messages to workers.
• Tool calls: the orchestrator is the only "agent"; workers are exposed as tools.

# Anthropic Research pattern (simplified)
async def research(query):
    sub_queries = await orchestrator.plan(query)
    findings = await asyncio.gather(*[
        researcher.search(sq) for sq in sub_queries
    ])
    review = await critic.review(query, findings)
    return await orchestrator.synthesise(query, findings, review)

Why an explicit graph

An implicit loop is fast to write and impossible to debug. After 30 tool calls, you can't tell which path the model took. An explicit graph makes the topology visible: every node has a name, every edge has a condition, every transition is recorded.

LangGraph

LangGraph (LangChain's graph framework) is the most popular Python/TypeScript option.[6] It models the agent as a state graph: nodes are functions, edges are conditional, the state is a TypedDict that flows through. Cycles are first-class (an agent can loop back to an earlier node), and checkpointer lets you resume from any past state.

# LangGraph minimal agent
from langgraph.graph import StateGraph, END
from typing import TypedDict

class State(TypedDict):
    messages: list
    step: int

def call_llm(state: State) -> State:
    response = llm.invoke(state["messages"])
    return {"messages": state["messages"] + [response], "step": state["step"] + 1}

def should_continue(state: State) -> str:
    last = state["messages"][-1]
    if hasattr(last, "tool_calls") and last.tool_calls:
        return "tool"
    return END

graph = StateGraph(State)
graph.add_node("llm", call_llm)
graph.add_conditional_edges("llm", should_continue, {"tool": "llm", END: END})
graph.set_entry_point("llm")
app = graph.compile()

Temporal, Restate, Mastra, Inngest

For long-running agents (minutes to hours), the graph needs to be durable — survive crashes, retries, and human-in-the-loop pauses. Temporal and Restate are workflow engines from the microservices world adapted to agents.[7] Mastra is TypeScript-first with built-in workflow engine, memory, and tool definitions. Inngest is event-driven serverless workflows with pay-per-step pricing.

Picking a graph framework

Scratchpad (working memory)

The scratchpad is the model's private notes for the current run. When the run ends, the scratchpad is discarded.

Semantic memory (long-term facts)

Semantic memory stores facts the agent has learned about the user, the project, the world. The classic implementation is a vector store indexed by meaning, plus a structured row store for exact matches.

Procedural memory (skills)

Procedural memory is the agent's playbook: "how to do X." Stored as versioned prompt templates or Markdown checklists.

Other memory types

Where memory breaks

• Too much memory: the agent over-relies on stale facts.
• Too little memory: the agent re-derives the same conclusion on every turn.
• Conflicting memory: two facts disagree.

The risk ladder

1. Read-only: search, fetch, query. Auto-approve.
2. Local-write: edit a file the agent owns. Auto-approve with a soft cap.
3. External-write: send an email, post to Slack. Approve per call.
4. Money: refund, charge, payout. Approve per call, plus a daily cap.
5. Destructive: delete a record. Two-person rule.

The checkpoint primitive

Every durable framework exposes a "wait for human" primitive. In LangGraph it's interrupt; in Temporal it's wait_for_signal; in Restate it's await call().ack().

# LangGraph interrupt + resume
def send_email(state):
    approval = interrupt({
        "to": state["recipient"],
        "subject": state["subject"],
        "body": state["draft"],
    })
    if not approval["approved"]:
        return {"messages": ["User rejected email send."]}
    email_client.send(approval)
    return {"messages": ["Email sent."]}

Asynchronous approvals

When the human can't respond synchronously:

• Soft timeout + default reject: if no response in 24 hours, the workflow terminates.
• Soft timeout + default approve with rollback: if no response, the action proceeds but is logged and reversible.
• Hard timeout + escalate: at 24 hours, route to a backup approver.

What a subagent is

A subagent is an agent launched by another agent. It has its own system prompt, its own tool surface, its own loop, and its own termination. It returns a final result to the parent — not intermediate state.

Manager–worker, not fork/join

The mental model is "manager → worker," not "fork/join." The parent does not poll the subagent. The parent does not stream the subagent's progress. The parent waits for the subagent to finish, then receives a single, structured result. The benefit: the parent's context window stays clean.

Claude Code's Task tool

Anthropic's Claude Code ships a Task tool that the model can invoke to spawn a subagent. The subagent runs in its own loop, with its own tools, and returns a summary. The parent never sees the intermediate steps — only the summary. This is the cleanest implementation of the pattern in 2025.

OpenAI's agents-sdk and CrewAI

OpenAI's openai-agents-sdk exposes Runner.run(agent, input) returning a final RunResult. CrewAI models multi-agent systems as crews (the team), agents (the roles), tasks (the work), and a process (sequential, hierarchical, or consensual).

When to use a subagent

• The subtask is well-defined and self-contained.
• The subtask needs different tools than the parent has.
• The subtask's intermediate steps would pollute the parent's context.
• You want the subtask to be retried independently of the parent.
• You want to bill the subtask to a different cost centre.

Layer 1: retry

Transient failures (rate limit, network blip, server overloaded) are best handled with a short retry. Exponential backoff is the default: 1s, 2s, 4s, 8s, with jitter to avoid thundering-herd. Cap at 3–5 attempts; beyond that, the failure is not transient.

# Exponential backoff with jitter
import random, time
def retry_with_backoff(fn, max_attempts=5, base=1.0):
    for attempt in range(max_attempts):
        try:
            return fn()
        except (RateLimitError, TimeoutError):
            if attempt == max_attempts - 1: raise
            sleep = base * (2 ** attempt) + random.uniform(0, base)
            time.sleep(sleep)

Layer 2: fallback

When the failure is not transient (the API is permanently down, the JSON schema is rejected because the model can't follow it, the function raised a domain-specific error), retrying the same call is pointless. Fall back to a different tool or model: use a smaller model with a simpler schema, use a different vendor, or skip the tool and let the model answer from its own knowledge.

Layer 3: reflect

When the failure was caused by a reasoning error (the model called the right tool with the wrong arguments, or made a logical mistake), neither retry nor fallback fixes the root cause. Feed the failure back into the model and ask it to revise.[3] Reflexion is the formal version of this.

Always bound retries

Circuit breakers

A circuit breaker protects against a downstream service that's failing in a way that looks transient but isn't. After 5 consecutive failures of the same kind, the breaker opens: all calls for the next 60 seconds fail fast with a "service unavailable" error, no retries. After 60 seconds, the breaker half-opens: one test call. If it succeeds, the breaker closes. If it fails, the timer resets.

Why "just save state" isn't enough

The naive solution is to checkpoint the agent's state to disk every step. But "the agent's state" is more than the message log: it's the in-flight tool calls, the partial writes to external services, the human approval state, the retry counters. Saving the messages and losing the rest gives you a corrupt resume.

Durable execution, defined

A durable execution engine records every step's intent and result to a durable store (a database, a journal, a queue). On crash, the next process loads the snapshot and resumes from the next un-run step. The engine handles retries, rate limits, and human-in-the-loop pauses natively. The application code is a series of workflow steps; the engine is the runtime that survives crashes.[7]

When you need it

• The agent runs for more than ~30 seconds (anything that might survive a deploy).
• Humans are in the loop and might take hours or days to respond.
• The agent makes irreversible side effects (sending money, deleting records) that you want to be able to retry without duplicating.
• The agent's work spans multiple services and you want to resume from the failed service.

The four durable engines

How computer use works

The model receives a screenshot of the desktop (or a specific window) and a list of available actions: mouse_move, left_click, right_click, type, key, screenshot. It picks an action, the runtime executes it, captures a new screenshot, and feeds both back. The loop continues until the model emits a done action.[9]

# Anthropic computer-use minimal loop (pseudocode)
while not done:
    screenshot = capture()
    response = client.messages.create(
        model="claude-sonnet-4-5",
        tools=["computer"],
        messages=[{"role": "user",
                   "content": [screenshot, goal]}],
    )
    for block in response.content:
        if block.type == "tool_use":
            execute(block.input)
    if response.stop_reason in ("end_turn", "max_tokens"):
        break

Where computer use wins

• No API exists. The site or app has no programmatic interface; the only way in is the GUI.
• The API is hostile. The vendor's API is rate-limited, expensive, or has bad docs; the GUI is faster.
• One-off tasks. You need to do this once, not at scale, so the cost-per-task of computer use is acceptable.
• Cross-app workflows. The task crosses SaaS boundaries (read from Gmail, post to Slack) and no integration exists.

Where computer use loses

• Cost. Computer-use calls are 5–30× the cost of an API call (vision tokens for every screenshot).
• Latency. A click is instant; a screenshot → LLM → action cycle is 1–5 seconds. A 20-step task takes a minute.
• Reliability. A shifted button, a popup, a captcha — any of these kills the run.
• Security. The agent has unrestricted mouse/keyboard. Pair with a sandboxed VM, a per-task browser profile, and a tight human-in-the-loop on destructive actions.

Browser agents vs computer use

Browser agents (ChatGPT's "Browse with Bing," Perplexity, Arc Max) are a constrained subset of computer use: the model can only see and act inside a single browser tab, not the whole desktop. They use the same screenshot → action loop but with a smaller surface area, which is why they're more reliable. If you can frame the task in a browser, prefer a browser agent over a full computer-use agent.

The reliability pattern

Production computer-use agents use three reliability patterns: (1) the model emits a structured plan up front ("I will click the blue button, then fill the form, then submit"), which makes mid-run debugging easier; (2) each step is checkpointed, so a crash resumes from the last successful step; (3) the agent runs in an isolated VM or browser profile, so a misclick can't reach the host system.