Agent Architecture

Core components, agent loop, state management, and orchestration — what makes an agent work.

🇷🇺 Russian version: architecture.ru.md

← AI agents · Patterns →

Contents

1. Agent components
2. Agent loop
3. State and memory
4. Context window
5. Orchestration
6. Whats next

1. Agent components

Any AI agent, regardless of complexity, consists of four mandatory components:

┌──────────────────────────────────────────────────────────┐
│                      AI AGENT                             │
│                                                          │
│  ┌──────────────┐  ┌──────────────┐  ┌────────────────┐  │
│  │     LLM      │  │   Tools      │  │  Orchestrator  │  │
│  │   (brain)    │  │   (hands)    │  │  (control)     │  │
│  └──────┬───────┘  └──────┬───────┘  └───────┬─────────┘  │
│         │                 │                   │           │
│         ▼                 ▼                   ▼           │
│  Makes decisions    Calls functions,     Manages loop,    │
│                     API, reads files,    state, memory    │
│                     searches the web                      │
│                                                          │
│  ┌──────────────────────────────────────────────────┐    │
│  │              Memory (state)                       │    │
│  │  Step history · Context · Results                │    │
│  └──────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────┘

1.1 LLM — decision making model

For local agents, the same models are used for chatting, but there are nuances:

Requirements:

• Tool calling — model must support function calling (Qwen 3.5+, Llama 3.1+)
• Structured output — model must return JSON by schema
• Long context — agent loop quickly fills context, need 32K+ tokens

1.2 Tools

Tools are functions the model can call. They extend the model’s capabilities beyond its knowledge.

Each tool is described by JSON Schema — the model “reads” the description and decides when to call it.

tool = {
    "type": "function",
    "function": {
        "name": "search_web",
        "description": "Search the web",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query"}
            },
            "required": ["query"]
        }
    }
}

1.3 Orchestrator (control loop)

The orchestrator is the code that manages the sequence: send request to model → get decision → execute tool → return result to model → repeat.

This is my job as Sisyphus. The orchestrator decides:

• When to hand control to a tool
• When to return response to user
• When to stop (step/time/token limit)
• What to do on error (retry, fallback, inform user)

1.4 Memory (State)

The agent must remember what it has done. Details in memory.md:

2. Agent loop

The fundamental cycle:

1. User sends request → system appends to history
2. LLM receives full history (system prompt + all previous steps)
3. LLM decides: answer text OR call a tool
4. If tool: system executes function → result added as observation → back to step 2
5. If answer: text returned to user

Basic cycle

  ┌──────────┐
  │  START   │
  └────┬─────┘
       │
       ▼
  ┌──────────┐     ┌─────────────┐
  │ LLM      │────▶│ Need a      │
  │ decides  │     │ tool?       │
  └──────────┘     └──────┬──────┘
       ▲                  │         ┌──────────┐
       │           ┌──────┤  YES ───▶  Call    │
       │           │      │         │ tool     │
       │           │      │         └────┬─────┘
       │           │      │              │
       │           │      │              ▼
       │           │      │         ┌──────────┐
       │           │      │         │ Result   │
       │           │      │         └────┬─────┘
       │           │      │              │
       │           │      └──────────────┘
       │           │
       │      ┌────┴────┐
       │      │   NO    │
       │      ▼         │
       │  ┌────────┐    │
       └──┤ANSWER  │◄───┘
          │ user   │
          └────────┘

Implementation in Python (no frameworks)

import requests, json

def agent_loop(model: str, user_input: str, tools: list, max_steps: int = 10):
    """Simple agent loop."""
    messages = [
        {"role": "system", "content": "You are a helpful assistant. Use tools when needed."},
        {"role": "user", "content": user_input}
    ]

    for step in range(max_steps):
        response = requests.post("http://localhost:11434/api/chat", json={
            "model": model,
            "messages": messages,
            "tools": tools,
            "stream": False
        })
        data = response.json()
        msg = data["message"]
        messages.append(msg)

        if msg.get("tool_calls"):
            for tc in msg["tool_calls"]:
                func_name = tc["function"]["name"]
                args = tc["function"]["arguments"]
                print(f"  Step {step+1}: calling {func_name}({args})")

                result = execute_tool(func_name, args)

                messages.append({
                    "role": "tool",
                    "name": func_name,
                    "content": json.dumps(result)
                })
        else:
            return msg["content"]

    return "Step limit reached"

This is literally the core of any agent. LangGraph, CrewAI, Agno all do the same thing internally.

Why go framework-free?

Writing the loop manually at least once is critically important because:

• You understand what happens “under the hood” of the framework
• You can debug when something goes wrong
• You are not dependent on abstractions that hide important details
• You can add your own logic (pauses, conditions, logging)

3. State and memory

Agent state is all information accumulated during its work.

agent_state = {
    "messages": [],
    "step": 0,
    "max_steps": 10,
    "tools_used": [],
    "errors": [],
    "intermediate_results": {},
    "plan": [],
    "user_intent": "",
}

How frameworks manage state

LangGraph — StateGraph with typed state:

class AgentState(TypedDict):
    messages: Annotated[list, add_messages]
    next_agent: str

CrewAI — each agent has internal state (role, goal, memory).

Agno — session_state for data between runs.

Common issues

4. Context window

On M1 16 GB with num_ctx: 4096 you hit the limit at 3-5 steps.

Solutions

1. Increase num_ctx — if it fits in RAM
2. Limit steps — max_steps=5 for most tasks
3. Context compression — summarize when too long
4. KV cache quantization — OLLAMA_KV_CACHE_TYPE=q4_0 gives 4× more space

def compress_if_needed(messages, max_messages=20):
    if len(messages) > max_messages:
        system = messages[0]
        recent = messages[-(max_messages-1):]
        return [system] + recent
    return messages

5. Orchestration

This is what I do as Sisyphus. One agent is just a loop. Multiple agents working together is orchestration.

┌─────────────────────────────────────────────────────┐
│                   ORCHESTRATOR                       │
│                                                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────────────┐   │
│  │ Agent 1  │  │ Agent 2  │  │    Agent 3       │   │
│  │ PM       │  │ Analyst  │  │  Developer       │   │
│  └────┬─────┘  └────┬─────┘  └───────┬──────────┘   │
│       │              │                │              │
│       └──────────────┴────────────────┘              │
│                      │                               │
│               Solve tasks,                            │
│               exchange results                        │
└─────────────────────────────────────────────────────┘

What the orchestrator decides:

• Which agent takes the task
• Order of work (sequential / parallel / hierarchical)
• How agents exchange data
• What to do if an agent fails
• When the task is complete

def orchestrate(agents: list, task: str):
    result = None
    for agent in agents:
        context = f"Previous result: {result}" if result else ""
        result = agent.run(f"{task}\n{context}")
    return result

Real orchestration is more complex — see multi-agent.md and 02-agent-team tutorial.

6. Whats next

| If you want | Go to | |————-|——-| | Study specific agent patterns (with code) | patterns.md | | How agents store information | memory.md | | How it works with frameworks | frameworks.md | | Write your first agent | tutorials/01-first-agent.md | | Back to navigation | README.md | —

In section: architecture · evaluation · frameworks · memory · multi-agent · ollama-for-agents · orchestrators · patterns · prompting · ready-made · safety · skills
Related sections: Zero Level · Local Models · Use Cases · Resources
Navigation: ← AI Agents · ↑ Back to main · 🇷🇺 Русский