How to Build an AI Agent from Scratch in Python

Building an AI agent from scratch in Python means writing the orchestration loop yourself rather than relying on a framework like LangChain or AutoGen to handle it. This gives you full control over how the agent decides what to do, how it calls tools, how it handles errors, and when it stops. For production agents where you need to understand exactly what is happening and why, a hand-built agent is often more reliable and easier to debug than a framework-based one. This guide walks through the core components — the planning loop, tool definitions, tool execution, and conversation memory — building up to a working agent you can extend for your own use cases.

What an AI Agent Actually Is

An AI agent, at its core, is a loop. The loop works like this: give the language model a goal and a set of available tools, ask it what to do next, execute whatever it decides, feed the result back, and repeat until the goal is achieved or the model decides it is done. The model is the brain — it reasons about the current state and decides which tool to call with which inputs. Your code is the body — it actually executes the tool calls and manages the conversation history that the model uses to reason.

The three key components you need to build:

  • Tool definitions — descriptions of what each tool does and what parameters it accepts, formatted so the model can understand and use them
  • Tool executor — the code that actually runs a tool when the model decides to call it
  • The agent loop — the orchestration code that sends messages to the model, receives tool call decisions, executes them, and continues until done

Defining Tools

Tools are described to the model as structured JSON schemas — a name, a description of what the tool does, and the parameters it accepts with their types and descriptions. The quality of the tool description directly affects how reliably the model uses the tool correctly. Vague descriptions produce incorrect tool calls; precise descriptions with clear parameter explanations produce accurate ones.

A well-written tool definition for a web search tool would specify: what the tool does (searches the web and returns relevant results), what the query parameter is (a search query string, specific rather than vague), and what the tool returns (a list of search results with titles, URLs, and snippets). The more precisely you describe the tool’s purpose and parameters, the less often the model will call it with wrong inputs or use it in situations where a different tool would be more appropriate.

Figure 1 — The AI agent loop: plan, act, observe, repeat

1. Plan LLM decides next tool 2. Act Your code runs tool 3. Observe Result added to history 4. Decide Done or loop again? ← repeat until goal achieved or max steps reached

Building the Tool Executor

The tool executor is a dispatcher — it receives a tool name and arguments from the model and routes the call to the appropriate Python function. The simplest implementation is a dictionary mapping tool names to functions. When the model returns a tool call, look up the function by name, call it with the provided arguments, and return the result.

Error handling in the executor matters significantly for agent reliability. If a tool call fails — network error, invalid input, API rate limit — the executor should catch the exception and return a structured error message rather than raising an exception that crashes the loop. The model can then read the error, understand what went wrong, and either retry with different parameters or try a different approach. Agents that crash on tool errors are frustrating to use; agents that handle errors gracefully and let the model adapt are substantially more robust.

Managing Conversation History

The conversation history is what the model uses to reason about what has happened so far and what to do next. It is a list of messages in the standard chat format: user messages containing the original goal and tool results, assistant messages containing the model’s reasoning and tool calls. Building and maintaining this list correctly is the most important implementation detail in a hand-built agent.

The history grows with each loop iteration: the model’s tool call decision gets added as an assistant message, the tool result gets added as a tool result message, and both are present for the model’s next reasoning step. Truncating or summarising old history when it grows long is necessary for agents that run many steps — context windows are finite and very long histories degrade model performance as the model struggles to attend to relevant information buried in distant context.

The Agent Loop

The agent loop ties everything together. It starts by building the initial messages — a system prompt describing the agent’s role and the available tools, and a user message containing the goal. It then enters a loop: call the model, check whether the response contains a tool call or a final answer, execute the tool if called, add the result to history, and repeat. The loop exits when the model produces a response with no tool call (indicating it has reached a final answer) or when a maximum step count is reached to prevent infinite loops.

The maximum step count is an important safety mechanism. Without it, a confused or looping agent can run indefinitely, consuming API credits and producing no useful output. A sensible default for most tasks is 10–15 steps; complex research or multi-stage tasks may warrant 20–30. When the maximum is hit, return what the agent has produced so far and log that the limit was reached — this is more useful than silently failing.

Adding a System Prompt

The system prompt shapes how the agent behaves across the entire session. A good agent system prompt covers: the agent’s role and purpose, the tools available and when to use each, how to handle uncertainty (ask for clarification vs make a best guess), and how to format the final answer. The system prompt is the primary lever for changing agent behaviour without touching the loop code — adjusting tone, changing decision-making heuristics, or adding new behavioural rules all happen here rather than in the Python code.

Keep system prompts specific and task-oriented. Generic prompts like “you are a helpful assistant” produce generic agent behaviour. Specific prompts like “you are a research assistant that answers questions by searching the web and synthesising information from multiple sources. Always cite your sources. If search results are contradictory, note the disagreement rather than picking one answer” produce focused, predictable behaviour that is easier to test and improve.

Streaming Responses

For agents used interactively, streaming the model’s output as it is generated produces a noticeably better experience — the user sees the agent’s reasoning and tool call decisions appear in real time rather than waiting for each full step to complete. The Anthropic and OpenAI SDKs both support streaming. The main implementation consideration is that you need to buffer the stream to detect tool calls, since a tool call may be split across multiple streaming chunks. Most production agent implementations either disable streaming for the tool-calling step and stream only the final response, or use a streaming parser that handles partial tool call JSON.

Testing Your Agent

A hand-built agent is testable in ways that framework-based agents often are not, because all the logic is visible in your code. Unit test the tool executor with known inputs and expected outputs — these tests run without any LLM calls and catch tool bugs quickly. Integration test the full loop with a small set of representative tasks — a research task, a multi-step calculation, a task that requires error recovery. Record the tool call sequences these tests produce and use them as regression baselines: if a code change causes the agent to take different steps on a task it previously handled correctly, the test fails and you investigate why.

Building an AI agent from scratch takes more initial effort than using a framework, but the result is code you understand completely, can debug directly, and can optimise for your specific use case. The loop is simple; the complexity comes from writing good tool descriptions, handling errors gracefully, and tuning the system prompt. Each of these is more tractable when you own all the code than when the framework handles them in ways that are difficult to inspect or override.

Observability: Logging Every Step

A hand-built agent is only as debuggable as its logging. Add structured logging at every key point in the loop: when a message is sent to the model, when a tool call is received, when a tool is executed, when a result is returned, and when the loop exits. Log the full content of each message and tool result, not just summaries — when an agent behaves unexpectedly, the full message log is almost always required to understand why. Use a structured logging format (JSON lines work well) so you can filter and search the logs programmatically when debugging a specific failure. For production agents, store logs to a persistent location and include a unique session ID in every log entry so you can reconstruct the full trace of any agent run after the fact. This investment in observability pays back every time you need to understand why an agent made a particular decision or took an unexpected path through a task.

Prompt Chaining vs Agents: Choosing the Right Approach

Not every multi-step AI task needs an agent. Before building an agent loop, consider whether the task can be solved with prompt chaining — a fixed sequence of LLM calls where the output of one feeds the input of the next. Prompt chaining is simpler, more predictable, easier to test, and cheaper than an agent loop for tasks where the sequence of steps is known in advance. Agents are the right choice when the sequence of steps depends on what the AI discovers as it works — when the task requires decision-making about what to do next based on intermediate results. A task like “write a blog post outline” is a prompt chain task. A task like “research this topic, identify the most credible sources, extract the key claims, and flag any contradictions” is an agent task because the steps depend on what the research finds. Defaulting to the simplest approach that solves the problem — prompt chain first, agent only when the task genuinely requires dynamic step selection — produces more reliable systems with less operational complexity.

Leave a Comment