How an agent loop works

Strip away the chat window and a language model is a function. You give it a list of messages. It gives you back one new message. Then it forgets the whole exchange.

That last part is the one people miss. When you chat with an assistant in an app, it feels like a continuing conversation. It is not. Each time you press send, the app collects the whole conversation so far and sends all of it again. The model reads the full transcript from the top, writes the next message, and is done.

This single fact explains most of the bugs you will meet. When a model "ignores" something, the first question is always the same: was that thing actually in the list on that call?

The list is made of small dictionaries. Each has a role, which says who is speaking, and content, which says what they said. There are four roles you need to know.

• system: standing instructions from you, the engineer. Tone, format, rules. It usually comes first.
• user: the question or request.
• assistant: something the model said. This can be text, or it can be a request to run tools.
• tool: the result of a tool the harness ran, sent back to the model.

The model has no way to run a tool itself. When it wants one, it writes an assistant message that contains a tool request: the tool's name, its arguments, and an id that identifies this request. Your code runs the tool. Then your code writes a tool message that carries the same id, so the model can match result to request.

Here is a tiny agent that answers travel questions. It has one tool, get_forecast(city, day). The user asks whether to pack an umbrella.

On the first call, the list is just the instructions and the question:

1[system]     You are a travel assistant. Use tools when you need facts.2[user]       Should I pack an umbrella for Lisbon tomorrow?

The model cannot know tomorrow's weather, so it replies with a tool request instead of an answer:

1[assistant]  wants to call get_forecast(city="Lisbon", day="tomorrow")   id=call_1

The harness runs the tool and gets a string back. Now the harness has to do two things before it calls the model again. It adds the assistant's request to the list, then it adds the result. The list for call 2 looks like this:

1[system]     You are a travel assistant. Use tools when you need facts.2[user]       Should I pack an umbrella for Lisbon tomorrow?3[assistant]  wants to call get_forecast(city="Lisbon", day="tomorrow")   id=call_14[tool]       id=call_1   "Lisbon, tomorrow: light rain, 14 to 18 C"

Now the model can see the whole story. It answers in plain text:

1[assistant]  Yes. Light rain is expected tomorrow, so pack an umbrella.

Notice what changed between call 1 and call 2. The list grew by two messages, and the model's behavior changed because of it. That is the entire mechanism. You do not make a model smarter during a run. You change what it can see.

You do not know in advance how many tool calls a question needs. Some questions need none. Some need one. Some need a search, then a read, then a second read to check something.

So the harness cannot be a fixed script. It has to repeat: call the model, look at what it wants, act on it, and call again. Written in words:

1. Send the list to the model.
2. If the reply is text with no tool requests, that is the answer. Stop.
3. If the reply asks for tools, record the request, run each tool, and record each result.
4. Go back to step 1.

A tool call is never the end of a task. It is the model asking for information so that it can take its next turn. If your loop stops right after a tool runs, the model never gets to use what the tool found.

A loop needs ways to end. There are three, and a good harness handles all of them on purpose.

• The model answers. No tool requests means it is done. This is the normal exit.
• The budget runs out. Every task has a cap on steps, and a step is one call to the model. The cap is what stops a confused model from looping forever. When you hit it, decide what to return, and do not let the function fall off the end with nothing.
• Progress stops. If the model asks for the exact same thing three times in a row, more steps will not help. Something is wrong with what it can see.

Budgets also cost money. Because every call re-sends the entire list, a long run gets more expensive with each step, since the list keeps growing. Short runs are cheaper, faster, and usually more accurate.

A model is allowed to ask for more than one tool at a time, such as the forecast for two cities. The reply then holds a list of tool requests, each with its own id.

Each request needs its own result, and each result must carry the matching id. If you skip one, or merge two results into one message, the model is left with a request that was never answered, and many APIs reject the next call outright.

When a loop misbehaves, the trace shows a pattern. These are the common ones.

• The model makes the same request again and again. It is not seeing the result of the earlier request. Check whether the result was added to the list, and whether it was added before the next call.
• The run ends right after the first tool result, with no answer. The loop returned or exited too early. Finishing a tool call is not finishing the task.
• The API rejects a call, saying a tool message has no matching request. The order of messages is wrong. The assistant message that asked for the tool must already be in the list when the result arrives.
• The run ends with the fallback value. The steps ran out. Look at what the model asked for on each step and ask why it never reached an answer.
• The right information is in the trace and the answer is still wrong. The loop is probably fine. Look at the context engineering guide: the problem may be what else is in the list.

Whenever a run looks wrong, stop reading the model's output and read its input. Take the call where things went wrong and print or open the exact list that was sent. Then read it as if you were the model, who has never seen anything else.

Can you tell what the user wants? Can you see the result of the last tool? Is anything confusing, contradictory, or missing? If you would be stuck, so is the model.

Problems 01 and 02 are about this loop. Both start from a harness that already calls the model and runs tools. When you open one, read the starter's trace and walk the list call by call, asking what the model could see each time. The gap you are looking for is between what the harness knew and what it put in the list.