Observability Best Practices for Complex Systems

The applications built so far are fairly simple. A query arrives, the system performs its steps in a fixed order, and a response leaves the system. Nothing in that design goes back to an earlier step. Every query moves forward along one path and then stops.

Real tasks often do not fit a single forward pass. A system may produce an answer and then discover that the answer is incomplete. The design question is how to let the flow of an application bend: revisit earlier steps, split work into parallel branches, and choose different paths depending on what has happened so far. Those capabilities are what agentic patterns provide, and they are the subject of this article.

The source gives a concrete case. After the system generates an output, it may judge that the task has not yet been accomplished. It then performs another retrieval to gather more information. The original response, together with the newly retrieved context, is passed into the same model or into a different one. That return path is the loop: the output of one pass becomes part of the input to the next pass.

Read the diagram as a route rather than a list. The decision node is where control branches. When the answer is no, control goes to the retrieval step and then returns to the model with more material. When the answer is yes, the flow exits. The source also allows the return pass to use a different model, because the combined response and context may go into either the same model or a different one.

The easiest way to understand a loop is to watch what the input to the model looks like on each pass. Each pass starts with whatever the previous pass left behind. The diagram below tracks that growth using a generic scenario: a request whose first answer is incomplete.

Notice what changed between the first and second passes. The loop did not restart from scratch. The earlier response survived and was combined with fresh retrieved material, so each iteration builds on the previous one. This is why a loop can improve an answer that a single pass left incomplete.

Loops are one of three flow features the source names: loops, parallel execution, and conditional branching. The source names all three as part of the same family of flow designs. It does not walk through a detailed example of the other two, so the central case to master is the loop, and the other two are understood as ways a flow can depart from a single line of steps.

So far the model's output has only been something the application reads, perhaps to decide whether to loop. A second capability is that a model's output can be used to invoke write actions. A write action is an action that changes the environment outside the model itself. The source gives three examples: composing an email, placing an order, and initializing a bank transfer.

The contrast in the diagram is the important part. A read-only retrieval returns text that the model can use as context, and nothing outside the system is altered. A write action leaves the system and changes something in the environment. If the model's decision was wrong, the error is no longer contained in a generated sentence; it has already been acted on.

After you add loops, branches, and write actions, the architecture has usually grown quite complex. The source makes a point that is easy to overlook: complex systems can solve more tasks, but they also introduce more failure modes. Each additional loop, branch, or external action is another place where something can go wrong.

Debugging becomes harder because a failure may depend on which path the flow took, how many times a loop ran, or what context was retrieved on a particular pass. The source attributes this difficulty to the many potential points of failure. The diagram below shows the order in which components can interact during one request, which is the sequence a trace would need to reveal.