Agents and Tool Use in AI Systems

A language model answers questions using two sources: what it absorbed during training, and whatever text appears in its current input. Neither source is ideal. Training knowledge goes stale, and the input has a limited size, so a model cannot simply be handed an entire company's documents and asked a question about them. Retrieval-Augmented Generation, usually shortened to RAG, is a design pattern that sidesteps both limits.

Instead of relying on memory alone, a RAG system first looks up material that might be relevant and then places that material in front of the model. This article focuses on the first half of that arrangement, the retriever, because the quality of a RAG system is tied closely to how well the retriever does its job.

A RAG system works in two stages. In the first stage, it retrieves information from an external memory, which is a store of documents, passages, or records that exists outside the model. In the second stage, it passes the retrieved material to the model, which uses it to generate a response. The retrieval stage supplies the facts, and the generation stage turns those facts into an answer.

Follow the arrows in the diagram. The query never reaches the model directly as a request to be answered from memory. Instead, the retriever uses it to search the external memory, and only the material it finds is passed forward. The final response is produced by the model, but its factual basis comes from what the retriever supplied. The result is expected to be more accurate than what the model could produce from its own memory, because the answer is anchored to material the system actually found.

A term-based retriever decides relevance by looking at the words themselves. The process runs in a sequence that you can follow step by step.

Systems such as Elasticsearch and the BM25 ranking method are examples of this approach. Their main practical advantage is that they are much lighter to set up and run. They also often give a solid starting point against which more advanced retrievers can be measured. The limitation follows directly from the method: the approach is only as good as the vocabulary overlap between the question and the documents. A relevant document written with different words can be missed.

An embedding-based retriever takes a different route. Rather than comparing words directly, it relies on vector search. Items are represented as vectors, and the system looks for the stored vectors that are closest to the query's vector. Because the comparison happens in this representation space, the approach has the potential to recognize relevance even when the wording differs.

The diagram shows the lookup. The query is converted into a vector, and the system measures how close that vector sits to each stored document vector. The document whose vector lies closest, here Document B, is the strongest candidate. The search does not depend on the documents sharing the same words as the query; it depends on where the vectors sit in the shared space.

The cost is computational. Embedding-based retrieval is more computationally intensive than term matching, so the system does more work for each lookup. Vector search is not a technique invented only for language models. It is also the backbone of many core internet applications, including search and recommender systems, and many vector search algorithms developed for those applications can be reused in RAG.

The two families differ along several dimensions at once: what they match on, how much effort they require, and where their advantages appear. Seeing them side by side makes the choice easier to reason about than reading about each separately.

Because the generator can only work with what the retriever supplies, the retriever largely determines RAG quality. A stronger generator cannot recover a relevant document that the retriever never returned. This is why the choice of retrieval method is a central design decision rather than a minor implementation detail.

RAG can be understood as a special case of an agent in which the retriever is one of the tools the model is allowed to use. Both RAG and agentic designs let a model get past its context limit and stay current, because both bring in outside information. The agentic pattern goes further. An agent is defined by its environment and by the tools it can access. Inside an AI-powered agent, the model acts as the planner: it analyzes the task, considers several possible solutions, and selects the most promising one.

Read the map from the top down. The language model is the planner inside an agent. An agent gains capability through the tools it can access, and RAG is one arrangement of that pattern in which the retriever is the tool. The same map also shows the cost of this flexibility. Tool use exposes the agent to a range of security problems, and because the retriever is itself a tool in the RAG-as-agent view, the same concern applies to the retrieval step.

RAG and agents share one more feature: both are prompt-based methods. They shape the model's output solely through what goes into its input, without changing the model's internal weights. That is what separates them from approaches that modify the underlying model itself.

Capability and exposure rise together. The more tools a model is given, the more capabilities it has, which lets it take on harder tasks. Planning also matters, because a complex task may need many steps and the model must be powerful enough to plan them. Planning can be strengthened with reflection and with a memory system that keeps track of progress.

As an agent becomes more automated, the consequences of its failures can become more severe. For agents to operate reliably in the real world, rigorous defensive mechanisms must be in place before they are trusted with consequential work. The source does not reduce this to a single rule; it treats automation, tools, and risk as a package that must be managed together.