Querying Ollama Models

Most of the time you will meet the Ollama REST API through a small function that wraps the request for you. This article starts from exactly that kind of function, called query_model, and works backward into the mechanics. Instead of memorizing the HTTP details first, you will follow one call from start to finish and watch what changes at each stage.

Keep one question in mind as you read. When you call the function with a prompt and get text back, that text passed through several stages: a dictionary was built, converted to bytes, sent over HTTP, and then read back in pieces. If the result is wrong, your goal is to identify the stage where it went off track.

The function accepts three parameters: prompt, model, and url. The prompt is the text you want the model to respond to. The model is the name of the Ollama model that should generate the reply. The url tells the function where the Ollama service is listening. Its default endpoint is http://localhost:11434/api/chat, which means that if you leave the url argument out, the request goes to the chat endpoint on the local machine.

Defaults are written into the function's signature, so you can read them there to see what a call uses when you leave arguments out. This matters when you trace a result: a value you did not pass explicitly may still be shaping the request.

The first thing query_model does is assemble a dictionary. This dictionary is the payload: the data the function will send to the server. Its outer structure has three keys, which are the model name, the messages, and the options. Each key holds a different part of the request, and the prompt you passed in ends up inside the messages part.

Once the payload dictionary exists, query_model converts it to JSON bytes and sends it as a POST request to the chat endpoint. The conversion step turns the Python dictionary into a serialized form that can travel over HTTP. The server then processes the request and sends its reply back to the function over the same connection.

The reply is not read as one block. The function reads it one line at a time, and each line carries a piece of the model's output in its message content. Reading line by line means the function handles each piece as it arrives instead of waiting for a single fixed-size result. The loop continues until there are no more lines to read.

Each line's message content is appended to a growing string. The string starts out holding nothing useful and gets longer with every line. When the reading loop finishes, the accumulated string is the reply, and the function returns it. This is why the value you receive is one complete piece of text even though it arrived in many fragments.

The source demonstrates query_model with a single example call. The call asks the llama3 model what llamas eat. The model name is stored in a variable first, and the prompt is passed directly as the first argument. Storing the model name in a variable makes it easy to change later without touching the rest of the call.

When a result from query_model looks wrong, do not guess. Walk the stages in order and check the value each one produced. The stages are building the payload, encoding it as JSON, sending the POST request, reading the lines, and gathering the content into the returned string. The Ollama service itself sits outside these stages and can also be the source of a surprising reply.

A useful habit is to ask, for each suspicious result, which stage could have produced that specific value. A wrong model name points toward the payload. A failure to reach the server points toward the request. A reply that is incomplete points toward the reading loop or the gathering step. A reply that is correct in form but unexpected in content may point to the Ollama service and the model it ran.