Computation Graphs

Most of the time, a model's prediction feels like a single action. You give it an input and it returns an answer. Underneath, though, the answer is produced by a chain of small operations, and each operation consumes the results of the one before it. The practical question this article asks is how to write that chain down so that every intermediate value has a place, a name, and a clear route to the final result.

The tool that answers this question is the computation graph. A computation graph is a sequence of computations drawn as a directed graph, so each intermediate value has a clear source and destination. Every step becomes a node, and every arrow points in the direction that data travels. Once a calculation is drawn this way, you can point at any value and see what produced it and what it will feed into next.

A logistic regression classifier is a single-layer neural network. Its forward pass multiplies the input by a weight, adds a bias, and applies a sigmoid to return a score between 0 and 1. The classifier has a small set of parts, and each one has a specific job in producing that score.

The weight and the bias are the parameters the model adjusts during training, while the input comes from the data. The label is the fourth quantity, but it does not take part in the prediction. It enters only at the comparison that follows the prediction, and that comparison is what produces the loss, the quantity the learning process tries to reduce.

The forward pass is the prediction step of the classifier. To trace it, read the computation from top to bottom and track dependencies rather than syntax. The net input cannot be computed until the feature, weight, and bias exist. The output cannot be computed until the net input exists. The loss cannot be computed until both the output and the label exist. Each step depends only on values defined before it.

The sigmoid is the function that keeps the output between 0 and 1. It takes the net input, which can be any number, and maps it into a bounded range. This is what lets the classifier's output be read as a score and kept in a form that can be compared directly with a label that is either 0 or 1.

In the worked example, the net input is 0 and the sigmoid returns 0.5. In the graph, the sigmoid sits after the net input and before the loss. Its output is the last score value that the comparison with the label receives, so every upstream value exists to feed the score that the sigmoid produces.

To turn the computation into a graph, each input and parameter becomes a starting node. Each operation becomes a node that takes its inputs from earlier nodes. Arrows point from the values that are used toward the results that use them. The result is a directed graph in which the inputs flow toward the loss at the far end.

The graph makes three distinctions visible that the code only hints at. The input features and parameters sit at the top because they are the starting values. The intermediate results, such as the net input, sit in the middle because they exist only to pass information along. The output and the loss sit at the bottom because they are the end products. The label enters from the side, because it never feeds the prediction, only the comparison that follows it.

A graph is also useful as a pipeline. Each intermediate value appears once, has one source, and is passed to the next step. Following the arrows from left to right shows the order in which values must be ready, and it shows that the score is the end of a chain rather than a single lookup.

The code and the graph describe the same computation, but they are not the same object. The code is a sequence of lines that create values and operations. The graph is the structure those lines produce: nodes for values and operations, and arrows for the dependencies between them. The structure is what represents the computation, because it shows which values must be ready before each step can run.