Installing a GPU-Compatible Version of PyTorch

Before you change any training code to use a GPU, it helps to ask a simple question first: does the runtime you are working in support GPU computing at all? PyTorch gives you a direct way to find out. This article starts with a short script that asks PyTorch for that answer, then uses what the script reveals to build an understanding of how PyTorch chooses where work happens.

The function torch.cuda.is_available() reports whether GPU computing is available in the current runtime. Notice what it does not report: it does not tell you where any particular tensor currently sits. Those are two separate questions. The first is about the environment you are running in; the second is about the placement of data and work, which the rest of this article covers.

The diagram shows the decision the check gives you. A True result tells you GPU computing can be used in this runtime. A False result tells you it cannot, so the work stays on the CPU. Either way, the check itself does not move any data or change where operations run. It only reports what the runtime supports.

Think of a device as a location with two jobs at once. It is the place where PyTorch carries out arithmetic, and it is also the place where the numbers involved in that arithmetic are held. Because both jobs belong to one concept, you cannot talk about where a computation happens without also talking about where its data lives.

Both the CPU and the GPU are devices, so both can host computations and data. They are not treated the same way by default, though. Computations in PyTorch are carried out on the CPU by default, so moving work to a GPU is a deliberate step rather than something that happens automatically.

Now consider a small computation: two tensors that can be added together. Before reading the result, try to predict what the program prints. Focus on the values first; the device question comes next.

Notice what the code does not do: it never names a device. Yet the source states that this computation is carried out on the CPU by default. Because the tensors were created without any move to another device, the addition runs where the tensors already are, which is the default CPU location.

The source states a specific relationship: a PyTorch tensor resides in a device, and its operations are executed on that same device. This gives two consequences. First, a tensor is never floating in no location; it always sits on some device. Second, when you ask PyTorch to perform an operation on a tensor, that operation runs on the device where the tensor is. The tensor and the work it triggers are tied together.

Read the diagram as a single chain of reasoning. The tensor resides in a device. The operation executes on that same device. Because both are tied to one device, the answer to 'where does this operation run?' is always the answer to 'where does this tensor reside?'. This is why the device concept covers both the arithmetic and the data at once.