Tensors and Basic Tensor Operations in PyTorch

When you start writing PyTorch code that handles tensors, a natural temptation is to switch everything over to a GPU as soon as one is mentioned. Before changing any code, it helps to ask a simple question first: does the runtime you are working in support GPU computing at all? PyTorch gives you a direct way to find out. This article starts with that check, then uses what it reveals to build an understanding of how PyTorch decides where work happens.

The question has two parts that are easy to confuse. The first is whether the current environment can use a GPU at all. The second is where a particular tensor and its computations actually live. This article treats them as separate questions, and the first half of the article is about the first one.

Think of a device as a location with two jobs at once. It is the place where PyTorch carries out arithmetic, and it is also the place where the numbers involved in that arithmetic are held. Because both jobs belong to one concept, you cannot talk about where a computation happens without also talking about where its data lives. A device is therefore not just a processor name; it is the combined answer to 'where is the work done' and 'where is the data kept'.

The two devices named in this course are the CPU and the GPU. Both are locations PyTorch can use for computation and for holding tensor data. The rest of the article explains how PyTorch chooses between them and why that choice matters for every operation you perform.

PyTorch provides torch.cuda.is_available() for the availability question. It reports whether GPU computing is available in the current runtime. A short script that asks PyTorch for that answer looks like this:

The flowchart shows that the check itself ends in a single answer: the runtime either supports GPU computing or it does not. Nothing in this step moves a tensor or changes where any operation runs. It only reports a fact about the environment.

The availability check answers whether a GPU can be used. It does not answer where a given tensor sits. For that, the important relationship is this: a PyTorch tensor resides in a device, and its operations are executed on that same device. The tensor and the work it triggers are tied together.

Two consequences follow from this relationship. First, a tensor is never floating in no location; it always sits on some device. Second, when you ask PyTorch to perform an operation on a tensor, that operation runs on the device where the tensor is. You do not pick the compute location separately from the data location.

The diagram captures the rule in words: the tensor resides in one device, and the operation is executed on that same device. Because of this, the device is decided by where the tensor already lives, not by a separate choice made at the moment of the operation.

Now consider a small computation: two tensors that can be added together. Before reading the result, try to predict what the program produces. Focus on the values first; the device question comes next.

The CPU and the GPU are both devices, so both can be the place where computation occurs and where data resides. They differ in which one PyTorch uses by default and in whether a given runtime can use a GPU at all. The comparison below keeps to those facts.