Tensors in PyTorch

Suppose a colleague tells you that a team is building its neural network experiments in PyTorch. Before you can follow that conversation, you need a clear picture of what the name refers to. This article starts from that practical question and builds toward a precise answer, so that later material about models, training, and experiments has a stable foundation to rest on.

The central object you will meet in PyTorch is the tensor. If you have already worked with NumPy arrays, it will look familiar almost immediately. The guiding question for this article is: what does PyTorch add on top of that familiar object, and why would deep learning need those additions?

A compact way to understand this definition is to read it one phrase at a time. The first phrase, open source, means the project is published in a form that anyone can inspect, use, and build upon, rather than being sold as a closed product.

The second phrase, Python-based, means that the everyday way of working with the library is through the Python language, the same language many beginners already use for scripting and data work.

The third phrase, deep learning library, is the most important for understanding the tool's purpose. A deep learning library is a collection of ready-made building blocks for constructing and training deep neural networks. Deep learning is the branch of machine learning that relies on neural networks with many layers. Instead of writing the mathematics of every operation from scratch, a practitioner calls into a library that already provides those operations. PyTorch's job is to supply the machinery a researcher or engineer needs in order to design a network and train it.

A definition tells you what a tool is, but adoption tells you whether people actually rely on it. The source points to two independent kinds of evidence.

The first comes from Papers With Code, a platform that tracks and analyzes research papers. According to that platform, PyTorch has been the most widely used deep learning library for research since 2019, and it held that position by a wide margin. The phrase research is doing real work here. The claim concerns the library researchers pick when they publish and experiment, not every possible use of deep learning.

The second comes from the Kaggle Data Science and Machine Learning Survey 2022. In that survey, approximately 40% of respondents reported using PyTorch. The source also notes that this share grows every year, so the usage picture is a trend moving upward rather than a single snapshot. Taken together, the research ranking and the survey share suggest that PyTorch is not a niche choice among a few specialists.

Popularity is usually a consequence of experience, so it helps to ask what a person feels when working with the library day to day. The source attributes much of PyTorch's appeal to two qualities that are normally in tension with each other.

The first quality is a user-friendly interface together with efficiency. This makes the library easy to approach and reasonably quick in operation. The second quality is flexibility. Many tools that are easy to start with become restrictive once you need something unusual. According to the source, PyTorch does not give up that flexibility. It lets advanced users reach into the lower-level aspects of their models to customize and optimize them.

The source describes this as a balance. A beginner can start with the friendly interface without being boxed in later, because the same library still exposes lower-level control to people who need it. That balance is the central reason the source gives for the library's popularity, and it explains why the adoption evidence and the qualities of the tool belong in the same discussion.

The source describes PyTorch tensors as similar to NumPy arrays, and that similarity is the foundation for everything else in this article. A tensor holds numerical data arranged in array-like form. You can think of it as a close relative of the NumPy array rather than an unrelated structure. This means NumPy knowledge transfers well.

The source is explicit that the similarity is not the whole story. PyTorch tensors have several additional features that matter for deep learning. So the right mental model is a NumPy-style array that has been extended for a specific job: training deep neural networks.

In this example the two objects behave alike for shape and summation, which is exactly the resemblance the source describes. The printed forms differ slightly: the PyTorch result is wrapped to show that it is a tensor. That small difference is a first hint that a tensor is a PyTorch object with its own identity, not merely a renamed array.

PyTorch adopts most of the NumPy array API and syntax for its tensor operations. In practice, the way you write many operations on tensors looks the same as the way you would write them on NumPy arrays. A learner who already knows NumPy method names and indexing patterns can read much PyTorch code without relearning the basic vocabulary.

A practical way to apply this is to sort your knowledge into two groups. The first group is the shared vocabulary: array-like data, common method names, and familiar indexing patterns. The second group is the set of places where PyTorch goes beyond a plain array, which the next sections cover. The source does not enumerate every exception to the NumPy resemblance, so the safe habit is to check unfamiliar behavior instead of carrying every NumPy assumption across unchanged.

Two additional features are named directly in the source. Each one addresses a practical need in deep learning. The first makes it easier to compute gradients. The second helps speed up the training of deep neural networks. The subsections below look at each in turn.