Scientific Computing in Python: Introduction to NumPy and Matplotlib
The first feature PyTorch adds on top of the NumPy-style array is a differentiation engine that makes computing gradients simpler.
Why Your NumPy Knowledge Carries Over
If you have already worked with NumPy arrays, the central object in PyTorch will look familiar almost immediately. PyTorch's tensor is built to resemble the NumPy array, so much of what you learned about arrays carries over. This article gives you a way to read PyTorch code without relearning the basic vocabulary.
As you read, keep one question in mind: what does PyTorch add on top of this familiar object, and why would deep learning need those additions? Answering that question is the goal of the rest of the article. You will trace the similarities first, then the two additions, and then the places where the similarity stops.
A Tensor as a NumPy-Style Array
A PyTorch tensor holds numerical data arranged in array-like form. It is best understood as a close relative of the NumPy array rather than an unrelated structure.
The source describes PyTorch tensors as similar to NumPy arrays, and that similarity is the foundation for everything else. The useful mental model is a NumPy-style array that has been extended for a specific job: training deep neural networks. The extension does not replace the array idea. It sits on top of it.
The similarity shows up in the API. PyTorch adopts most of the NumPy array API and syntax for its tensor operations. In practice, the way you write many operations on tensors looks the same as the way you would write them on NumPy arrays. A learner who knows NumPy method names and indexing patterns can read much PyTorch code immediately.
Automatic Differentiation: Computing Gradients Without Hand Work
The first addition the source names is an automatic differentiation engine. The source says this engine simplifies computing gradients. A gradient tells you how a quantity changes when an input changes, and training a deep neural network depends on computing gradients many times.
The purpose of the engine is practical. Deep learning needs gradients for the whole network, and computing them by hand for every operation would be tedious and error-prone. PyTorch's automatic differentiation engine is built to handle that work so that you do not have to derive each gradient yourself.
The first feature PyTorch adds on top of the NumPy-style array is a differentiation engine that makes computing gradients simpler.
GPU Computation: Speeding Up Network Training
The second addition is GPU support. The source says PyTorch tensors support GPU computations to speed up deep neural network training. Where a NumPy array is a CPU-oriented structure in the source's framing, a PyTorch tensor can be used for computation on a GPU, which is why the source ties it directly to faster training.
The diagram below places the GPU feature in the broader path a deep learning workflow follows. Read it left to right as the order of the source's reasoning. It is not a detailed description of how memory moves between devices.
The second feature PyTorch adds is GPU computation, which the source connects to speeding up the training of deep neural networks.
Where the NumPy-Like API Holds and Where It Breaks
The word most in the source matters. PyTorch adopts most of the NumPy array API, not all of it. Reading PyTorch code with NumPy habits is a good starting point, but it is not a guarantee that every operation behaves the same way. The mistakes below come from treating the similarity as complete.
Common mistake Assuming every NumPy function or method exists on tensors with the same name
Why it failsThe source says PyTorch adopts most of the NumPy API, not all of it.
Do this insteadCheck the specific operation you need in PyTorch rather than relying on memory of NumPy alone.
Common mistake Treating a tensor as only a NumPy array with a new name
Why it failsThe source says the similarity is not the whole story; PyTorch adds automatic differentiation and GPU computation for deep learning.
Do this insteadThink of the tensor as a NumPy-style array extended for training deep neural networks.
Common mistake Assuming gradient computation is something you must write by hand in PyTorch
Why it failsThe source states that PyTorch's automatic differentiation engine simplifies computing gradients.
Do this insteadUse the built-in engine for gradients instead of deriving every gradient manually.
Trace It: Testing the Shared-Memory Question
A natural question when you move data between NumPy and PyTorch is whether the two objects are linked. This source does not document that behavior, so the way to answer it is to run a test and observe the result yourself. The example below is an original teaching example, not one taken from the source.
Before you run the code, predict what print(t) will show. Then decide what your result would mean for the question of whether a NumPy array and a tensor point to the same data. Write down your prediction, run the code, and compare.
Hints
- Look at which object you modified and which object you printed.
- Compare the printed values with the values you assigned to the NumPy array.
- Whatever you observe is the evidence here; do not rely on a remembered rule.
Checking Your Understanding
In your own words, explain why the source says a PyTorch tensor is 'a NumPy-style array that has been extended', naming the two additions and the job they serve.
Hints
- Start from the phrase NumPy-style array, then name what is added.
- Connect each addition to a need in deep learning: computing gradients, and speeding up training.
Key Takeaways
- A PyTorch tensor is similar to a NumPy array, so much of the NumPy array API and syntax carries over to tensor operations.
- PyTorch adopts most of the NumPy API, not all of it, so each operation should be checked rather than assumed equivalent.
- PyTorch adds an automatic differentiation engine that simplifies computing gradients for deep learning.
- PyTorch tensors support GPU computation, which the source connects to speeding up the training of deep neural networks.