Gradients and Directional Change

A partial derivative measures the rate at which a multivariate function changes with respect to one of its variables.

Programming

A Function With Two Inputs

Many useful quantities depend on more than one input at once. A quantity might depend on a position, a temperature, and a time all at the same moment. When a function has several inputs, a natural question arises: if only one of those inputs changes, how quickly does the output change? This article is about answering that question, and about how the answers are collected into a single object called a gradient.

What do you think happens?

Take f(x, y) = x^{^2} y and hold y fixed at 3. If only x changes, what expression describes how fast f changes with respect to x?

3x
6x
2x^2
x^{^2} + 3

Reveal answer

Answer:

With y held at 3, the function becomes 3x^2 when y is fixed. The ordinary derivative of 3x^2 with respect to x is 6x. Notice that the answer depends on x, which is why a rate like this is read at a particular point.

Holding Every Other Input Still

The word partial is the key to the whole idea. When you take the partial derivative with respect to one variable, you let that variable move and you study the resulting change in the output, while every other input is held where it is. Nothing new is computed mechanically. What changes is the question being asked: instead of asking how the output responds to all inputs together, you ask how it responds to one input alone.

Seen this way, a partial derivative is just an ordinary derivative computed along one direction of the input space. Every other input is treated as a fixed number during that computation. The effect of holding things still is that the problem turns into a single-variable derivative, which you already know how to handle.

For each partial derivative, one input is allowed to vary while the other input is frozen at its current value.

Reading the Partial Derivative Notation

For a function $f(x, y)$ with two inputs, the partial derivative with respect to $x$ is written $\frac{\partial f}{\partial x}$, and the partial derivative with respect to $y$ is written $\frac{\partial f}{\partial y}$. The curly symbol $\partial$ is a reminder that the output is being watched while only one input is allowed to vary.

Definition

A partial derivative measures the rate at which a multivariate function changes with respect to one of its variables, with every other variable held fixed.

The partial derivative with respect to $x$ can depend on $y$ as well. In the example $f(x, y) = x^{^2} y$, the rate of change with respect to $x$ is scaled by $y$. This is why a partial derivative is best read as the rate of change in one direction, at one particular point, rather than as a single fixed number for the whole function. The two partial derivatives of the same function are different expressions, and each is evaluated at a point.

Computing a Partial Derivative Step by Step

Worked example

Partial Derivatives of x squared times y

For $f(x, y) = x^2 y$, find $\frac{\partial f}{\partial x}$ and $\frac{\partial f}{\partial y}$, then evaluate both at the point where $x = 2$ and $y = 3$.

1
Differentiate with respect to x
Treat y as a fixed constant. The expression x^2 y is then a constant times x^2, so the rule for differentiating x^2 applies and the constant y stays in front: $\frac{\partial f}{\partial x} = 2xy$.
2
Differentiate with respect to y
Now treat x as the fixed constant. The expression is x^2 times y, and differentiating y leaves x^2 behind: $\frac{\partial f}{\partial y} = x^2$.
3
Evaluate the first partial at the point
Substitute x = 2 and y = 3 into 2xy to get $2 \cdot 2 \cdot 3 = 12$. This is the rate of change of f with respect to x at that point.
4
Evaluate the second partial at the point
Substitute x = 2 into x^{^2} to get $2^{^2} = 4$. This is the rate of change of f with respect to y at that point.

Result

At the point (2, 3), the partial derivative with respect to x is 12 and the partial derivative with respect to y is 4. Each number describes only the effect of its own input.

In the real world

Picture a quantity that depends on a position and a time. Freezing time and moving only position gives one rate, the change with respect to position. Freezing position and moving only time gives a second rate. Each rate is an ordinary single-variable derivative, computed along one direction while the other input stays where it is.

Gathering the Partials Into a Gradient

Once you have one partial derivative for every input, a natural next step is to keep them together. A gradient is a vector containing all of the partial derivatives of a multivariate function. It does not add new information beyond the partial derivatives. It packages them into a single object, so that the rates of change for every input travel together as one quantity.

Rule

Here f has n inputs x_1 through x_n, and the gradient holds one partial derivative for each of them, in a fixed order. For the function $f(x, y) = x^{^2} y$, the gradient is $\nabla f = (2xy, \; x^{^2})$. The vector has as many entries as the function has inputs, and each entry answers the single-variable question for its own input. The first entry belongs to x and the second to y, so the order of the entries carries meaning.

The gradient of x^{^2} y holds one entry per input, in a fixed order, and each entry is the partial derivative for that input.

Mistakes Beginners Make With Partials

Common mistake
Letting the other variables move while differentiating
What it looks like
Differentiating x^2 y with respect to x and writing 2x, dropping the y.
Why it fails
A partial derivative holds every other input fixed. Because y is a fixed value during the x computation, it remains as a factor in the result.
Do this instead
Write 2xy. Keep y as a constant multiplier when differentiating with respect to x.
Common mistake
Treating a partial derivative as one fixed number for the whole function
What it looks like
Concluding that the rate with respect to x of x^2 y is always 12.
Why it fails
The partial with respect to x is 2xy, which changes with both x and y. The value 12 applies only at the point (2, 3).
Do this instead
Read a partial derivative as a rate in one direction at a particular point, and state the point when you report a value.
Common mistake
Mixing up the entries of the gradient
Why it fails
The gradient is a vector whose entries follow a fixed order of inputs, so swapping them assigns a rate to the wrong input.
Do this instead
Keep the entries in the same order as the inputs, so the first entry is the partial with respect to the first input.
Common mistake
Forgetting the curly ∂ notation signals a partial, so an ordinary d can hide that other inputs are held fixed.

Checking Your Understanding of Partials

EASY

For f(x, y) = 3x + y^{^2}, find the partial derivative with respect to x and the partial derivative with respect to y. Then write the gradient as a vector.

Hints

When differentiating with respect to x, treat y^{^2} as a constant that disappears under the derivative.
When differentiating with respect to y, treat 3x as a constant.
The gradient lists the x partial first and the y partial second.

Your attempt (not saved)

Key Takeaways

A partial derivative measures the rate at which a multivariate function changes with respect to one variable, with every other input held fixed.
Computing a partial derivative turns a multi-input problem into an ordinary single-variable derivative along one direction of the input space.
The partial derivative notation with the curly ∂ symbol signals that only one input is allowed to vary while the output is watched.
A gradient is a vector containing all of the partial derivatives of a function in a fixed order; it packages the rates together without adding new information.

Related concepts

Functions of Several Variables

More concepts starting with G