how exactly is partial derivative different from gradient of a function?

In both the case, we are computing the rate of change of a function with respect to some independent variable. While I was going through Gradient Descent, there also the partial derivative term and the gradient were written and used separately.

What's the catch?

• It's a theorem. The gradient is the derivative of a function $\mathbb{R}^m\to\mathbb{R}$, as explained at stats.stackexchange.com/a/257616/919. As such it is a linear form on $\mathbb{R}^m$. It is defined without reference to any particular basis of $\mathbb{R}^m$. The partial derivatives are the derivatives of functions $\mathbb{R}\to\mathbb{R}$ defined by holding all but one variable fixed. They depend on the basis chosen for $\mathbb{R}^m$. The theorem asserts that the components of the gradient with respect to that basis are the partial derivatives.
– whuber
Commented Jun 16, 2017 at 14:26

Gradient is the partial derivatives :

$$\nabla f = \left(\frac{\partial f}{\partial x_1};\frac{\partial f}{\partial x_2};...;\frac{\partial f}{\partial x_n}\right)$$

Eg : $f=x^2y$

$$\nabla f =(2xy;x^2)$$

Gradient gives the rate of change in every direction $e$ ($e$ is a unit vector) thanks to the dot product $\nabla f.e$ :

Eg :$\nabla f.(0;1)=\frac{\partial f}{\partial y}$

If a function $f$ takes the parameters $x_1, \ldots, x_n$, then the partial derivatives w.r.t. the $x_i$ determine the gradient:

$$\nabla f = \frac{\partial f}{\partial x_1 }\mathbf{e}_1 + \cdots + \frac{\partial f}{\partial x_n }\mathbf{e}_n.$$

If you look at the definition of the gradient-descent method, it is completely defined in terms of the gradient.

how exactly is partial derivative different from gradient of a function?

A partial derivative may be taken also w.r.t. a different variable, e.g.,

$$\frac{\partial f}{\partial z },$$

where $z = z(x_1, \ldots, x_n)$ is some function of the xs.

• can gradient be defined as partial derivatives w.r.t say yi's instead of xi's. Is it allowed? Commented Jun 16, 2017 at 8:52
• @Snowbell You could form a vector composed of partial derivatives w.r.t. yis, but, in general, it will not correspond to the direction of greatest change. Commented Jun 16, 2017 at 8:54