11
$\begingroup$

how exactly is partial derivative different from gradient of a function?

In both the case, we are computing the rate of change of a function with respect to some independent variable. While I was going through Gradient Descent, there also the partial derivative term and the gradient were written and used separately.

What's the catch?

$\endgroup$
1
  • 1
    $\begingroup$ It's a theorem. The gradient is the derivative of a function $\mathbb{R}^m\to\mathbb{R}$, as explained at stats.stackexchange.com/a/257616/919. As such it is a linear form on $\mathbb{R}^m$. It is defined without reference to any particular basis of $\mathbb{R}^m$. The partial derivatives are the derivatives of functions $\mathbb{R}\to\mathbb{R}$ defined by holding all but one variable fixed. They depend on the basis chosen for $\mathbb{R}^m$. The theorem asserts that the components of the gradient with respect to that basis are the partial derivatives. $\endgroup$
    – whuber
    Commented Jun 16, 2017 at 14:26

2 Answers 2

10
$\begingroup$

Gradient is the partial derivatives :

$$\nabla f = \left(\frac{\partial f}{\partial x_1};\frac{\partial f}{\partial x_2};...;\frac{\partial f}{\partial x_n}\right)$$

Eg : $f=x^2y$

$$\nabla f =(2xy;x^2)$$

Gradient gives the rate of change in every direction $e$ ($e$ is a unit vector) thanks to the dot product $\nabla f.e$ :

Eg :$\nabla f.(0;1)=\frac{\partial f}{\partial y}$

$\endgroup$
5
$\begingroup$

If a function $f$ takes the parameters $x_1, \ldots, x_n$, then the partial derivatives w.r.t. the $x_i$ determine the gradient:

\begin{equation} \nabla f = \frac{\partial f}{\partial x_1 }\mathbf{e}_1 + \cdots + \frac{\partial f}{\partial x_n }\mathbf{e}_n. \end{equation}

If you look at the definition of the gradient-descent method, it is completely defined in terms of the gradient.

how exactly is partial derivative different from gradient of a function?

A partial derivative may be taken also w.r.t. a different variable, e.g.,

\begin{equation} \frac{\partial f}{\partial z }, \end{equation}

where $z = z(x_1, \ldots, x_n)$ is some function of the xs.

$\endgroup$
2
  • $\begingroup$ can gradient be defined as partial derivatives w.r.t say yi's instead of xi's. Is it allowed? $\endgroup$
    – Upendra01
    Commented Jun 16, 2017 at 8:52
  • $\begingroup$ @Snowbell You could form a vector composed of partial derivatives w.r.t. yis, but, in general, it will not correspond to the direction of greatest change. $\endgroup$
    – Ami Tavory
    Commented Jun 16, 2017 at 8:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.