2
$\begingroup$

how exactly is partial derivative different from gradient of a function?

In both the case, we are computing the rate of change of a function with respect to some independent variable. While I was going through Gradient Descent, there also the partial derivative term and the gradient were written and used separately.

What's the catch?

$\endgroup$
  • $\begingroup$ It's a theorem. The gradient is the derivative of a function $\mathbb{R}^m\to\mathbb{R}$, as explained at stats.stackexchange.com/a/257616/919. As such it is a linear form on $\mathbb{R}^m$. It is defined without reference to any particular basis of $\mathbb{R}^m$. The partial derivatives are the derivatives of functions $\mathbb{R}\to\mathbb{R}$ defined by holding all but one variable fixed. They depend on the basis chosen for $\mathbb{R}^m$. The theorem asserts that the components of the gradient with respect to that basis are the partial derivatives. $\endgroup$ – whuber Jun 16 '17 at 14:26
2
$\begingroup$

If a function $f$ takes the parameters $x_1, \ldots, x_n$, then the partial derivatives w.r.t. the $x_i$ determine the gradient:

\begin{equation} \nabla f = \frac{\partial f}{\partial x_1 }\mathbf{e}_1 + \cdots + \frac{\partial f}{\partial x_n }\mathbf{e}_n. \end{equation}

If you look at the definition of the gradient-descent method, it is completely defined in terms of the gradient.

how exactly is partial derivative different from gradient of a function?

A partial derivative may be taken also w.r.t. a different variable, e.g.,

\begin{equation} \frac{\partial f}{\partial z }, \end{equation}

where $z = z(x_1, \ldots, x_n)$ is some function of the xs.

$\endgroup$
  • $\begingroup$ can gradient be defined as partial derivatives w.r.t say yi's instead of xi's. Is it allowed? $\endgroup$ – Upendra Pratap Singh Jun 16 '17 at 8:52
  • $\begingroup$ @Snowbell You could form a vector composed of partial derivatives w.r.t. yis, but, in general, it will not correspond to the direction of greatest change. $\endgroup$ – Ami Tavory Jun 16 '17 at 8:54
2
$\begingroup$

Gradient is the partial derivatives :

$$\nabla f = \left(\frac{\partial f}{\partial x_1};\frac{\partial f}{\partial x_2};...;\frac{\partial f}{\partial x_n}\right)$$

Eg : $f=x^2y$

$$\nabla f =(2xy;x^2)$$

Gradient gives the rate of change in every direction $e$ ($e$ is a unit vector) thanks to the dot product $\nabla f.e$ :

Eg :$\nabla f.(0;1)=\frac{\partial f}{\partial y}$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.