1
$\begingroup$

I'm trying to calculate the gradient of multivariate function g using NumPy.

g = lambda w: -np.sin(np.pi*np.sum(w**2)) + np.log(np.sum(w**2))

gradient = lambda w: ...

the parameter w is a vector, for example, w = np.array([0.5,0.5]). I calculated the gradient like this; gradient = lambda w: np.pi*np.sum(2*w)*-np.cos(np.pi*np.sum(w**2)) + np.sum(2*w)*(1/np.sum(w**2))

It does not give meaningful results. Is this formula correct or not?

$\endgroup$
1
  • 2
    $\begingroup$ Apply the Chain Rule. $\endgroup$
    – whuber
    Commented Apr 10, 2022 at 12:16

1 Answer 1

0
$\begingroup$

If we write it more explicitly, $$f(w)=-\sin\left(\pi\sum w_i^2\right)+\ln \left(\sum w_i^2\right)$$

$${\partial f \over \partial w_i}=-\cos\left(\pi\sum w_i^2\right)\pi2w_i+\frac{2w_i}{\sum w_i^2}$$

So, the derivative is a vector; which means your expression will look like

gradient = lambda w: -np.pi*2*w*np.cos(np.pi*np.sum(w**2)) + 2*w*(1/np.sum(w**2))

when used denominator layout.

$\endgroup$
5
  • $\begingroup$ Thanks. w is an array which consists of w0 and w1, for example w = np.array([0.5,0.5]). In this case np.sum(w^2) is equivalent to w0^2 + w1^2, so I considered the derivation (2 * w0 + 2 * w1) which is equivalent to np.sum(w*2). $\endgroup$
    – Oguz Aktas
    Commented Apr 11, 2022 at 5:01
  • $\begingroup$ Why do you sum the derivatives? $\endgroup$
    – gunes
    Commented Apr 11, 2022 at 5:15
  • $\begingroup$ Because in the function there is np.sum(w^2) part which is equivalent to w0^2 + w1^2. The parameter w is a 2-element array, the square roots of w0 and w1 added in the function, so I think the derivative of (w0^2 + w1^2) part should be np.sum(2*w). $\endgroup$
    – Oguz Aktas
    Commented Apr 11, 2022 at 5:38
  • $\begingroup$ No, derivative of w0^2 + w1^2 wrt w0 is 2w0, and wrt w1 is 2w1. You do not sum the derivatives. $\endgroup$
    – gunes
    Commented Apr 11, 2022 at 6:49
  • $\begingroup$ That is correct, in none of them it’s summed. $\endgroup$
    – gunes
    Commented Apr 11, 2022 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.