0
$\begingroup$

I am trying to fit a generalized logistic function to a dataset and am having trouble computing the partial derivatives with respect to each of the variables. My cost function is as follows:

$$ J(\theta) =\sum _{i=1}^m\left(y_{i}\:-\:k\cdot \left(\frac{1}{1+10^{A\cdot \left(B\:-x_i\right)}}\right)\right)^2 $$

And I am trying to compute $\frac{\partial J(\theta)}{\partial k}$, $\frac{\partial J(\theta)}{\partial A}$ and $\frac{\partial J(\theta)}{\partial B}$ but while I managed to get $\frac{\partial J(\theta)}{\partial k} = 2K*\left<H,H \right> - 2 * \left<H,Y\right>$ through some linear algebra (and a tutorial I found on the Web) I can't seem to be able to compute the other ones. Could anyone guide me through one of them, or point me towards a solution?

$\endgroup$
0
$\begingroup$

Computing $\partial_a J$ and $\partial_b J$ is not harder than the derivative with respect to $k$. You just need to apply the chain rule (since we are looking at a composite function); e.g. for $a$,

$\partial_a J = \partial_H J \; \partial_a H$

where $h_i = h(a,b,x_i) = 1 / \left(1 + 10^{\, a \, (b-x_i)}\right)$ and $H = \{h_i\}, \forall i$, if $a, b \in \mathbb{R}$.

Or are you interested in a set of vector parameters, as in $h(A, B, x_i) = 1 / \left( 1 + 10^{\langle A,\, B-x_i\rangle}\right)$, where $x_i, A, B \in \mathbb{R}^m$ ?

For the latter case, I get

$\partial_{a_j}\, h_i = (B - x_i)_j \, \xi(A, B, x_i)$

$\partial_{b_j}\, h_i = a_j \, \xi(A, B, x_i)$ where $\xi(A, B, x_i) = - \frac{ \ln(10) \, 10^{\langle A, B-x_i\rangle}}{\left( 1+10^{\langle A, B-x_i\rangle} \right)^2}$, $\forall j \in {1 .. m}$

$\endgroup$
10
  • $\begingroup$ $H$ is a vector built from $h(x) = \frac{1}{1 + 10^{A(B-x_i)}}$ which I separated from the original prediction function in order to reformulate the cost function as $||KH - Y||^2$. I guess my calculus is just out of shape because I have been stuck on this for the past few hours. $\endgroup$
    – jjanssen
    Nov 17 '15 at 14:00
  • $\begingroup$ @jjanssen Edited, HTH ! $\endgroup$
    – ocramz
    Nov 17 '15 at 15:26
  • $\begingroup$ Why is there a j index in the partial derivatives? $\endgroup$
    – jjanssen
    Nov 17 '15 at 16:33
  • $\begingroup$ I made this up because you didn't specify the dimensionality of the parameters $A, B$ (and therefore of $k$ and $x_i$). Are they scalars or vectors or what else? If they are scalars forget about $j$ (i.e. A and B and $x_i$ have a single component respectively). $\endgroup$
    – ocramz
    Nov 17 '15 at 16:43
  • 1
    $\begingroup$ Thank you for your very helpful answer, I will try to see if I can work my way to them by myself! $\endgroup$
    – jjanssen
    Nov 17 '15 at 16:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.