# Partial derivatives of cost function using generalized logistic function as a predictor

I am trying to fit a generalized logistic function to a dataset and am having trouble computing the partial derivatives with respect to each of the variables. My cost function is as follows:

$$J(\theta) =\sum _{i=1}^m\left(y_{i}\:-\:k\cdot \left(\frac{1}{1+10^{A\cdot \left(B\:-x_i\right)}}\right)\right)^2$$

And I am trying to compute $\frac{\partial J(\theta)}{\partial k}$, $\frac{\partial J(\theta)}{\partial A}$ and $\frac{\partial J(\theta)}{\partial B}$ but while I managed to get $\frac{\partial J(\theta)}{\partial k} = 2K*\left<H,H \right> - 2 * \left<H,Y\right>$ through some linear algebra (and a tutorial I found on the Web) I can't seem to be able to compute the other ones. Could anyone guide me through one of them, or point me towards a solution?

## 1 Answer

Computing $\partial_a J$ and $\partial_b J$ is not harder than the derivative with respect to $k$. You just need to apply the chain rule (since we are looking at a composite function); e.g. for $a$,

$\partial_a J = \partial_H J \; \partial_a H$

where $h_i = h(a,b,x_i) = 1 / \left(1 + 10^{\, a \, (b-x_i)}\right)$ and $H = \{h_i\}, \forall i$, if $a, b \in \mathbb{R}$.

Or are you interested in a set of vector parameters, as in $h(A, B, x_i) = 1 / \left( 1 + 10^{\langle A,\, B-x_i\rangle}\right)$, where $x_i, A, B \in \mathbb{R}^m$ ?

For the latter case, I get

$\partial_{a_j}\, h_i = (B - x_i)_j \, \xi(A, B, x_i)$

$\partial_{b_j}\, h_i = a_j \, \xi(A, B, x_i)$ where $\xi(A, B, x_i) = - \frac{ \ln(10) \, 10^{\langle A, B-x_i\rangle}}{\left( 1+10^{\langle A, B-x_i\rangle} \right)^2}$, $\forall j \in {1 .. m}$

• $H$ is a vector built from $h(x) = \frac{1}{1 + 10^{A(B-x_i)}}$ which I separated from the original prediction function in order to reformulate the cost function as $||KH - Y||^2$. I guess my calculus is just out of shape because I have been stuck on this for the past few hours. Nov 17 '15 at 14:00
• @jjanssen Edited, HTH ! Nov 17 '15 at 15:26
• Why is there a j index in the partial derivatives? Nov 17 '15 at 16:33
• I made this up because you didn't specify the dimensionality of the parameters $A, B$ (and therefore of $k$ and $x_i$). Are they scalars or vectors or what else? If they are scalars forget about $j$ (i.e. A and B and $x_i$ have a single component respectively). Nov 17 '15 at 16:43
• Thank you for your very helpful answer, I will try to see if I can work my way to them by myself! Nov 17 '15 at 16:52