# Derivative of all the parameters in Logistic Regression

$$\mathcal{L}$$ is the loss function, $$\mathcal{L} = y_i \text{log} \sigma(z) + (1-y_i) \text{log} (1-\sigma(z))$$, where $$z = \sum_i w_ix_i$$, with $$w_i$$ representing the weights and $$x_i$$ the features. Typically, it is required to take a derivative of $$\mathcal{L}$$ with respect to $$w_1$$ or $$w_2$$. But I am required to provide derivative with respect to parameters which includes all weights.

Could anyone please recommend a solution or some tutorials? Thanks

• What do you mean by providing derivative wrt parameters which includes all weights'?' May 21, 2020 at 8:08
• What is $\sigma(x)$? The sigmoid function?
– Tinu
May 21, 2020 at 8:18

Since $$\mathcal{L}(w_1, w_2, ..., w_n)$$ is a scalar loss function depending on multiple weights $$w_i$$ (neglecting features $$x$$ and labels $$y$$ for a moment) taking the gradient with respect to all would give you a vector
$$\nabla \mathcal{L} = [\frac{\partial \mathcal{L}}{\partial w_1}, ...,\frac{\partial \mathcal{L}}{\partial w_n}]$$.
Now you can calculate each individual component with the following formula, using the sigmoid derivative $$\sigma'(x) = \sigma(x)(1 - \sigma(x))$$ and chain rule:
$$\frac{\partial \mathcal{L}}{\partial w_i} = \frac{y_ix_i\sigma(z)(1 - \sigma(z))}{\sigma(z)} - \frac{(1 - y_i)x_i\sigma(z)(1-\sigma(z))}{1 - \sigma(z)}$$.