0
$\begingroup$

The equation to find the partial derivative of a cost function with respect to a parameter θj is given in the book Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow:

$$ \frac{\partial}{\partial \theta_j}\operatorname{MSE}({\boldsymbol{\theta}}) = \frac{2}{m} \sum_{i=1}^m \left(\boldsymbol{\theta}^\intercal \boldsymbol{x}^{(i)} - y^{(i)}\right)x_j^{(i)} $$

  • m = number of instances in the dataset
  • x = input vector for the prediction
  • y = label for the input vector

Is $\theta_j$ in the equation a unique combination of bias and weights which is reflected in the vector $\boldsymbol{\theta}$ in the equation? Or is it a single parameter inside a vector?

$\endgroup$
0

2 Answers 2

0
$\begingroup$

In response to question and also following comment:

Sorry, but could I ask a small follow-up question? If θj = 5 then should the jth element in θ also be 5? E.g - If on the left hand side θj= θ3 = 5 then should the θ on the right-hand-side of the equation be [θ0, θ1, θ2, 5, θ4, θ5, θ6]?

Which is at:

Meaning of a varaible for calculating the partial derviative of MSE cost function

Take care - you cannot apply analagous reasoning as you have in your comment above as for the question you asked previously, concerning $\mathbf{x}^{(i)}$.

Concerning:

Is $\theta_j$ in the equation a unique combination of bias and weights which is reflected in the vector $\boldsymbol{\theta}$ in the equation? Or is it a single parameter inside a vector?

The answer is the latter; $\frac{\partial \operatorname{MSE}(\boldsymbol{\theta})}{\partial \theta_j}$ is the partial derivative of the MSE with respect to a single, $j$-th element of the parameter vector $\boldsymbol{\theta}$. Whereas $\boldsymbol{\theta}$ on the RHS is a vector.

If that continues to remain unclear, and you are unable to see how equation (4.5) and equation (4.6) are connected in Geron's book, prompt and I will be happy to unpack the linear algebra.

To the best of my knowledge, I am unable to detect the presence of a typo.

$\endgroup$
0
$\begingroup$

Yes, and Yes. Let me explain.

In mathematics this is called a gradient, i.e. a vector. This means that you take a partial derivative over each parameter of a scalar function, which forms a vector. So, each of the elements is a partial derivative of the function over one parameter, but together they are a gradient.

Since it's a vector, it must be pointing somewhere. It does point in the direction of the greatest sensitivity to the change of the parameters of the function.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.