# Understanding Equation (3.13) from Bishop's Pattern Recognition and Machine Learning

I'm having trouble deriving the mentioned equation from the text. Specifically, I'm having trouble obtaining the $$\mathbf{\phi}(x_n)^T$$ term. I checked the errata and while the lack of the $$\beta$$ parameter is mentioned there is nothing about this term.

Given that $$\ln\left(p(\mathbf{t}|\mathbf{w}, \beta)\right) = \frac{N}{2} \ln(\beta) - \frac{N}{2} \ln(2 \pi) - \beta \frac{1}{2} \sum\limits_{i=1}^{N} \left(t_i - \mathbf{w}^T \mathbf{\phi}(\mathbf{x}_i) \right)^2.$$ The author states that $$\nabla \ln(p(\mathbf{t}|\mathbf{w}, \beta) ) = \beta \sum\limits_{i=1}^{N} \left(t_i - \mathbf{w}^T \mathbf{\phi}(\mathbf{x}_i) \right) \mathbf{\phi}(\mathbf{x}_i)^T,$$ where $$\nabla=(\partial w_0, \dots, \partial w_M)^T$$, $$\mathbf{\phi}(\mathbf{x}_i) = (\phi_0(\mathbf{x}_i), \dots, \phi_M(\mathbf{x}_i))^T$$ and $$\mathbf{w}=(w_0, \dots, w_M)^T$$.

However, when I expand the inner product and take the gradient I obtain \begin{aligned} \nabla \ln(p(\mathbf{t}|\mathbf{w}, \beta) &= -\beta \frac{1}{2} \sum\limits_{i=1}^{N} \nabla \left(t_i - \mathbf{w}^T \mathbf{\phi}(\mathbf{x}_i) \right)^2 \\ &= -\beta \frac{1}{2} \sum\limits_{i=1}^{N} \nabla \left(t_i - w_0 \phi_0 - \dots - w_M \phi_M \right)^2 \\ &= -\beta \sum\limits_{i=1}^{N} \left(t_i - w_0 \phi_0 - \dots - w_M \phi_M \right) \left(-\phi_0, -\phi_1, \dots, -\phi_M \right)^T \\ &= \beta \sum\limits_{i=1}^{N} \left(t_i - \mathbf{w}^T \mathbf{\phi}(\mathbf{x}_i) \right) \mathbf{\phi}(\mathbf{x}_i) \end{aligned} Can somebody please illustrate where the error comes from?

If denominator layout is used while taking derivatives, then $$\mathbf \phi(x)$$ is correct; if numerator layout is used, $$\phi(x)^T$$ is correct. And, the author doesn't specify which one is used, but as it appears, it implicitly chooses the numerator layout. So, there is no right or wrong answer. Same follow-up equations can be derived by manipulating both expressions.
$$0 = \sum\limits_{i=1}^{N} \left(t_i - \mathbf{w}^T \mathbf{\phi}(\mathbf{x}_i) \right) \mathbf{\phi}(\mathbf{x}_i)^T \rightarrow0=\sum_i t_i\phi(\mathbf x_i)^T-\mathbf{w}^T\sum_i (\phi(\mathbf x_i)\phi(\mathbf x_i)^T)$$
For the other version, you can have $$0 = \sum\limits_{i=1}^{N} \left(t_i - \mathbf{w}^T \mathbf{\phi}(\mathbf{x}_i) \right) \mathbf{\phi}(\mathbf{x}_i)=\sum\limits_{i=1}^{N} \mathbf{\phi}(\mathbf{x}_i)\left(t_i - \mathbf{\phi}(\mathbf{x}_i)^T\mathbf{w} \right)\\\rightarrow0=\sum_i t_i\phi(\mathbf x_i)-\left(\sum_i\phi(\mathbf x_i)\phi^T(\mathbf x_i)\right)\mathbf{w}$$
The error is that the left side of (3.13) should actually be the transpose of the gradient vector which is defined to be a column vector, for the purpose of constructing $$\bf{\Phi}^T\bf{\Phi}$$.