1
$\begingroup$

I had this question when I read equation (C.20) in Appendix C of "Pattern Recognition and Machine Learning" written by Christopher M. Bishop. Here I copy the equation below for reference:

enter image description here

Capital bold $\mathbf A$ and $\mathbf B$ are matrices; so is their product. Lower case bold $\mathbf x$ is a column vector, according to the convention in this book. But, what is the derivative of a matrix with regard to a vector defined, like $\frac{\partial\mathbf A}{\partial\mathbf x}$? It seems that $\frac{\partial\mathbf A}{\partial\mathbf x}$ is still a matrix of the same size as $\mathbf A$. Otherwise, the subsequent matrix multiplication with $\mathbf B$ can't be conducted. But what are the elements of $\frac{\partial\mathbf A}{\partial\mathbf x}$? I have googled and read wiki page like this, but that page does not cover such a derivative type. If you happened to read this book before, can you please let me know the definition of the derivative of a matrix with regard to a vector in this equation? Thanks a lot.

Improve this question
$\endgroup$
8
  • 1
    $\begingroup$ Check out this thread math.stackexchange.com/questions/822068/… $\endgroup$
    – DevD
    Commented Jun 29, 2022 at 8:26
  • $\begingroup$ Related math.SE question on the product of a tensor and a matrix: math.stackexchange.com/questions/1953185/… $\endgroup$
    – B.Liu
    Commented Jun 29, 2022 at 8:59
  • $\begingroup$ So, for $\frac{\partial\mathbf A}{\partial\mathbf x}$, the author here actually means a tensor, and the subsequent multiplication with B is an n-mode product? $\endgroup$
    – zzzhhh
    Commented Jun 29, 2022 at 9:18
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/257579 for a general answer. The notation indicates $A$ and $B$ are matrix-valued functions. The derivative is defined by considering an $m\times n$ matrix as an element of $\mathbb{R}^{mn}$ and $x$ as an element of $\mathbb{R}^k.$ That makes "$A$" a function from $\mathbb{R}^p$ to $\mathbb{R}^{mn}$ and $B:\mathbb{R}^p\to\mathbb{R}^{np},$ say; and therefore $AB:\mathbb{R}^{mn}\times\mathbb{R}^{np}\to\mathbb{R}^{mp}.$ From this point on, you may consult any textbook on multivariable Calculus for definitions, examples, theorems, proofs, and so on. $\endgroup$
    – whuber ♦
    Commented Jun 29, 2022 at 15:01
  • $\begingroup$ @whuber ♦ So, $\frac{\partial\mathbf A}{\partial\mathbf x}$ should be a matrix, right? To be able to conduct matrix multiplications $\mathbf{AB}$ and $\frac{\partial\mathbf A}{\partial\mathbf x}\mathbf B$, the width of matrix $\frac{\partial\mathbf A}{\partial\mathbf x}$ must be the same as the width of matrix $\mathbf A$ and height of matrix $\mathbf B$ (n). But your definition of differentiability requires the width of $\frac{\partial\mathbf A}{\partial\mathbf x}$ be the dimension of $\mathbf x$ (k), which in general does not equal the width of A (n). How to reconcile this contradiction? $\endgroup$
    – zzzhhh
    Commented Jul 5, 2022 at 9:11

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.