Rewrite linear expression

Question

Is it possible to rewrite $$ \frac{-1}{2}\left(x^T\Gamma^{-1}(\mu_1-\mu_0)+(\mu_1-\mu_0)^T\Gamma^{-1}x\right) $$ as $$ -\theta^Tx $$ where $x,\mu\in\mathbb{R}^d, \Gamma\in\mathbb{R}^{d\times d}$ and $\theta$ is some function of $\mu,\Gamma?$

Tried to rewrite as $$ \sum x_i\Gamma^{-1}_i\mu_i + \sum \mu_i\Gamma^{-1}_ix_i $$ $$ \sum x_i\Gamma^{-1}_i\mu_i + \mu_i\Gamma^{-1}_ix_i $$ $$ 2\sum \Gamma^{-1}_i\mu_ix_i $$ where $\mu_i=(\mu_1-\mu_0)$ because $x_i, \mu_i$ are scalars so it doesn't matter the order in which it multiplies the ith row of $\Gamma$?

Answer 1

Here $\Gamma$ is a $d\times d$ matrix, and $x,\mu$ are vectors of length $d$. When using matrix algebra it is common convention to express vectors as column vectors. Therefore, we will think of $x$ and $\mu$ as matrices of size $d\times 1$.

Therefore, the product $\Gamma^{-1}(\mu_1-\mu_0)$ is a matrix product of size $(d\times d)(d\times 1)$, i.e. it is a matrix of size $d\times 1$. In your expression you have $x^t\Gamma^{-1}(\mu_1-\mu_0)$, therefore the matrix product is $(1\times d)(d\times d)(d\times 1)$, this is because you wrote $x^t$, so your replace $x$, which is assumed to be in column form, into row form and so the dimension numbers get flipped.

Now, $x^t\Gamma^{-1}(\mu_1-\mu_0)$ is a scalar, because it is a $(1\times 1)$ matrix, i.e. a number. So, in particular, it is equal to its own transpose (the transpose of a $(1\times 1)$ matrix is itself): $$ \left( x^t \Gamma^{-1}(\mu_1 - \mu_0) \right)^t = x^t \Gamma^{-1}(\mu_1 - \mu_0) $$ But, at the same time, let us see what happens if we use the ``transpose properties'':

• Transpose of a product is the reverse-product of the transposes
• Transpose of an inverse is the inverse of the transpose
• Transpose of the transpose is the original matrix

Therefore, by using these three, $$ \left( x^t \Gamma^{-1}(\mu_1 - \mu_0) \right)^t = (\mu_1 - \mu_0)^t (\Gamma^{-1})^t (x^t)^t = (\mu_1 - \mu_0)^t (\Gamma^t)^{-1} x$$

It appears that the missing information is that $\Gamma$ is symmetric, i.e. $\Gamma^t = \Gamma$. If we accept the symmetric assumption then we have derived that, $$ x^t \Gamma^{-1}(\mu_1 - \mu_0) = (\mu_1 - \mu_0)^t \Gamma^{-1} x$$ From here you can substitute this in and get the $\theta$ you are looking for.