# Rewrite linear expression

Is it possible to rewrite $$\frac{-1}{2}\left(x^T\Gamma^{-1}(\mu_1-\mu_0)+(\mu_1-\mu_0)^T\Gamma^{-1}x\right)$$ as $$-\theta^Tx$$ where $$x,\mu\in\mathbb{R}^d, \Gamma\in\mathbb{R}^{d\times d}$$ and $$\theta$$ is some function of $$\mu,\Gamma?$$

Tried to rewrite as $$\sum x_i\Gamma^{-1}_i\mu_i + \sum \mu_i\Gamma^{-1}_ix_i$$ $$\sum x_i\Gamma^{-1}_i\mu_i + \mu_i\Gamma^{-1}_ix_i$$ $$2\sum \Gamma^{-1}_i\mu_ix_i$$ where $$\mu_i=(\mu_1-\mu_0)$$ because $$x_i, \mu_i$$ are scalars so it doesn't matter the order in which it multiplies the ith row of $$\Gamma$$?

• What have you tried so far? Mar 14 at 3:31
• I omitted all the work up to this since it's for a hw assignment. The question was to show that, for a gaussian distribution, $p(y=1|x)=\frac{1}{1+exp(-\theta^Tx+\theta_0}$. The expression I included is part of what I derived and what I assume to be relevant for the $-\theta^Tx$ part.
– jroc
Mar 14 at 3:38
• @ShawnHemelstrand edited
– jroc
Mar 14 at 4:03

Here $$\Gamma$$ is a $$d\times d$$ matrix, and $$x,\mu$$ are vectors of length $$d$$. When using matrix algebra it is common convention to express vectors as column vectors. Therefore, we will think of $$x$$ and $$\mu$$ as matrices of size $$d\times 1$$.

Therefore, the product $$\Gamma^{-1}(\mu_1-\mu_0)$$ is a matrix product of size $$(d\times d)(d\times 1)$$, i.e. it is a matrix of size $$d\times 1$$. In your expression you have $$x^t\Gamma^{-1}(\mu_1-\mu_0)$$, therefore the matrix product is $$(1\times d)(d\times d)(d\times 1)$$, this is because you wrote $$x^t$$, so your replace $$x$$, which is assumed to be in column form, into row form and so the dimension numbers get flipped.

Now, $$x^t\Gamma^{-1}(\mu_1-\mu_0)$$ is a scalar, because it is a $$(1\times 1)$$ matrix, i.e. a number. So, in particular, it is equal to its own transpose (the transpose of a $$(1\times 1)$$ matrix is itself): $$\left( x^t \Gamma^{-1}(\mu_1 - \mu_0) \right)^t = x^t \Gamma^{-1}(\mu_1 - \mu_0)$$ But, at the same time, let us see what happens if we use the transpose properties'':

• Transpose of a product is the reverse-product of the transposes
• Transpose of an inverse is the inverse of the transpose
• Transpose of the transpose is the original matrix

Therefore, by using these three, $$\left( x^t \Gamma^{-1}(\mu_1 - \mu_0) \right)^t = (\mu_1 - \mu_0)^t (\Gamma^{-1})^t (x^t)^t = (\mu_1 - \mu_0)^t (\Gamma^t)^{-1} x$$

It appears that the missing information is that $$\Gamma$$ is symmetric, i.e. $$\Gamma^t = \Gamma$$. If we accept the symmetric assumption then we have derived that, $$x^t \Gamma^{-1}(\mu_1 - \mu_0) = (\mu_1 - \mu_0)^t \Gamma^{-1} x$$ From here you can substitute this in and get the $$\theta$$ you are looking for.

• Oh I see. Thanks for the nice explanation!
– jroc
Mar 14 at 4:31
• Because the original expression is manifestly a linear function of $x,$ there is no necessity for $\Gamma$ to be symmetric: coefficients $\theta$ can still be found (indeed, they can be read directly off the original expression).
– whuber
Mar 14 at 16:25