1
$\begingroup$

Is it possible to rewrite $$ \frac{-1}{2}\left(x^T\Gamma^{-1}(\mu_1-\mu_0)+(\mu_1-\mu_0)^T\Gamma^{-1}x\right) $$ as $$ -\theta^Tx $$ where $x,\mu\in\mathbb{R}^d, \Gamma\in\mathbb{R}^{d\times d}$ and $\theta$ is some function of $\mu,\Gamma?$

Tried to rewrite as $$ \sum x_i\Gamma^{-1}_i\mu_i + \sum \mu_i\Gamma^{-1}_ix_i $$ $$ \sum x_i\Gamma^{-1}_i\mu_i + \mu_i\Gamma^{-1}_ix_i $$ $$ 2\sum \Gamma^{-1}_i\mu_ix_i $$ where $\mu_i=(\mu_1-\mu_0)$ because $x_i, \mu_i$ are scalars so it doesn't matter the order in which it multiplies the ith row of $\Gamma$?

$\endgroup$
3
  • $\begingroup$ What have you tried so far? $\endgroup$ Commented Mar 14, 2023 at 3:31
  • $\begingroup$ I omitted all the work up to this since it's for a hw assignment. The question was to show that, for a gaussian distribution, $p(y=1|x)=\frac{1}{1+exp(-\theta^Tx+\theta_0}$. The expression I included is part of what I derived and what I assume to be relevant for the $-\theta^Tx$ part. $\endgroup$
    – jroc
    Commented Mar 14, 2023 at 3:38
  • $\begingroup$ @ShawnHemelstrand edited $\endgroup$
    – jroc
    Commented Mar 14, 2023 at 4:03

1 Answer 1

2
$\begingroup$

Here $\Gamma$ is a $d\times d$ matrix, and $x,\mu$ are vectors of length $d$. When using matrix algebra it is common convention to express vectors as column vectors. Therefore, we will think of $x$ and $\mu$ as matrices of size $d\times 1$.

Therefore, the product $\Gamma^{-1}(\mu_1-\mu_0)$ is a matrix product of size $(d\times d)(d\times 1)$, i.e. it is a matrix of size $d\times 1$. In your expression you have $x^t\Gamma^{-1}(\mu_1-\mu_0)$, therefore the matrix product is $(1\times d)(d\times d)(d\times 1)$, this is because you wrote $x^t$, so your replace $x$, which is assumed to be in column form, into row form and so the dimension numbers get flipped.

Now, $x^t\Gamma^{-1}(\mu_1-\mu_0)$ is a scalar, because it is a $(1\times 1)$ matrix, i.e. a number. So, in particular, it is equal to its own transpose (the transpose of a $(1\times 1)$ matrix is itself): $$ \left( x^t \Gamma^{-1}(\mu_1 - \mu_0) \right)^t = x^t \Gamma^{-1}(\mu_1 - \mu_0) $$ But, at the same time, let us see what happens if we use the ``transpose properties'':

  • Transpose of a product is the reverse-product of the transposes
  • Transpose of an inverse is the inverse of the transpose
  • Transpose of the transpose is the original matrix

Therefore, by using these three, $$ \left( x^t \Gamma^{-1}(\mu_1 - \mu_0) \right)^t = (\mu_1 - \mu_0)^t (\Gamma^{-1})^t (x^t)^t = (\mu_1 - \mu_0)^t (\Gamma^t)^{-1} x$$

It appears that the missing information is that $\Gamma$ is symmetric, i.e. $\Gamma^t = \Gamma$. If we accept the symmetric assumption then we have derived that, $$ x^t \Gamma^{-1}(\mu_1 - \mu_0) = (\mu_1 - \mu_0)^t \Gamma^{-1} x$$ From here you can substitute this in and get the $\theta$ you are looking for.

$\endgroup$
2
  • $\begingroup$ Oh I see. Thanks for the nice explanation! $\endgroup$
    – jroc
    Commented Mar 14, 2023 at 4:31
  • $\begingroup$ Because the original expression is manifestly a linear function of $x,$ there is no necessity for $\Gamma$ to be symmetric: coefficients $\theta$ can still be found (indeed, they can be read directly off the original expression). $\endgroup$
    – whuber
    Commented Mar 14, 2023 at 16:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.