I want to write the expression for covariance matrix of following linear model

$$x= As+ v$$ Where $A\in R^{N\times p}, s\in R^{p\times 1}$, $v$ consists of MVN noise only with mean $Bz$ and covariance $R=\sigma^2I$, $v\sim \mathcal{N}[Bz, \sigma^2I ]$: $B\in R^{N\times t}, \phi\in R^{t\times 1}, t<N-p$. Now the matrices $A$ and $B$ are linearly independent, i.e $A^TB \neq 0 $ and each matrix is full rank.

Although, I know how to write the covariance matrix expression for following signal $x$ which obeys linear subspace model

$$x=As+n$$ Where $s$ is independent of noise,then covariance $C=AE[ss^T]A^T+\sigma^2I$ for Gaussian noise of zero mean. What if I write the covariance matrix expression for first model as $C=AE[ss^T]A^T+ BE[zz^T]B^T+\sigma^2I$, is it the right way ?

Please suggest as I can not find a reference.

Appreciate your suggestions!

  • $\begingroup$ Much is unstated in your question; presumably $s$ and $z$ are zero-mean, and you imply that $s$ and $n$ at least are independent; you should make such things explicit. Note that Var(X+Y) = Var(X) + Var(Y) + Cov(X,Y) + Cov(Y,X) (or see Wikipedia for properties of covariance) -- so it depends on whether $E[sz^T] \neq 0$ (or if $z$ is correlated with $n$); if any of those are the case, you'll have more terms. I think this is a duplicate, so I'm trying to locate one. $\endgroup$ – Glen_b -Reinstate Monica Feb 1 '17 at 3:53
  • $\begingroup$ @Glen_b, I have edited my question and added detail. Can you please have a look again? $\endgroup$ – Vendetta Feb 1 '17 at 4:50

Your statement for the distribution of $v$ should be $v|z\sim\mathcal{N}[Bz,\sigma^2I]$. That is, you specified conditional distribution of $v$ on $z$. If you condition on $z$, the first model is as easy as the second model to describe. The conditional variance of $x$ given $z$ is $$Var(x|z)=A\mathbb{E}[ss^T|z]A^T + \sigma^2I$$ Since you want the unconditional variance, you can apply the familiar formula there. $$Var(x)=E[Var(x|z)]+Var(E[x|z]) = A\mathbb{E}[ss^T]A^T + \sigma^2I + Var(A\mathbb{E}[s|z]+Bz)$$ Similar to what you got, right? Only if you assume that $s$ and $z$ are independent, or more directly $E[s|z]$ is constant (which is a weaker condition), then your formula is right.

[EDIT] You can't do much about the term $Var(A\mathbb{E}[s|z]+Bz)$. You can try to expand as $$AVar(\mathbb{E}[s|z])A^T + ACov(\mathbb{E}[s|z],z)+ Cov(z,\mathbb{E}[s|z])A^T + B\mathbb{E}[zz^T]B^T$$ which has the term you came up with. This also shows that a sufficient condition for your solution being right is that $\mathbb{E}[s|z]=C$, where $C$ is a constant.

| cite | improve this answer | |
  • $\begingroup$ What this last is representing $Var(AE[s|z]+Bz)$, is it equivalent to what I wrote $BE[zz^T]B^T$? Can you please elaborate a little more? $\endgroup$ – Vendetta Feb 1 '17 at 10:25
  • $\begingroup$ Please see the edit. $\endgroup$ – Julius Feb 1 '17 at 17:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.