I have a basic question. MLE/LMSSE is introduced as follows: $$Y = H\theta + W$$ where $H$ is the linear model matrix, $W$ is measurement noise (let's assume it is normal so MLE = LMSSE). $\theta$ is the vector of parameters.

The ML estimate is well-known: $(H'H)^{-1}H'\bf{Y} = \widehat{\theta}\quad\quad$ ($\bf{Y}$ is the vector of observations.)

Let's say I am only interested estimating one of the parameters (say the first parameter).
Is it equal to the first entry of $\widehat{\theta}$?

More generally is the ML estimate of $S\theta$ where $S$ is some low-rank matrix the same as $S\widehat{\theta}$? I do not think that is the case, but I don't know how exactly to deal with it.

  • 1
    $\begingroup$ Cross posted here: math.stackexchange.com/questions/858695/… $\endgroup$ – Kirill Jul 7 '14 at 7:29
  • $\begingroup$ I suggested to close the cross-posted question above on math (first voter). We should try to not close it on both sites. $\endgroup$ – gnometorule Jul 7 '14 at 7:52

The posterior distribution over $\theta$ is Gaussian here.

When you apply the linear function $\theta\mapsto S\theta$ to it, the new random variable $S\theta$ is still Gaussian, so its MLE is just its expectation, and $\mathbb{E}[S\theta] = S\mathbb{E}[\theta]$.

  • $\begingroup$ Thanks... I dont have a good theoretical background in estimation theory. Let me try to rework this one with this pointer. I will get back to you. $\endgroup$ – user161378 Jul 7 '14 at 15:55
  • $\begingroup$ I think I am getting a little confused. As I understand the linear model changes. I guess it would help if you could point me to some tutorial on MLE. or a proof :) THanks $\endgroup$ – user161378 Jul 7 '14 at 18:57
  • $\begingroup$ @user161378 The way you phrased the question is that you estimate the whole model (all $\theta$'s), then take a subset, which is what I answered. If instead you change your model to only consider one of the parameters, then of course the estimate will be different; the new model matrix will be just the first column of the original matrix. $\endgroup$ – Kirill Jul 7 '14 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.