0
$\begingroup$

I have a basic question. MLE/LMSSE is introduced as follows: $$Y = H\theta + W$$ where $H$ is the linear model matrix, $W$ is measurement noise (let's assume it is normal so MLE = LMSSE). $\theta$ is the vector of parameters.

The ML estimate is well-known: $(H'H)^{-1}H'\bf{Y} = \widehat{\theta}\quad\quad$ ($\bf{Y}$ is the vector of observations.)

Let's say I am only interested estimating one of the parameters (say the first parameter).
Is it equal to the first entry of $\widehat{\theta}$?

More generally is the ML estimate of $S\theta$ where $S$ is some low-rank matrix the same as $S\widehat{\theta}$? I do not think that is the case, but I don't know how exactly to deal with it.

Improve this question
$\endgroup$
2
  • 1
    $\begingroup$ Cross posted here: math.stackexchange.com/questions/858695/… $\endgroup$
    – Kirill
    Commented Jul 7, 2014 at 7:29
  • $\begingroup$ I suggested to close the cross-posted question above on math (first voter). We should try to not close it on both sites. $\endgroup$
    – gnometorule
    Commented Jul 7, 2014 at 7:52

1 Answer 1

Reset to default
0
$\begingroup$

The posterior distribution over $\theta$ is Gaussian here.

When you apply the linear function $\theta\mapsto S\theta$ to it, the new random variable $S\theta$ is still Gaussian, so its MLE is just its expectation, and $\mathbb{E}[S\theta] = S\mathbb{E}[\theta]$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks... I dont have a good theoretical background in estimation theory. Let me try to rework this one with this pointer. I will get back to you. $\endgroup$
    – user161378
    Commented Jul 7, 2014 at 15:55
  • $\begingroup$ I think I am getting a little confused. As I understand the linear model changes. I guess it would help if you could point me to some tutorial on MLE. or a proof :) THanks $\endgroup$
    – user161378
    Commented Jul 7, 2014 at 18:57
  • $\begingroup$ @user161378 The way you phrased the question is that you estimate the whole model (all $\theta$'s), then take a subset, which is what I answered. If instead you change your model to only consider one of the parameters, then of course the estimate will be different; the new model matrix will be just the first column of the original matrix. $\endgroup$
    – Kirill
    Commented Jul 7, 2014 at 19:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.