(This question is an attempt to zoom in on the key issue in this question using as little information as possible.)

Lets say I want to derive the likelihood function of $\beta$ given $x$ and $y$ for the model $$y=x\beta+u$$ where $$u\sim NID(0,1).$$ I would start with these steps: \begin{align} \mathcal{L}(\beta\mid x,y) &=f_{XY}(x, y \mid \beta)\\ &=f_u(y=x\beta + u\mid\beta, x, y). \end{align} I think that I need the second step to continue the derivation, but I don't know how to motivate it. I take it entirely on intuition. Is it correct? If so, how do I justify it? If not, how would the next few steps look like?

Edit: Just to give a sense of where I'm going with this, the rest of the derivation would look like this: \begin{align} &=f_u(y-x\beta=u\mid\beta, x, y)\\ &=\varphi(y-x\beta). \end{align}

  • $\begingroup$ Bayes: $P(\beta|x,y)P(x,y)=P(x,y|\beta)P(\beta)$, so you say $P(\beta|x,y)\sim P(x,y|\beta)P(\beta)$. If you don't know anything about $\beta$, you can say $P(\beta|x,y)\sim P(x,y|\beta)$ and maximize it. $\endgroup$
    – Aksakal
    May 27, 2015 at 13:03
  • $\begingroup$ @Aksakal I'm not entirely sure how that relates to what I'm doing here. How would that let me impose the model structure on the probability? (It's not that I don't know what to do after the last step in the question. It's just that I don't know how to motivate that last step.) $\endgroup$
    – Fredrik P
    May 27, 2015 at 13:52
  • 1
    $\begingroup$ What you call "model structure" in practical terms is simply a function that allows you imply the residuals from the data set given parameters, in other words that is essentially your likelihood function $\endgroup$
    – Aksakal
    May 27, 2015 at 13:55
  • $\begingroup$ Closely related threads, such as stats.stackexchange.com/questions/49443 and stats.stackexchange.com/questions/32103, illustrate the general procedure. It's not perfectly clear what you're trying to ask, though. It sounds at some points like you are requesting an exposition of Maximum Likelihood theory and at other points like you just want to derive ordinary least squares from a Normal likelihood. Could you edit this post to make it more apparent what kind of answers you are seeking? $\endgroup$
    – whuber
    May 27, 2015 at 19:34
  • $\begingroup$ @whuber I guess it is more in line of an exposition of Maximum Likelihood theory that I'm requesting (but perhaps without the Maximum part--I don't think that I'm maximizing anything here). I have tried to make it more apparent what kind of answer I'm seeking. Was it at all improved? $\endgroup$
    – Fredrik P
    May 27, 2015 at 19:59


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.