Derivation of MLE of linear regression: and now? Why is there discrepancy to lm in R? I want to understand the ML Estimation of the linear model from top to bottom or vice versa ;-).
I totally get the part of formulating the LogLikelihood function and how to get the derivatives of beta and sigma... 
Now then I have the formula for beta=X'Y/XX' 
If I produce a simple data set in R:
x <- matrix(c(60, 50, 30, 120, 200, 70))
y <- matrix(c(8, 7, 5, 10, 11, 6))

I get the slope beta1 and the intercept beta0, to be:
b1 <- (t(x)%*%y)/(t(x)%*%x)
b1
0.07239819
b0 <- y[1]-x[1]*b1
b0
3.656109

If I compare that to the output of lm in R:
lm(y~x)
Coefficients:
(Intercept)            x  
    4.89393      0.03328  

So 1. why is there a discrepancy? 2. The fit of lm really is better if visualised in a plot. 
So what am I not getting there? I thought in my result there is the implicit assumption on $y\sim N(\mu,1)$ whereas in lm the sigma is also estimated. But I'm not sure if this is true.
So I read about the Hessian matrix, var-cov matrix. I don't know where to put them in this context...
Thanks
Franz
 A: In short, the discrepancy is because you're not doing it correctly. 
The least squares estimates of regression coefficients (which are ML at the normal) are $(X'X)^{-1}X'y$, where $X$ consists of a column of 1's beside a column of the independent variable. (In practice, you don't actually compute the inverse.)
However, your approach works for regression through the origin. Ordinary linear regression passes through $(\bar{x},\bar{y})$.
So if you do it by mean correcting first, your approach should work for simple linear regression (at least if you correct $XX'$ in your post to $X'X$ first):
step 1: mean correct x and y:
 x <- matrix(c(60, 50, 30, 120, 200, 70))
 y <- matrix(c(8, 7, 5, 10, 11, 6))
 xm <- x-mean(x)
 ym <- y-mean(y)

step 2: apply your approach:
 slope=crossprod(xm,ym)/crossprod(xm)  # a more efficient way to do your calculation
 intercept=mean(y)-slope*mean(x)
 print(c(intercept,slope),d=4)
[1] 4.89393 0.03328

A: Thanks a lot!
I understood the problem and your solution.
But I don't get how this could be computed using the $\textbf{X}$ 2xn matrix.
I don't know how to solve this for $\beta_0$ and $\beta_1$:
$$ \beta = \begin{bmatrix} \beta_0 \\ \beta_1 \end{bmatrix} = \mathbf{X}^T y [\mathbf{X}^T \mathbf{X}]^{-1}= \frac{\begin{bmatrix} 1& 1 & \dots &1 \\ x_1 & x_2 & \dots & x_n \end{bmatrix} \begin{bmatrix}y_1\\y_2\\ \vdots \\ y_n \end{bmatrix}} {\begin{bmatrix} 1& 1 & \dots &1 \\ x_1 & x_2 & \dots & x_n \end{bmatrix} \begin{bmatrix}x_1 \\x_2\\\vdots \\x_n \end{bmatrix}}$$
This should then go through the origin right?
I know that $\beta_0$ is set 0 when forced through the origin, but why do I need the 1 column in the $\textbf{X}$ matrix then?
Best,
Franz
A: The expression $\hat{\beta} = [X^TX]^{-1}X^Ty$ is a product of matrices. Matrix products mostly does not commute and the term $[X^TX]^{-1}$ is a matrix inverse. You can find information about this by doing a search on Linear Algebra.
Further the ones are always needed since the expression derives from $y = [1, x][\beta_0, \beta_1]^T$.
