I'm trying to find out how log-likelihood function works for linear regression. I found the formula here and here. Making some experiments with it (see code below), I was quite surprised that the likelihood uses
SSE/n instead of
MSE was used everywhere up to now! I thought MSE is much better estimator of $\sigma^2$ mentioned in the formula in the 1st resource (page 6) - the actual residual variance. But the 2nd resource and my experiment clearly states that $\sigma^2$ is defined as
SSE/n (where n is length of the outcome variable vector).
Here is the code to play with:
set.seed(128) y = c(rnorm(200, 20, 4), rnorm(300, 30, 4), rnorm(400, 40, 4), rnorm(500, 50, 4)) cat1 = as.factor(c(rep(1, 200), rep(2, 300), rep(3, 400), rep(4, 500))) rand_order = sample(1:length(cat1)) cat2 = cat1[rand_order] cat2y = c(rep(1, 200), rep(-2, 300), rep(3, 400), rep(-4, 500)) y = y + cat2y[rand_order] m1 = lm(y ~ 0 + cat1 + cat2) # logLik using residual degrees of freedom (-3941.94): -length(m1$model$y)/2*log(2*pi) - length(m1$model$y)/2*log(sum((m1$residual)^2)/m1$df.residual) - 1/2*m1$df.residual # logLik using N (-3941.931) -length(m1$model$y)/2*log(2*pi) - length(m1$model$y)/2*log(sum((m1$residual)^2)/length(m1$model$y)) - 1/2*length(m1$model$y) # real logLik (-3941.931) logLik(m1)