Is there a way to estimate beta from regression analysis other than the mele estimation? Or is there any other distribution regression where the closed form solution exists other than normal regression?
1
-
$\begingroup$ en.wikipedia.org/wiki/Linear_regression#Estimation_methods $\endgroup$– user2974951Commented Apr 7, 2022 at 10:05
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Only under an assumption of a normal error term does ordinary least squares coincide with maximum likelihood estimation, so if you assume a different distribution of the errors (or make no assumption at all) yet run through the usual $\hat\beta_{ols}=(X^TX)^{-1}X^Ty$ calculation, you’re not doing maximum likelihood estimation.
Even if you do assume a Gaussian error term, there are alternatives to the ordinary least squares (equivalent to maximum likelihood) estimator. One such option is ridge regression, which does have a closed-form solution. Let $I$ be the identity matrix with the same dimensions as $X^TX$, and let $k>0$ be a number of your choosing (perhaps by some kind of cross validation to find the value that gives the best out-of-sample cross validation performance). Then the ridge regression estimator is:
$$ \hat\beta_{ridge,k}= (X^TX+kI)^{-1}X^Ty $$
Note that the ridge estimator is a biased estimator. While “bias” as a technical term in statistics does not have the negative connotations that the word often has in colloquial English, it is worth being aware of this fact.
