2
$\begingroup$

I was checking out Genarlised Least Squares on Wikipedia and wondered if it had anything to do with Generalised Linear Models, which I have been using for quite a while. The Wikipedia showed me that one was a model and the other a method of estimation (see picture below). Although I perhaps at a very basic level understand the difference (I guess one is about the relation you describe, where the other is about how you calculate the coefficients), I am wondering if anyone would be willing to give a more elaborate explanation about models and estimation and how they relate. As an example, are some estimation methods exclusively connected to some models? And does GLM have any relation to GLS?

enter image description here

$\endgroup$

1 Answer 1

2
$\begingroup$

Yeah I definitely get the confusion. Generalized Linear Models estimate the parameters via maximizing likelihood which, when assuming normality, is equal to OLS estimated typically via the closed solution:enter image description here

GLS on the other hand is there to address certain inefficiencies. Directly from the wiki:

"In statistics, generalized least squares (GLS) is a technique for estimating the unknown parameters in a linear regression model when there is a certain degree of correlation between the residuals in a regression model. In these cases, ordinary least squares and weighted least squares can be statistically inefficient, or even give misleading inferences."

One example is heteroskedasticity and using WLS to address it. WLS is just a specific case of GLS. But basically all we do with GLS is to introduce a weighting matrix in our OLS formula which can, in the presence of these inefficiencies, attempt to reconcile the problems and allow us to interpret the results as usual. So GLS introduces this omega: enter image description here And depending on the problem we can adjust how omega is calculated.

To answer your larger question: Yes, a lot of methods are exclusively related to certain models because of the nature of the model. Methods are ways to estimate the parameters of the model like the coefficients, but models don't always have the same parameters or the same assumptions on the nature of the data. For example, we can't use OLS to solve for the coefficients of a Logistic regression.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.