TL;DR: Specifying a model (a collection of restrictions over a sample space) specifies the model parameters. Specifying an estimation procedure adds additional number of restrictions (assumptions?). Do the parameters being estimated remain invariant given this additional number of restrictions?

I just read a really good exposition that helps interpret WLS coefficients:

Interpret regression coefficients after WLS

(Top answer, by Fg)

One of the lines really struck me with something I've been trying to understand for a while:

The key is in the way you say it -- OLS and WLS estimates of the model parameters, implying one set of population parameters being estimated by both estimators.

Fg's answer implied that model parameters are invariant to the estimation technique being used. Surely, however, the population error covariance matrix ($\Sigma$) is considered a member of the parameter space themselves, and the OLS and WLS specify different outcomes for these parameters. Or are we only interested in getting accurate estimates for $\beta$, the true values of which remain invariant?

To illustrate: given my data, I specify a linear regression model:

$$Y \sim N(\beta^T X, \Sigma )$$

So the parameters I want to find are $(\beta,\Sigma)$. According to Fg the parameters $\beta$ are identical whether or not I use OLS or WLS estimation.

But OLS adds an additional assumption that the covariance matrix follows a certain pattern:


Which the WLS procedure does not impose. Therefore the population parameter $\Sigma$ being estimated in both examples is likely to be different. How does this couch with Fg's statement?

Note: One could easily extend the above argument to state that the specification of any model causes this confusion, for example if you specified a Poisson regression, the values being estimated change (necessarily). That happens to be the broader issue I've been struggling with. How do the subjective assumptions that define the model/estimation procedure lead to objective targets for parameter estimation? Where does model specification/restriction end and assumptions for estimation begin?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.