Timeline for When would least squares be a bad idea?
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 4, 2014 at 23:16 | comment | added | Dennis | A singular design ($\textbf{X}$) matrix is not necessarily a problem for least squares. It is symptomatic of an over parameterized model. This certainly affects the inferences that are possible (in that only linear combinations of $\beta$ that belong to the column space of $\textbf{X}$ can be estimated uniquely). This is the case in ANOVA models. Of course, one can also choose to reparameterize the model to have a full (column) rank design matrix. | |
| Jan 21, 2014 at 18:48 | comment | added | Manuel | Brian, thanks so much for answering. I decided to acept Bill's answer because it was easier to understand what was happening. | |
| Jan 21, 2014 at 17:02 | comment | added | Manuel | I belive that your second comment is what i am looking for. It just bothers me the low probability of happening. Also from that example it's pretty clear how to build a better estimator when knowing the distribution of the errors. | |
| Jan 20, 2014 at 23:29 | comment | added | Brian Borchers | If you want something really weird, consider using an appropriately scaled Student's t distribution with 4 degrees of freedom. This is a well known distribution with finite mean and variance but unbounded fourth moment. Now, suppose that you have 1 observation with $X=1$, and $\beta=0$. The distribution of $\hat{\beta}$ will have finite mean and variance but unbounded fourth moment. | |
| Jan 20, 2014 at 23:26 | comment | added | Brian Borchers | Consider the distribution where $\epsilon_{i}=0$ with probability 0.9999, and $\epsilon_{i}=100$ with probability $0.00005$ and $\epsilon_{i}=-100$ with probability $0.00005$. Now, suppose $X=I$ (the y's are just direct measurements of the unknown parameter $\beta$), and you have about 100 observations. It's most likely that your estimate will be perfect, but there's a significant possibility of an estimate that includes one of the rare bad $\epsilon$ values and is off as a result. | |
| Jan 20, 2014 at 23:20 | comment | added | Manuel | Besides, Gauss-Markov ensure least squares be a minimum variance unbiased estimator among linear. May be linear estimators are not reazonable for some kind of distributions. That's what i want to understand. | |
| Jan 20, 2014 at 23:19 | comment | added | Manuel | What would be an extreme distribution of $\varepsilon$ ? Remember it has identity covariance matrix. | |
| Jan 20, 2014 at 23:10 | history | answered | Brian Borchers | CC BY-SA 3.0 |