5
$\begingroup$

What am I supposed to do when I want to interpret significances, although I know that standard errors are biased because of wrong error term assumptions? I know that there is the possibility to use White estimators, weighted OLS. But my prof told me not to do so.

Maybe some extra information:

1.) I am analyzing an OLS with a whole bunch of dummy variables.

3.) the assumption of normal distributed error terms is wrong (they are t-distributed) and heteroskedasticity occurs.

4.) I am doing cross sectional analysis

5.) I have a lot of Interaction in the model. And most of the variables are non significant (p-values around 0.8, so that the coefficients are close to zero). My prof doesn't want me to get rid of these variables, although they are not significant (He said that this is not a good way, because stepwise elimination is trouble because of deleting wrong variables and choosing the right criteria).

On the one hand I understand why there is no way to interpret the significances. But on the other hand it makes interpreting not easier. Sure I can change the model, but I have to do OLS first, before I am allowed to switch!

|cite|improve this question|||||
$\endgroup$
  • 1
    $\begingroup$ Could you elaborate on the models that you are considering, and how the error term assumptions deviate from the data? $\endgroup$ – Jeromy Anglim Jun 3 '11 at 7:42
  • $\begingroup$ Why your professor is against the heteroscedasticity corrections? Well, in general for data mining you can assume much higher not-elimination criteria of $0.5$, so stepwise deletion will leave more of important information. Though I guess that you have something structural, i.e. a priori expectations of signs and what variables should cause the dependent one. $\endgroup$ – Dmitrij Celov Jun 3 '11 at 14:32
  • $\begingroup$ No there're no a priori assumptions or expectations. I mean what I'd do is to estimate robust standard errrors and then just analyze the significant variables, but this is not what I should do... $\endgroup$ – MarkDollar Jun 3 '11 at 15:08
1
$\begingroup$

You can't interpret the $p$-values. The long-tailed errors you're describing often act to underestimate the standard errors, making your $p$-values too small (not to mention that $\hat{\beta}$ isn't normally distributed in finite samples). I suggest a non-parametric bootstrap so you can characterize the sampling distribution of your coefficient estimates without making an unwarranted assumptions about the error distribution.

|cite|improve this answer|||||
$\endgroup$
  • $\begingroup$ Hmm how can you characterize the sampling distribution with bootstrap methodes? Would you be so kind to describe this for an amateur? Thanks alot $\endgroup$ – MarkDollar Jun 4 '11 at 12:38
  • $\begingroup$ The basic idea is to view your data set as an estimate of the distribution of $(Y,X)$. Therefore, by re-sampling from your data set and recalculating $\beta$ each time, you can estimate the sampling distribution of $\hat{\beta}$. Have a look at en.wikipedia.org/wiki/Bootstrapping_%28statistics%29 for more info. In your situation I think it would suffice to re-sample with replacement $n$ points from your data set, calculate $\beta$, and repeat a large number times - say, 1000. At the end, you can use the quantiles of those 1000 points to give approximate confidence intervals for $\beta$. $\endgroup$ – Macro Jun 4 '11 at 15:56
0
$\begingroup$

Regarding point 5.): I don't know exactly what you're working on, but generally, keeping nonsignificant terms (when they're not of genuine prior interest) is also not good practise. It leads to overfitting and meaningless results.

You should consider potentially important interactions, but if they aren't significant, they need to go. Although you should keep nonsignificant main effects if they're involved in significant interactions.

There are less subjective alternatives to stepwise deletion in any case, like searches for the most parsimonious model given a set of variables. In R, packages like AICcmodavg, MuMIn, or glmulti might be able to help you with that.

There are some good papers out there on model selection. If you're a biologist, try this one: http://faculty.washington.edu/skalski/classes/QERM597/papers/Johnson%20and%20Omland.pdf

|cite|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.