0
$\begingroup$

I'm wondering if anyone could provide some code (preferably in R) which demonstrates violated assumptions leading to type 1 errors. Some concrete examples of errors arising from assumption violations would be useful teaching tools.

I've generated 10,000s of null regression models (slope 0), with major violations of some of the assumptions---such as non-independent residuals, or serious heteroskedacity---but my type 1 error rate never rises above .05. As far as I can tell regression (at least simple regression) seems to be totally 'robust' to assumption violations. Please somebody show me how things can go wrong!

Thanks,

$\endgroup$
2
  • $\begingroup$ You get type I errors whether the assumptions are violated or not. Do you mean "an increased rate of type I errors" or "a different rate of type I errors than you'd expect if the assumptions were true" or something else? $\endgroup$ – Glen_b Sep 26 '14 at 8:36
  • 1
    $\begingroup$ Can you show what you did? $\endgroup$ – Glen_b Sep 26 '14 at 8:37
1
$\begingroup$

Here's one for heteroskedasticity:

N = 100000
X = runif(N)*10
beta = 0#The true beta is 0
Y = beta*X+rnorm(N)*X 

D = data.frame(X,Y)
pr = c()
sf = c()
for (i in 1:10000){
    d = D[rownames(D) %in% sample(rownames(D),200),]
    m= lm(Y~X,data=d)
    sf[i] = summary(m)[[4]][2,4]<.05
    pr[i] = mean(sf)
    plot(pr,cex=0,ylim=c(0,1))
    lines(pr,col='red')
    abline(h=.05)
    }

You don't need to let the whole thing play out to 10,000 runs -- just stop it once you feel convinced.

$\endgroup$
1
$\begingroup$

Here is one simulation using Stata using skewed error terms ($\chi^2(2)$) error terms, showing that you need a small number of observations before that matters.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.