Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 2 down vote favorite

What is the canonical example which show situation when robust linear regression has advantage over least square linear regression ? I was trying to simulate situation when some errors (20% of them) are generated from t-student distribution and 80% are from normal - both distribution with the same variance ! on datasets with 50 observation, and I cant see clearly that robust regression is better, here is my R code for this experiment :

library(MASS)
n=50 # size of datasets
N=1000 # number of regressions
wynik=matrix(0,N,2) # matrix with estimated coefficients
v=5  # parametr of t-student distribution
Sd=(v/(v-2))^.5 # standard deviation of gaussian distribution
a=1 # coefficient 

for(i in 1:N){

x=rnorm(n,mean=1,sd=Sd)

e_norm<-rnorm(n,sd=Sd)
e_t<-rt(10, df=v )

y_norm=a*x+e_norm
y_t=a*x+c(e_t,rnorm(40,sd=Sd)) # wariant 2 część to outliery

Zm1=lm(y_t~x)$coef[2]
    Zm2=rlm(y_t~x)$coef[2]

wynik[i,1]=Zm1
wynik[i,2]=Zm2
plot(1,1,main=paste(i))
}

plot(density(wynik[,1]),main="density of LS estimator(black) and robust estimator (green)")
lines(density(wynik[,2]),col="green")
# average values of LS and robust estimator
colMeans(wynik)
share|improve this question
2  
What are the degrees of freedom for the t? If df is > 5 the tails are not very large and the difference from a standard normal are not that great. Why not try a Cauchy? Why mix the t with a normal? Try all errors Cauchy. Did you pick a model where the slope(s) of the regression curve (surface) is (are) large? You could also pick a few points that would have errors that would make them highly influential for the least square regression parameters or the bivariate correlation between a predictor and the response. – Michael Chernick Sep 16 '12 at 20:28
1  
As far as I understand robust regression and robust methods in general, they work best when the sample is contaminated with a few outliers. This means that certain percent of the sample points do not conform to any relationship. What you have basically done was comparing regressions with iid noise vs regression with independent non-identically distributed noise. Asymptotically the loss of non-indenticallity is not a big deal, i.e. central limit theorem still holds. Thus you do not see big difference, since in your case simple OLS is robust. – mpiktas Sep 17 '12 at 11:39
with three degrees of freedom I see higher variance of estimator for OLS – Qbik Sep 18 '12 at 18:30

Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook.

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.