# Linear regression with fat-tailed errors

I'm testing a linear model that explains stock returns with some contemporaneous factors; the model is assumed to satisfy OLS assumptions except that the errors (i.e., unexplained stock returns) have fat tails: let's say they are described by an alpha stable or Student's t distribution.

How should I estimate the model? I assume I should transform the data, or use a generalized linear model? I'm willing to be liberal with the distribution assumptions, as long as the calibration of the distribution and the estimation itself are not too complex.

• One possibility is to choose all the parts of your model (such as say linear regression with student t errors) and write down the log-likelihood function and simply maximize that. – Glen_b -Reinstate Monica Aug 15 '13 at 10:08
• What about some form of robust regression? – Peter Flom - Reinstate Monica Aug 15 '13 at 11:45
• @Glen_b That would work. Is there a reasonable fat tailed error distribution that results in a closed form solution? If not, is there any off the shelf software that can do this estimation or do I have to code it from scratch? – max Aug 15 '13 at 16:03
• Well, obviously anything that solves the general problem will need some level of input - chances are even very nice software doesn't know your density until you tell it, so there's always going to be some level of 'programming'. But in R for example, there's functions to do ML estimation if you supply the functions you want optimized. On the other hand, if you specifically want "linear regression with t-errors" and are willing to use R, there's the function tlm in the package hett on CRAN. Or you could do something like M-estimation, or robust regression more generally – Glen_b -Reinstate Monica Aug 15 '13 at 16:11
• Thanks. And yes, robust regression is exactly what I need, but the variety of techniques in this category is so great. In particular, it's hard for me to gauge whether I should go with M-estimation versus MLE with a specific distribution etc. – max Aug 15 '13 at 16:19

What software are you using? If you use Stan, for example, one of the examples in the manual (Section 11.3) uses a t-distribution instead of normal. You can use Stan from within R, with the package rstan -- Stan comes bundled with rstan, so no need to download and build Stan separately.

As Peter Flom says, that's a type of robust regression, so perhaps a straight-up robust regression (most any statistical package will have one or two versions) would work.

• Thanks, rstan sample code is great. We also tried Theil-Sen, but it only works if we exclude all the data points where the explanatory variable is close to zero (presumably because there are so many of them, that with the median approach, the informative points further away from zero have almost no influence on the beta estimation). – max Aug 15 '13 at 16:14

If the problem is only the fat tail, (i.e. the residuals are still constant mean equal to 0, constant variance and symmetric etc.), the OLS coefficients are still BLUE. So this method is still the "Best".

Intuitively, I think it is still equal to MLE.

The problem resuls that the t-test and F-test are not valid, and the AIC in the result is not the true AIC value. For such tests, maybe you can calculate the likelihood function (with t family or others) and employ a deviance test. Or more simply, compare the cross-validation errors with/without that variable.

• You said -- "the OLS coefficients are still BLUE. So this method is still the "Best"" -- but 'BLUE' is telling you it's 'best among unbiased linear estimators'. The problem is that if your response's conditional distribution is sufficiently non-normal, all linear estimators may be arbitrarily bad; in that case, being the best among them is a very low hurdle. – Glen_b -Reinstate Monica Aug 15 '13 at 16:15
• I feel in my case, the OLS estimates are way off because when I remove a couple "outliers" the betas move dramatically. (I have thousands of points of data). Once the outliers are removed, the results are reasonably robust. – max Aug 15 '13 at 16:23