2
$\begingroup$

When I run the least absolute deviation regression using command 'lad' from the 'L1pack' package, I get the following result but cannot determine the significance levels for each coefficients.

 Call:
 lad(formula = co[, 12] ~ co[, 2] + co[, 3] + co[, 5] + 
     co[, 7] + co[, 8] + co[, 9] + co[, 11] + co[, 2] * 
     co[, 11])
 Converged in 14 iterations

 Coefficients:
           (Intercept)            co[, 2]            co[, 3] 
              12.6123              -0.4472              -0.0528 
            co[, 5]               co[, 7]            co[, 8] 
              -0.1073               0.0051               0.0004 
            co[, 9]               co[, 11]           co[, 2]:co[, 11] 
              -9.6531              -0.9975               0.2303 

 Degrees of freedom: 101 total; 92 residual
 Scale estimate: 2.675847

Is there a way to figure out some general regression statistics like t-test, f-test, R square, etc. from this 'lad' regression? The 'summary' command does not seem to give out these as in the linear regression command 'lm'.

I tried using robust regression 'rlm' in 'MASS' package and see that I get required summary statistics for coefficients and their significances. However, I still do not get values for f-test, R square or so.

How about using this 'rlm' to resolve the outlier problem?

Besides, what methods would be appropriate for getting the f-test and R square values which I suppose cannot simply bring from simple linear regression 'lm' summary result?

Improve this question
$\endgroup$
3
$\begingroup$

There's not actually a t-test, because the estimate divided by its standard error doesn't have a t-distribution. Similar for an F-test.

Being a maximum likelihood estimate, there would be an asymptotic z-test, or an asymptotic chi-square test.

[There's the possibility of using some resampling-based tests as well, permutation tests or bootstrapping. You could also use L1pack's ability to simulate from L1 models to do a parametric bootstrap.]

You could compute something like an $R^2$ but it doesn't quite make sense because you're not computing something that tries to maximize that; it might make more sense to compute some analogous statistic, but $R^2$ has a number of properties and it depends on which things you try to carry over and which you don't.

[I note that quantreg::rq (which by default does L1 regression) will give an interval for the coefficients; this allows for a test (since you can see if the interval includes 0). There are also some other testing options in that package]

Improve this answer
$\endgroup$
18
  • $\begingroup$ Thanks but then how do people usually report the LAD regression results if one cannot show t-test, f-test, R square, etc. to check the coefficients and overall model's validity? How about using these values directly from the ones produced using simple linear regression with the same data? $\endgroup$ – Eric May 5 '17 at 19:35
  • $\begingroup$ If this question is not easy to answer, then probably the fundamental question may be better to approach, I suppose. If I want to run the regression by adjusting it with outliers, what is the best way to do this except simply removing the outliers and want to see the full report of the regression analysis for interpretations? $\endgroup$ – Eric May 5 '17 at 20:16
  • $\begingroup$ I tried using robust regression 'rlm' in 'MASS' package and see that I get required summary statistics for coefficients and their significances. However, I still do not get values for f-test, R square or so. How about using this 'rlm' to resolve the outlier problem? Besides, what methods do you think would be appropriate for getting the f-test and R square values which I suppose cannot simply bring from simple linear regression 'lm' summary result? $\endgroup$ – Eric May 5 '17 at 20:35
  • $\begingroup$ Just because rlm will print a t-value doesn't mean the t-value it gives you has a t-distribution (and it doesn't, though it is reasonably close to the asymptotic normal distribution across a variety of situations). The problem is that t- and F-distributions for test statistics ultimately rely on the very normality you are no longer willing to assume. As I have already said, you can test coefficients or the overall model, and in some cases groups of coefficients, in any of several potential ways, but small-sample normal theory tests don't just plug into things that aren't normal. ... ctd $\endgroup$ – Glen_b May 5 '17 at 23:54
  • $\begingroup$ ctd... You can sometimes get asymptotic approximations (sort-of t-like statistics that will go asymptotically to a normal and sort-of F-like statistics that go asymptotically to a chi-square ... these are mentioned already in my post). If something will give you a coefficient and a standard error you can call the calculated result a "t-ratio" if you like (as rlm has done) but it won't have a t-distribution and you'll have to see how it behaves in situations like the one you're interested in -- it may work okay for some estimators with some particular standard error calculations. ... ctd $\endgroup$ – Glen_b May 5 '17 at 23:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.