3
$\begingroup$

I have been thinking lately about the following:

1. Is the normality assumption of the error term really needed in order to make use of p-values for linear regression models?

A previous CV post (see also image below) showed that the estimator of a linear regression model is normally distributed, given that the error is normally distributed.

However, the normality assumption of the error term is not really needed in order to make use of p-values given that we have a large sample size $(N>30)$, right? The derivation of the asymptotic normality $\sqrt{T}(\hat{\beta_T}-{\beta}) \rightarrow N(0,\sigma^2)$ did not make use of the normality assumption of the error term. enter image description here

Secondly, I have read many posts regarding the use of p-values in the case of robust linear regression models. However, I do not see why the use of p-values is complicated when we make use of robust linear regression.

2. What complicates the use of p-values in the case of robust linear regression models? And how would one assess the significance of the coefficients in a robust linear regression model?

EDIT: Derivation of asymptotic normality of estimator of linear regression only makes use of the mean property:

enter image description here

$\endgroup$
  • $\begingroup$ Can you mention specifically which "robust" models are being considered? And what complications in using p values for theses models are being discussed? Are these models like quantile requession and Kendall–Theil regression, or something closer to classic linear regression? $\endgroup$ – Sal Mangiafico Oct 19 at 17:20
  • $\begingroup$ I was asking this question in a general sense of robust linear regression models in R (so, more closer related to classic linear regression). There are many threads and links related to 'p-values and robust regression in R' . Many people have asked why they do not obtain P-values in the output. Most answers boil down to: it is not straightforward to interpret/obtain p-values from robust linear regression models. I am wondering why that is the case? Why does it become difficult to obtain p values for robust linear regression models? $\endgroup$ – Nadia Merquez Oct 19 at 17:41
  • $\begingroup$ The robust regression model I am particularly interested is the Least Absolute Deviations. See: rdocumentation.org/packages/Blossom/versions/1.4/topics/lad $\endgroup$ – Nadia Merquez Oct 19 at 18:18
  • $\begingroup$ I believe that the derivation of p-values for robust regression models is complicated due to the potential lack of an analytically tractable expression of the estimators. Furthermore, the t-statistic is sensitive to the presence of the outliers itself, complicating the process of obtaining a correct p-value. $\endgroup$ – Nadia Merquez Oct 19 at 19:08
  • $\begingroup$ I couldn't get the Blossom package to install for me. It's been removed from the CRAN archive. So it might better to look to other packages. In any case, it looks like the Blossom package relies on permutation tests and in some cases Monte Carlo simulation. If this is the case, it wouldn't rely on the assumptions of t tests, and might have few assumptions. $\endgroup$ – Sal Mangiafico Oct 19 at 20:01
4
$\begingroup$

I'm not a statistician, but I'll try to weigh-in.

Note: edited somewhat in response to comments by OP, to make this response more relevant for the linear regression case.

  1. For the p value to be valid, the assumptions for the test need to be met. These may include normality of errors for some tests, but other tests will have different assumptions. That is, if alternatives to classic linear regression are considered, the assumptions of p values for parameters for those models may not include that assumption. Questions about what assumptions or complications go into considering the reliability of robust models depend upon the specific robust model being considered.

  2. Relying on the central limit theorem to ensure the normality of parameters without examining the data is not a great practice. When thinking about e.g. if the sample mean is likely to come from a normal distribution, if the population distribution is very skewed, it may take a very large sample size to ensure an approximately normal distribution of means (and 30 is not necessarily a large sample size). (See Rand Wilcox quote below which considers specifically the two sample test, but I think is instructive.) It's not entirely clear to me how these considerations can be used practically in assessing the appropriateness of classic linear regression.

Rand Wilcox, 2017, Modern Statistics for the Social and Behavioral Sciences. Section 7.3.4.

Three Modern Insights Regarding Methods for Comparing Means

There have been three modern insights regarding methods for comparing means, each of which has already been described. But these insights are of such fundamental importance that it is worth summarizing them here.

• Resorting to the central limit theorem in order to justify the normality assumption can be highly unsatisfactory when working with means. Under general conditions, hundreds of observations might be needed to get reasonably accurate confidence intervals and good control over the probability of a Type I error. Or in the context of Tukey's three-decision rule, hundreds of observations might be needed to be reasonably certain which group has the largest mean. When using Student's T, rather than Welch's test, concerns arise regardless of how large the sample sizes might be.

• Practical concerns about heteroscedasticity (unequal variances) have been found to much more serious than once thought. All indications are that it is generally better to use a method that allows unequal variances.

• When comparing means, power can be very low relative to other methods that might be used. Both differences in skewness and outliers can result in relatively low power. Even if no outliers are found, differences in skewness might create practical problems. Certainly there are exceptions. But all indications are that it is prudent not to assume that these concerns can be ignored.

Despite the negative features just listed, there is one positive feature of Student's T is worth stressing. If the groups being compared do not differ in any manner, meaning that they have identical distributions, so in particular the groups have equal means, equal variances, and the same amount of skewness, Student's T appears to control the probability of a Type I error reasonably well under nonnormality. That is, when Student's T rejects, it is reasonable to conclude that the groups differ in some manner, but the nature of the difference, or the main reason Student's T rejected, is unclear. Also note that from the point of view of Tukey's three-decision rule, testing and rejecting the hypothesis of identical distributions is not very interesting.

$\endgroup$
  • $\begingroup$ Thanks for emphasizing that in some cases the sample size N requires to be very large in order to merely rely on CLT. I think that's a very good point to make. However, with regards to the second point: in the linear regression model, one is always interested in the so called mean of the estimator since $E(\hat{\beta}) = \beta$. $\endgroup$ – Nadia Merquez Oct 19 at 16:11
  • $\begingroup$ The first point mentioned in the answer was not so clear for me: 1. For the p value to be valid, the assumptions for the test need to be met ...... related tests. I am conducting a hypothesis test with the intention to see if an estimated coefficient from the linear regression model is statistically significantly different from zero $\endgroup$ – Nadia Merquez Oct 19 at 16:27
  • $\begingroup$ Thanks for the clarification. I do not think it is necessary to delete, you may clarify a little with the comments you just gave. The remarks may be of interest for other readers. $\endgroup$ – Nadia Merquez Oct 19 at 17:02
  • $\begingroup$ Thanks. I've tried to clean up my response to make it a little more focused on the case of classic linear regression. $\endgroup$ – Sal Mangiafico Oct 19 at 17:13
0
$\begingroup$

1) No. I recall normality condition is needed only for small sample (finite sample) cases. For large samples, asymptotic normality holds given some assumptions. (See Halbert White, Asymptotic theory for econometricians. Sorry for non-Statistic biased recommendation)

"Large" is ambiguous, but often say something like N>30. I suspect this came from t-dist and normal dist, that t-dist(N>30) becomes close enough to normal dist. Maybe you will find about this in Statistics. At least I haven't heard of Econometrics, though.

2) Though I don't have a clear answer for this, I guess this is coming from robust estimators' distributions being less trivial with finite samples. In this case, bootstrapped p-value can be used instead. One resamples, estimate coefficient in order to get "empirical distribution of coefficient". Then p-value can be calculated.

(reference: Wooldridge, Introductory Econometrics: A Modern Approach, Econometric analysis of cross section and panel data - sorry again for the "tilt")

$\endgroup$
  • $\begingroup$ The first point is in line with the way I understood it. However, the second point you mention ' I guess this is coming from robust estimators' distributions being less trivial with finite samples' Why would distribution of the robust model be different from t-distribution with small sample size? And what about the case when there is a sufficiently large N and using robust regression? $\endgroup$ – Nadia Merquez Oct 19 at 16:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.