9
$\begingroup$

What can you say about your linear regression if the residuals are uniformly distibuted (and not normal)? I would like to consider the case I have a histogram showing residuals which are uniformly distributed between -1 and 1.

We still have errors that are symmetrically distributed around 0, so I would have thought that the estimates would still be the same as for the normally distributed errors. Is this true? What else can we say? What about the variances/p-values of the estimates?

Improve this question
$\endgroup$
14
  • 3
    $\begingroup$ Hi: what is your methodology for concluding that they are uniform ? $\endgroup$
    – mlofton
    Commented Aug 1, 2020 at 13:42
  • 1
    $\begingroup$ How many observations are in the dataset? $\endgroup$
    – whuber ♦
    Commented Aug 1, 2020 at 16:55
  • 1
    $\begingroup$ 1. Importantly, what values can the response (DV) take? 2. That consideration aside, I'd put more stock in the plot of residuals vs fitted than a single histogram; I expect that doesn't look uniform at each fitted value, and may carry some important clues you will not see in a histogram. (Misspecification of the conditional mean or variance means that the histogram of residuals is not useful) $\endgroup$
    – Glen_b
    Commented Aug 2, 2020 at 9:17
  • 2
    $\begingroup$ Oh, interview questions are important -- they reveal at least as much about the situation with the employer as they do about the potential employee. If they had an issue with an answer that insisted on considering both the support of the DV and the conditional distribution of the residuals (as the distributional assumption actually relates to) rather than their marginal distribution before proceeding to say anything about what was implied by it, that would indicate a problem with the people involved in writing and posing the question -- and it would be a major red flag for me. $\endgroup$
    – Glen_b
    Commented Aug 2, 2020 at 9:41
  • 1
    $\begingroup$ statisticssolutions.com/normality "There are few consequences associated with a violation of the normality assumption, as it does not contribute to bias or inefficiency in regression models. It is only important for the calculation of p values for significance testing, but this is only a consideration when the sample size is very small. When the sample size is sufficiently large (>200), the normality assumption is not needed at all as the Central Limit Theorem ensures that the distribution of disturbance term will approximate normality." 1/2 $\endgroup$
    – Trajan
    Commented Aug 2, 2020 at 16:04

2 Answers 2

Reset to default
11
$\begingroup$

There are a few things we can say about this situation

  • the condition of normality of residuals only needs to hold approximately. If the sample size is small then it can be difficult to distiguish a uniform from a normal distribution, and it is reasonable in such circumstances to assess the residuals as plausibly normal. With a large sample size this obviously is not the case.

  • the estimates will be unbiased

  • the estimates will be consistent

  • the regression coefficient estimators will not be t distributed if using least squares, so the associated p values will be unreliable.

Improve this answer
$\endgroup$
12
  • $\begingroup$ What do you mean by "in general the standard errors (from t tests of the estimates) will be wrong"? Are the std. errors not found by $\hat \sigma (X^\top X)^{-1}$ which is standardly said to assume only homoscedasticity. $\endgroup$
    – Jesper for President
    Commented Aug 1, 2020 at 16:17
  • $\begingroup$ @JesperforPresident the t tests assume normality. It's easy to see what happens if you do a simple simulation: X <- seq(20); Y <- 10 + X + rnorm(N); lm(Y ~ X) %>% summary(); Y <- 10 + X + runif(N, -5, 5); lm(Y ~ X) %>% summary() $\endgroup$
    – Robert Long
    Commented Aug 1, 2020 at 16:23
  • 2
    $\begingroup$ They are valid asymptotically. So to say they are wrong is in my opinion too strong since this in my interpretation spells "do not use them". Still I'm not sure what im supposed to conclude from your simulation. Sure with the uniform dist. the std's are larger ... dont see how that makes them "wrong". $\endgroup$
    – Jesper for President
    Commented Aug 1, 2020 at 16:33
  • 1
    $\begingroup$ @JesperforPresident thanks, and I see your point but I think maybe I haven't worded that part of my answer well. I will have another go a bit later. $\endgroup$
    – Robert Long
    Commented Aug 1, 2020 at 16:39
  • $\begingroup$ the second my last bullet point is the core to this answer and is a bit hard to understand here. Also see Jesper's comment above $\endgroup$
    – Trajan
    Commented Aug 1, 2020 at 18:46
4
$\begingroup$

A classic linear regression model works under the assumption that the data can be modeled as

y = Ax + b + eta

where eta ~ N(0,sigma) .

if your residuals are uniformly distributed, it means that the above assumptions doesn't hold.

However, this linear regression can still work for you, depending on the application- both a uniform and Gaussian model are symmetric, with E(data) = median(data). So even though the data is not really 'Gaussian', the line that best fit the data (according to the mean/median) will be the same (again, depending on the application).

An option that might work for you is Bayesian Linear Regression (BLR):

in BLR, you can choose your model assumptions: eta ~ U(-1,1), eta ~ N(0,sigma) or eta ~ Beta(2,2) are all valid assumptions. anything that best fit your data

Improve this answer
$\endgroup$
2
  • $\begingroup$ are you saying that the estimates will be the same? what about the variance of these estimates or similar? $\endgroup$
    – Trajan
    Commented Aug 1, 2020 at 20:42
  • 1
    $\begingroup$ If you have a lot of data points, the mean and std for a uniform/Gaussian models will converge to the same values. if you have a small sample, your std estimate could be inaccurate. $\endgroup$
    – Jonathan
    Commented Aug 1, 2020 at 23:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.