4
$\begingroup$

In OLS regression:

$$Y=\beta_0+\beta_1 X_1+ \beta_2 X_2+\beta_3 X_3 + \beta_4 X_4+\beta_5 X_5+\beta_6 X_6 + \varepsilon,$$

what is $\varepsilon$? Is it Gaussian noise or random error? What is a difference? Why we add it to multiple regression model? In most of papers authors refer it to random error but without clarification.

I need a simple and good reason why authors add it to their model.

$\endgroup$
  • 1
    $\begingroup$ "is it Gaussian noise or random error" -- yes, that random error term is usually taken to be Gaussian noise, though if you're only estimating the coefficients (rather than computing intervals or doing hypothesis tests) it doesn't have to be Gaussian. If it's Gaussian then LS is optimal in several different senses at once. If it's not Gaussian then you still have that it's best linear unbiased. $\endgroup$ – Glen_b Oct 10 '13 at 6:43
  • $\begingroup$ Thanks a lot for reply :). the model is going to test hypotheses that are made before, then it's Gaussian noise. would you please give some reasons why we add this term to the model? $\endgroup$ – user31315 Oct 10 '13 at 6:56
  • 1
    $\begingroup$ Because the observations don't actually lie on a line/plane/hyperplane. $\endgroup$ – Glen_b Oct 10 '13 at 7:50
  • $\begingroup$ Does the wikipedia answer your question: en.wikipedia.org/wiki/Linear_regression? $\endgroup$ – mpiktas Oct 10 '13 at 8:28
  • $\begingroup$ this is wikipedia's definition: ε is called the error term, disturbance term, or noise. This variable captures all other factors which influence the dependent variable yi other than the regressors xi. The relationship between the error term and the regressors, for example whether they are correlated, is a crucial step in formulating a linear regression model, as it will determine the method to use for estimation. I'm writing a research paper, is this definition good enough to bring in research paper? $\endgroup$ – user31315 Oct 10 '13 at 8:59
1
$\begingroup$

The wikipedia definition is a fine definition that you can use for your paper if you need one but I think you're missing something.

The $\epsilon$ is random error, which is synonymous with noise. In practice, the random error can be Gaussian distributed, in which case it is Gaussian noise, but it could take on other distributions. If the distribution of $\epsilon$ happens to be Gaussian then you've met one of the theoretical assumptions of the model and things like interval estimation are better justified. If it's not Gaussian then, like Glen_b said, you still have that it's best linear unbiased.

Theoretically, the random error (noise) is supposed to be Gaussian distributed but the outcome could be anything. So, in order to answer your question you'd need to state whether you want to know the distribution of your particular noise or what the distribution of the noise should be. For the former you'd need data.

$\endgroup$
  • $\begingroup$ Thanks a lot for your answer. it is really good. and another question is: in my paper, I introduced some hypotheses and a regression model to determine the effect on each independent variable on dependent variable, I tested hypotheses by running multiple linear regression in SPSS software. in this case, do I need to specify what the noise should be? $\endgroup$ – user31315 Oct 10 '13 at 21:02
  • $\begingroup$ Generally, no, but I don't know nearly enough details about your paper. The shape of the noise is an assumption of your regression model and usually not something you need to specify in the paper. $\endgroup$ – John Oct 10 '13 at 21:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.