While these two ubiquitous terms are often used synonymously, there sometimes seems to be a distinction. Is there indeed a difference, or are they exactly synonymous?
$\begingroup$
$\endgroup$
1
-
2$\begingroup$ Check out Qin & Gilbert "The Error Term in the History of Time Series Econometrics" for a comprehensive treatment if you have time. Oh, but this is only for time series data. $\endgroup$– Richard HardyCommented Jan 14, 2015 at 15:56
Add a comment
|
5 Answers
$\begingroup$
$\endgroup$
6
Errors pertain to the true data generating process (DGP), whereas residuals are what is left over after having estimated your model. In truth, assumptions like normality, homoscedasticity, and independence apply to the errors of the DGP, not your model's residuals. (For example, having fit $p+1$ parameters in your model, only $N-(p+1)$ residuals can be independent.) However, we only have access to the residuals, so that's what we work with.
answered Jan 14, 2015 at 15:40
-
31$\begingroup$ (+1) Residuals can be considered estimates of the errors. $\endgroup$ Commented Jan 14, 2015 at 17:09
-
1$\begingroup$ @ABC,
DGPstands for data generating process. Even if your model is appropriate & reflects the true structure of the DGP, the residuals won't necessarily be normal, homoscedastic & independent if the underlying errors weren't. $\endgroup$ Commented Apr 28, 2015 at 14:01 -
1$\begingroup$ @Scortchi Hi, do you happen to have any references that expand on your comment? I'm trying to understand why exactly residuals can be used as estimates of the error, because I see people checking normality of residuals in regression analysis when the actual assumption is normality of error, and I don't know precisely why that is valid. $\endgroup$– AustinCommented Aug 31, 2017 at 14:30
-
$\begingroup$ @Austin, if you are still interested in that, you should ask a new question. $\endgroup$ Commented Aug 12, 2019 at 16:51
-
$\begingroup$ What is p and N in the answer please elaborate it @gung-ReinstateMonica $\endgroup$ Commented Mar 24, 2020 at 11:28
$\begingroup$
$\endgroup$
4
An error is the difference between the observed value and the true value (very often unobserved, generated by the DGP).
A residual is the difference between the observed value and the predicted value (by the model).
-
$\begingroup$ In response to the error being "very often unobserved", in reality we could never observe the absolute true value. Every measurement device has finite error inherent in the measurement device. The best you could claim is that the true value at a point equals whatever measured plus/minus the observational error, or measurement error, e.g., my ruler can increment distances down to 1 mm, so the best I can measure the true length of my pencil is within $\pm$0.1mm. Note this differs from any systematic error, e.g., my ruler was somehow stretched out during manufacturing and measures too long. $\endgroup$ Commented Nov 21, 2017 at 14:22
-
$\begingroup$ This link (ece.rochester.edu/courses/ECE111/error_uncertainty.pdf) provides a good explanation with references to Bevington's and Taylor's texts on the subject. $\endgroup$ Commented Nov 21, 2017 at 14:26
-
1$\begingroup$ In machine learning language, is residue the training error and error is test error? $\endgroup$ Commented Dec 6, 2018 at 21:06
-
$\begingroup$ @CharlesChow It depends on which dataset you use. If you use training set, then it is training error; if you use test set, then it is test error. $\endgroup$ Commented Nov 15, 2019 at 6:13
$\begingroup$
$\endgroup$
Error term is a theoretical concept that can never be observed, but the residual is a real world value that is calculated for each time a regression is done
answered Jun 6, 2017 at 15:06
$\begingroup$
$\endgroup$
1
Error of the data set is the differences between the observed values and the true / unobserved values. Residual is calculated after running the regression model and is the differences between the observed values and the estimated values.
-
4$\begingroup$ This doesn't need a downvote, it isn't incorrect. I think the reason you got a downvote is because your answer is the same as several answers that already exist on this thread & doesn't add any new information. That will often be the case, when it is, you just don't post your answer on that thread. Find another thread where you can make a contribution & answer there. $\endgroup$ Commented Jan 1, 2020 at 14:00
$\begingroup$
$\endgroup$
Error term is an unknown value that could never be known unless the DGP is known. Therefore, theoretically, one can generate a variable x from say a Normal random variable and the error from a normal random variable. Then construct the variable $y$ as follow
$$ y_t=\beta x_t+e_t $$
Here, $e_t$ stands for error term the difference between the true variable $y_t$ and the expected value $\beta x_t$.
$\beta$ is unknown, once the beta is estimated then we get
$$ y_t=\hat{\beta} x_t+\hat{e}_t $$
Then $\hat{e}_t$ is not the error anymore it is the residual, the difference between the true value $y_t$ and the estimated value $\hat{\beta} x_t:=\hat{y}_t$.
This comes in line with another question, what is the difference between the Mean Squared Error and Mean Squared residual. There is nothing called MSR: Means squared residual.
$$ MSR=\frac{1}{n}\sum_{i=1}^{n}\hat{e}_i $$
However, many practitioners treat them the same. MSE is a theoretical concept that is always translated to MSR by practitioners due to their unfamiliarity between theory and practice.
$$ MSE=E(e_{t}^{2}) $$
