0
$\begingroup$

The residual can be seen as the distance between the observed data and the predicted data

In an a simple regression model (i.e. $x\in \mathbb{R}^{n\times m}$, $m = 1$, $y \in \mathbb{R}$) we have

  • $\textbf{Measured value}$: $$y_i$$
  • $\textbf{Predicted value}$: $$\hat{y}_i = f(x_i) = \hat{\beta}_0 + \hat{\beta}_1x_i$$
  • $\textbf{Residual}$: difference between measured and predicted value of response variable: $$r_i = y_i - f(x_i) = y_i - \beta_0 - \beta_1x_i$$
  • $\textbf{Residual sum of squares}$ is defined as

$$RSS = \sum_{i = 1}^n(y_i - f(x_i))^2 = \sum_{i = 1}^nr_i^2$$

Is it true to say that in a multiple model (i.e. $x\in \mathbb{R}^{n\times m}$, $m > 1$, $y \in \mathbb{R}$), this distance can be computed as the euclidean norm between those 2 vectors namely:

  • $\textbf{Residual}$: difference between measured and predicted value of response variable: $$r_i = \|y_i - f(x_i)\|$$
  • $\textbf{Residual sum of squares}$ is defined as $$RSS = \sum_{i = 1}^n(\|y_i - f(x_i)\|)^2 = \sum_{i = 1}^nr_i^2$$

Edit (Corrected answer reformulated)

  • $\textbf{predictors}$ $$x_i \in \mathbb{R}^n, n = 1 \text{ for simple regression }, n>1 \text{ for multiple regression}$$
  • $\textbf{Measured value}$: $$y_i \in \mathbb{R}$$
  • $\textbf{Predicted value}$: $$\hat{y}_i = f(x_i) = \hat{\beta}_0 + \hat{\beta}_1x_i \in \mathbb{R}$$
  • $\textbf{Residual}$: difference between measured and predicted value of response variable: $$r_i = y_i - f(x_i) = y_i - \beta_0 - \beta_1x_i \in \mathbb{R}$$
  • $\textbf{Residual sum of squares}$ is defined as $$RSS = \sum_{i = 1}^n(y_i - f(x_i))^2 = \sum_{i = 1}^nr_i^2 \in \mathbb{R}$$
$\endgroup$
3
$\begingroup$

"Multiple" regression only refers to having multiple predictors, not to multiple dependent variables. Even if your $x_i\in\mathbb{R}^m$ for $m>1$, you still have $y_i\in\mathbb{R}^1$, and similarly $f(x_i)\in\mathbb{R}^1$.

So your residuals are still defined as $r_i=y_i-f(x_i)$, which is a subtraction of real numbers, not higher dimensional vectors. And you still define absolute residuals and RSS in the exact same way, no vector norms required.

$\endgroup$
  • $\begingroup$ Many thanks for your answer @StephanKolassa. I inserted the reformulated question as an edit in my question. Is it correct? $\endgroup$ – ecjb May 7 '19 at 6:29
  • $\begingroup$ Yes, that is correct. $\endgroup$ – Stephan Kolassa May 7 '19 at 6:33
  • $\begingroup$ many thanks! @StephanKolassa $\endgroup$ – ecjb May 7 '19 at 6:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.