How to compute residuals in multiple linear regression model

Question

The residual can be seen as the distance between the observed data and the predicted data

In an a simple regression model (i.e. $x\in \mathbb{R}^{n\times m}$, $m = 1$, $y \in \mathbb{R}$) we have

• $\textbf{Measured value}$: $$y_i$$
• $\textbf{Predicted value}$: $$\hat{y}_i = f(x_i) = \hat{\beta}_0 + \hat{\beta}_1x_i$$
• $\textbf{Residual}$: difference between measured and predicted value of response variable: $$r_i = y_i - f(x_i) = y_i - \beta_0 - \beta_1x_i$$
• $\textbf{Residual sum of squares}$ is defined as

$$RSS = \sum_{i = 1}^n(y_i - f(x_i))^2 = \sum_{i = 1}^nr_i^2$$

Is it true to say that in a multiple model (i.e. $x\in \mathbb{R}^{n\times m}$, $m > 1$, $y \in \mathbb{R}$), this distance can be computed as the euclidean norm between those 2 vectors namely:

• $\textbf{Residual}$: difference between measured and predicted value of response variable: $$r_i = \|y_i - f(x_i)\|$$
• $\textbf{Residual sum of squares}$ is defined as $$RSS = \sum_{i = 1}^n(\|y_i - f(x_i)\|)^2 = \sum_{i = 1}^nr_i^2$$

Edit (Corrected answer reformulated)

• $\textbf{predictors}$ $$x_i \in \mathbb{R}^n, n = 1 \text{ for simple regression }, n>1 \text{ for multiple regression}$$
• $\textbf{Measured value}$: $$y_i \in \mathbb{R}$$
• $\textbf{Predicted value}$: $$\hat{y}_i = f(x_i) = \hat{\beta}_0 + \hat{\beta}_1x_i \in \mathbb{R}$$
• $\textbf{Residual}$: difference between measured and predicted value of response variable: $$r_i = y_i - f(x_i) = y_i - \beta_0 - \beta_1x_i \in \mathbb{R}$$
• $\textbf{Residual sum of squares}$ is defined as $$RSS = \sum_{i = 1}^n(y_i - f(x_i))^2 = \sum_{i = 1}^nr_i^2 \in \mathbb{R}$$

Answer 1

"Multiple" regression only refers to having multiple predictors, not to multiple dependent variables. Even if your $x_i\in\mathbb{R}^m$ for $m>1$, you still have $y_i\in\mathbb{R}^1$, and similarly $f(x_i)\in\mathbb{R}^1$.

So your residuals are still defined as $r_i=y_i-f(x_i)$, which is a subtraction of real numbers, not higher dimensional vectors. And you still define absolute residuals and RSS in the exact same way, no vector norms required.