A function $f : \mathbb{R}^D \rightarrow \mathbb{R}$ is linear if both of the following conditions hold.
(1) For all $\textbf{x}, \textbf{y} \in\mathbb{R}^D, f(\textbf{x} + \textbf{y}) = f(\textbf{x}) + f(\textbf{y})$.
(2) For all $\textbf{x} \in \mathbb{R}^D$ and $a \in \mathbb{R}$, $f(a\textbf{x}) = af(\textbf{x})$.
Consider the hypothesis class of linear regression.
That is, any predictor $y(\textbf{x}) = \textbf{w}^{\text{T}}\textbf{x}$ with $\textbf{w} \in \mathbb{R}^D$
Prove that every linear regression predictor is a linear function.
May use the definition of $\textbf{w}^{\text{T}}\textbf{x} = \sum_j w_jx_j$ as well as any basic fact about arithmetic without proof.

$\textbf{My Attempt:}$
Let $\textbf{x}$, $\textbf{y}, \textbf{w} \in \mathbb{R}^D$ and $a\in\mathbb{R}$.
For (1): since, $y(\textbf{x}+\textbf{y})=\textbf{w}^{\text{T}}(\textbf{x}+\textbf{y})=\sum_jw_j(x_j+y_j)=\sum_jw_jx_j+\sum_jw_jy_j=y(\textbf{x})+y(\textbf{y})$
For (2): since, $y(a\textbf{x})=\textbf{w}^{\text{T}}(a\textbf{x})=\sum_ja~(w_jx_j)=a~\sum_jw_jx_j=a~y(\textbf{x})$.
So, every linear regression predictor is a linear function.
$\textbf{Is that my prove correct ? }$


Yes, your proof is correct.

The statement itself is so simple, that maybe the person reading your proof (this looks like some assignment) will be extra picky, but I wouldn't worry about that. In this case I would rather make sure I have some intuitive grasp on what does it mean for a function to be linear and how does $y(x) = w^Tx$ look like geometrically.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.