Problem
You have $N$ data points $y_1 \le \cdots \le y_N$ and wish to estimate a 'representative' value.
1) If your algorithm is to find the hypothesis $h$ that minimizes the in-sample sum of squared deviations,
$$E_{in}(h) = \sum_{n=1}^{N}{(h- y_n)^2},$$
then show that your estimate will be the in sample mean,
$$h_{mean} = \frac{1}{N} \sum_{n=1}^{N} { y_n }.$$
2) Suppose $y_N$ is perturbed to $y_N + E$, where $E \to \infty$. So the single data point $y_N$ becomes an outlier. What happens to your two estimators $h_{mean}$ and $h_{med}$?
My Answer
I first minimized the function by taking the derivative and set it to zero :
$f(h) = E_{in}(h) = \sum_{n=1}^{N} {(h - y_n)^2}$
$ {{a}/{ay}} ((h - y)^2)$
$2y - 2h \to 2y = 2h \to y = h$
In this case, $h$ is just a number
$h_{mean} = \frac{1}{N} \sum_{n=1}^{N} {y_n}$
However, I am failing to see how this "proof" actually makes sense regarding $h_{mean}$. Would anyone clarify it please? Am I in the correct path?
And about the other question, I am really unsure but my answer is this :
Since a single point would obviously be the outlier to infinite, the $h_{mean}$ would increase; however, the $h_{med}$ would remain the same.
I find this answer too simple and I think I am missing something!
REFERENCE
[self-study]
tag & read its wiki. $\endgroup$self-study
tag to your question. Also, why do you expect that the proof about the mean value with respect to minimizing residual sums of squares should have anything to do with the median? The median isn't in part (1) of the problem, just in part (2). There's a reason for that. $\endgroup$