For the computation of the median, let $x_1,x_2,\ldots,x_n$ be the data. Assume, for simplicity, that $n$ is even, and the points are distinct! Let $y$ be some number. Let $f(y)$ be the 'sum-of-absolute deviations' of $y$ to the points $x_i$. This means that $f(y) = |x_1 - y| + |x_2 - y| + \ldots + |x_n - y|$. Your goal is to find the $y$ that minimizes $f(y)$. Let $l$ be the number of the $x_i$ that are less than or exactly equal to $y$ at a given point in time, and let $r = n - l$ be the number that are strictly greater than $y$. Pretend you are 'moving $y$ to the right', that is, increase $y$ slightly. What happens to $f(y)$?
Suppose you add an amount of $\Delta y$ to $y$. For those $x_i$ which are less than or equal to $y$, we have $|x_i - y|$ increases by $\Delta y$. And for those greater than $y$, we have $|x_i - y|$ decreases by $\Delta y$. (This assumes $\Delta y$ is so small that $y$ does not cross any of the points). Thus the change in $f(y)$ is $l\Delta y - r \Delta y = (l-r)\Delta y$. Note that this change in $f(y)$ does not depend on the values of the $x_i$ but only on the number to the left and right of $y$. By definition, $y$ is a median value when moving it to the left or right does not increase or decrease $f(y)$. This would mean that $l-r = 0$, and thus the number of $x_i$ to the left of $y$ is equal to the number to the right of $y$. And thus the median does not depend on the values of $x_i$, just their locations.
edit For the mean: the function $f(y)$ becomes $f(y) = (x_1 - y)^2 + \ldots + (x_n - y)^2$. Clearly the change in $f(y)$ for a small change in $y$ now depends on the magnitudes of the $x_i$, not just the number to the left and right of $y$.
Note that this business about the 'small change' is just covert talk for the derivative of $f(y)$...