Timeline for Why square the difference instead of taking the absolute value in standard deviation?
Current License: CC BY-SA 2.5
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 27, 2014 at 4:11 | comment | added | Neil G | @JeromeBaum: en.wikipedia.org/wiki/Median_of_medians | |
| Dec 27, 2014 at 2:38 | comment | added | Anonymous | @NeilG how do you propose to find the sample median in linear time? | |
| May 19, 2014 at 20:02 | comment | added | Neil G | @Rich: Both the variance and the median can be found in linear time, and of course no faster. Median does not require sorting. | |
| Jul 24, 2010 at 9:10 | comment | added | Rich | Yes, but finding the actual number you want, rather than just a descriptor of it, is easier under squared error loss. Consider the 1 dimension case; you can express the minimizer of the squared error by the mean: O(n) operations and closed form. You can express the value of the absolute error minimizer by the median, but there's not a closed-form solution that tells you what the median value is; it requires a sort to find, which is something like O(n log n). Least squares solutions tend to be a simple plug-and-chug type operation, absolute value solutions usually require more work to find. | |
| Jul 24, 2010 at 6:01 | comment | added | robin girard | I do not agree with this. First, theoretically, the problem may be of different nature (because of the discontinuity) but not necessarily harder (for example the median is easely shown to be arginf_m E[|Y-m|]). Second, practically, using a L1 norm (absolute value) rather than a L2 norm makes it piecewise linear and hence at least not more difficult. Quantile regression and its multiple variante is an example of that. | |
| Jul 24, 2010 at 2:55 | comment | added | Rich | Yeah, finding quantiles in general (which includes optimizing absolute values) tends to churn up linear programming type problems, which -- while they're certainly tractable numerically -- can get fiddly. They typically don't have an analytical closed-form solution, and are a bit slower and a bit more difficult to implement than least-square-type solutions. | |
| Jul 23, 2010 at 23:59 | comment | added | Vince | @robin: while the absolute value function is continuous everywhere, its first derivative is not (at x=0). This makes analytical optimization more difficult. | |
| Jul 23, 2010 at 21:40 | comment | added | robin girard | said "it's continuously differentiable (nice when you want to minimize it)" do you mean that the absolute value is difficult to optimize ? | |
| Jul 19, 2010 at 21:14 | history | answered | Rich | CC BY-SA 2.5 |