Timeline for Is the Error rate a Convex function of the Regularization parameter lambda?
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 5, 2020 at 15:22 | history | edited | whuber♦ | CC BY-SA 4.0 |
Trying to dodge a bug in rendering the final figure.
|
| Jun 11, 2020 at 14:32 | history | edited | CommunityBot |
Commonmark migration
|
|
| Aug 22, 2017 at 17:42 | comment | added | user795305 | @whuber Thanks! I agree: the (true) distribution between the training and test points should be the same, and there needs to be enough samples that the empirical distributions of the training and test set have agreement. (It seems I phrased that poorly in my earlier comment.) For instance, if $(\mathbf x, y)$ has a jointly normal distribution (with nondegenerate covariance), I suspect the probability of the error curve having a unique local min converges to 1 (if, say, there's $n$ samples in training and test set with $n \to \infty$ with $p$ fixed (or even increasing slowly relative to $n$)) | |
| Aug 22, 2017 at 17:06 | comment | added | whuber♦ | @Ben It's not the number of test points that matters: this result depends entirely on the distribution of test points relative to the distribution of training points. Therefore the issue of "with high probability" will not be answerable without making some specific assumptions about the multivariate distribution of the regressor variables. Also, with many variables in play this phenomenon of multiple local minima is going to be much more likely. I suspect that random selection of a large test set (with many times as many observations as variables) might often have a unique global min. | |
| Aug 22, 2017 at 16:27 | comment | added | user795305 | Great answer (+1). In practice, I think there's often not so few training and test data points. Does the conclusion of this answer change when there's enough training and test data points drawn from the same (fixed and sufficiently regular) distribution? In particular, under this scenario, is there a unique local minimum with high probability? | |
| Aug 18, 2017 at 14:17 | history | edited | whuber♦ | CC BY-SA 3.0 |
added 1315 characters in body
|
| Aug 18, 2017 at 5:32 | comment | added | rf7 | Thank you for your elaborate answer. If possible review the question as I edited and update your response. | |
| Aug 18, 2017 at 5:23 | vote | accept | rf7 | ||
| Aug 17, 2017 at 22:40 | history | answered | whuber♦ | CC BY-SA 3.0 |