Timeline for Measuring predictive accuracy of an ordinal outcome when the predictor is continuous
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 14, 2019 at 14:33 | comment | added | Trisoloriansunscreen | @whuber, the ordinal values are the ground truth against which the models' predictions are measured, so I'd say the comparison is between the prediction errors of the different model. The above question is about the definition of these errors when our access to the ground-truth is limited by quantization. From our discussion, I realize that my assumptions about the mapping between the latent outcome and the observed outcome are stronger than rank-preserving mapping. I can delete this question and post a more explicit one if this is appropriate. | |
| Jun 14, 2019 at 12:43 | comment | added | whuber♦ | I see no distinction between "scientific" and "applied:" by its very nature, any use of mathematics in science is an application of mathematics. Comparing your predictions to those of alternative models is quite different than comparing your predictions to the ordinal values! | |
| Jun 14, 2019 at 4:48 | comment | added | Trisoloriansunscreen | @whuber, the context is scientific, not applied, so the predictions serve only the testing of the underlying model, and more importantly, comparing it to alternative models. Perhaps a way to paraphrase this problem is to consider the evaluation of regression models in a scenario in which our outcome measurements were recorded with a very limited precision. | |
| Jun 13, 2019 at 21:21 | comment | added | whuber♦ | That begs the question of how these predictions will be used: in practice, how would one react to a prediction of, say, 3.4? | |
| Jun 13, 2019 at 20:44 | answer | added | Julian | timeline score: -1 | |
| Jun 13, 2019 at 20:43 | comment | added | Trisoloriansunscreen | Thank you @whuber, the way I see it, the continuous predictions I have are 'real' - the problem is with the quantized outcome. Obviously, I can transform the continuous predictions into ordinal predictions that will perfectly match the outcome, but this throws relevant information about the predictions' quality (predicting 3 by 3.1 is better than predicting 3 by 3.49, you can't see that if you rounded the predictions). | |
| Jun 13, 2019 at 20:35 | history | edited | Trisoloriansunscreen | CC BY-SA 4.0 |
added 351 characters in body
|
| Jun 13, 2019 at 19:11 | comment | added | whuber♦ | It seems to me you are missing a step: you don't actually have a prediction. You need to specify a procedure that unambiguously translates each numerical value currently labeled "prediction" into a unique possible outcome value. Once you have accomplished that, the meaning of "prediction error" will be clear and what remains is to define a loss function to translate errors into some cost that (a) is meaningful to your problem and (b) makes sense to aggregate across many individuals. Please edit your post to indicate which of these two issues you are concerned with. | |
| Jun 13, 2019 at 18:54 | history | asked | Trisoloriansunscreen | CC BY-SA 4.0 |