I am facing a rather strange problem. Machine learning gives us a probability of a data point belonging to either class (classification) or the actual estimate of the dependent variable given independent variables (regression).

I am interested in building a model that has output such as $P(y \leq \hat{y} | x)$, that is, the 'probability of actual value to be less than or equal to predicted-value given x'.

I have gone through confidence intervals and predictions intervals for Logistic Regression. Are they best, I can do? Is there stream of research or machine learning, which I am missing.

  • $\begingroup$ These are two different things. Confidence intervals are used to express the uncertainty in estimate of a parameter based on your data. Prediction intervals are used to express the uncertainty in a future observation based on the given data and the model. $\endgroup$ – Michael R. Chernick Feb 21 '19 at 6:20
  • $\begingroup$ @MichaelChernick, by definition of Confidence intervals, I can say that $P(l \leq y \leq h)$, i.e., there is a 95% chance that true $y$ will fall in range between $l$ and $h$. I just thought, CI provides a good way forward to handle this problem. Please correct me if there is something wrong, in my understanding. I am happy to modify my question. :-) $\endgroup$ – neo Feb 21 '19 at 6:50
  • $\begingroup$ Please don't ask the same question twice. If they are different, please edit it to make it clear what exactly is different between them? $\endgroup$ – Tim Feb 21 '19 at 8:08
  • $\begingroup$ PS 95% CI does not tell you that there is 95% chance that true $y$ will fall iton range $[l, h]$. $\endgroup$ – Tim Feb 21 '19 at 8:09