# Machine learning predicted value

When we fit a generalized linear regression (e.g., logistic regression, gamma regression) we are estimating the population average Y given the predictors $X$ ( i.e., $E(Y | X)$ ).

When we fit a machine learning model such as an ANN, SVM, or a decision tree, does this notion still apply? In other words, are we estimating the population average value of $Y$ or isn't that idea applicable and we are just predicting "Y"?

I. What aspect of the theory of a predictive modeling algorithm tells us that we are modeling E(Y|X) versus just Y|X ? Is it the use of an error term that follows a certain distribution? For instance, what is it about ANN versus a decision tree which tells us the former models E(Y|X) while the latter is modeling Y|X?

II. Is there any connection between these and say a confidence interval versus prediction interval in linear regression?

-
Imo one can't answer a question like "is all of stats or ML this or that way" in a meaningful way other than by saying "No" (although the type of question pops up regularly). It depends on the technique, not the field. GLM, trees, SVM and ANN are different. Take your question: The $E(Y|X)$ part only makes sense for some probability measure to relate the expectation to, so if the technique has that then maybe, else no. SVM for example can be constructed completely devoid of probability. – Momo Dec 25 '12 at 1:54
Ok so to enumerate, I was interested in 1) gradient boosted regression trees, 2) svm and 3) artificial neural networks specifically. So is the deciding factor to my question , if the technique is derived in a probabilistic manner or not? – B_Miner Dec 25 '12 at 5:07
@B_Miner I know this is an old questions but did you figure this out? I'm currently thinking about confidence intervals and prediction intervals for ML predictions (GBM and RF in particular) and first I need to figure out the more fundamental questions. Hoping you're able to share some of your learnings. – organic agave May 24 at 19:38