Do discriminative models model conditional expectation? In Machine Learning classic models like MLP, Logistic Regression or Linear Regression are called discriminative models.
I frequently read that those models estimate the conditional probability distribution P(y|x). However, if they would really represent this distribution they should be able to return multiple likely outcomes but in reality they usually return only a single y. Doesn't this mean what they actually model is not the conditional distribution, P(y|x) but the conditional expectation E[y|x]?
 A: Discriminative methods are posed in contrast to generative methods, which model the joint distribution $p(x, y)$ of the input $X$ and output $Y$. But, the terminology isn't always consistent across authors. A discriminative model may refer to 1) A model of the conditional distribution $p(y \mid x)$ and/or 2) Any non-generative model that can predict $Y$ as a function of $X$. In the second case, the model need not be probabilistic (e.g. support vector machines).
Methods like ordinary least squares (OLS) and logistic regression do indeed model the conditional distribution. For example, in OLS it's Gaussian:
$$p(y \mid x) = \mathcal{N}(x \cdot \beta, \sigma^2)$$
where $\beta$ are the weights and $\sigma^2$ is the noise variance (which can be estimated as the variance of the residuals, given the usual assumptions). The conditional expectation $E[y \mid x] = x \cdot \beta$ is a point prediction. But, the conditional distribution can also be used to compute prediction intervals, sample multiple $y$ values, etc.
For logistic regression (with $Y \in \{0,1\}$), the conditional distribution is Bernoulli where:
$$p(Y=1 \mid x) = \frac{1}{1 + \exp(-x \cdot \beta)}$$
