About Murphy's notation: why is $p(y|x, \theta)$ a conditional expectation when there is no probabilistic interpretation on $x$ or $\theta$?

In section 1.4.5 of Kevin Murphy ML textbook, he introduces linear regression where for a given data $$x$$, the target $$y$$ is assumed to be obtained through

$$y(x) = w^Tx + \epsilon, \text{ where } \epsilon \sim \mathcal{N}(\mu, \sigma^2)$$

Since $$\epsilon$$ is a random variable, therefore $$y$$ is a random variable as well induced solely by $$\epsilon$$.

However, the author then defines $$p(y|x, \theta) = \mathcal{N}(y|\mu(x), \sigma^2(x))$$

where $$\theta = (w, \sigma^2)$$

First of all, $$x$$ here is explicitly just a vector with no assumption that is was generated according to a distribution. Secondly, $$\theta$$ here is just the parameters associated with the Gaussian as well as the model, there is no probablistic interpretation as well.

How does the author condition on two deterministic variables?

Relevant text below:

• The author doesn't condition: this is merely an abuse of notation. The two other variables are parameters.
– whuber
Jul 24 '20 at 13:29
• @whuber But this doesn't make sense if you consider the parallel development for logistic regression. It is always $P(y|x)$ conditioned on $x$, web.stanford.edu/class/archive/cs/cs109/cs109.1166/pdfs/… Jul 24 '20 at 19:32
• It is not. In logistic regression there is no need to assume $x$ is a random variable. One proof of that is that there is no probability (or density) associated with $x$ in the expression for the likelihood given in the "Derivations" section.
– whuber
Jul 24 '20 at 19:35

If $$x$$ is not stochastic, then $$y$$ is just an affine transformation of the multivariate normal variable $$\epsilon$$.

When $$x$$ is stochastic, no distributional assumption is needed. All that is required is that the conditional distribution of $$y$$ given $$x$$ is normal and the marginal distribution of $$x$$ is unrestricted (see Hansen, Econometrics, p. 141).

As to $$\theta$$, in a frequentist setting parameters are just unknown numbers, and one would rather write $$p(y;x,\theta)$$, but in a bayesian setting parameters are random variables. And Murphy is going to introduce Bayesian concept learning...

• The first characterization is deceiving, because which transformation $y$ is depends on $x.$ And it's an affine transformation: the linear part merely is $\epsilon$ itself.
– whuber
Jul 24 '20 at 14:01
• @whuber Yes, "affine" is more technical. I was just saying that if $x$ is non stochastic, then $y=w^Tx+\epsilon$ is like the familiar "linear" (more technically: affine) transformation $Y=a+bX$. Jul 24 '20 at 14:08
• The concern isn't linear vs affine; it's that even if we use affine, it's not the case that "$y$ is just a [linear or affine] transformation of a normal variable."
– whuber
Jul 24 '20 at 16:35
• Please, could you explain this point? Jul 24 '20 at 16:39
• I've edited my answer. As to how $y$ (a random variable) is related to $x$ (when a non stochastic variable) another time :) Thanks! Jul 24 '20 at 20:38