Does a quadratic covariance function imply joint Gaussian distribution? Consider a linear model $f(x) = \phi(x)^\top w$; where x is deterministic, $\phi$ is some fixed transformation, $w$ is independent and  $w\sim \mathcal{N(0, \Sigma_p)}$. The covariance function is $Cov(f(x),f(y)^\top) = \phi(x)^\top \Sigma_p \phi(y)$. Does this imply that the joint distribution of $f(x)$ and $f(y)$ is Gaussian?
 A: Yes, a linear combination of jointly Gaussian random variables is Gaussian. You do not need to turn to covariance functions. The covariance function calculation is separate... The Gaussian property is most easily proven using moment generating functions.
A: A random vector $X = (X_1, ..., X_k)$ is said to have a multivariate Gaussian distribution iff. one of the following equivalent conditions is satisfied (see e.g. https://en.wikipedia.org/wiki/Multivariate_normal_distribution#Definition):


*

*Every univariate linear combination of its components $Y = a_1X_1 + ... + a_kX_k$ is normally distributed.

*There exists a vector $Z = (Z_1, ..., Z_d)$ (with $d$ potentially being smaller than $k$) of independent univariate Gaussians and a matrix $A$ and a vector $\mu$ such that $X = AZ + \mu$

*$X$ has a certain characteristic function.


Using 1. we see that $f(x)$ is nothing else but a constant vector times a multivariate Gaussian, hence a univariate Gaussian. The same holds for $f(y)$. Now if you assume that $f(x)$ and $f(y)$ are independent (i.e. the $w$ in $f(x)$ and the $w$ in $f(y)$ are independent which I assume is true as $x$ and $y$ are two different observations or something that are unrelated) then $(f(x), f(y))$ is a vector consisting of two independent univariate Gaussians. By condition 2. this is a multivariate Gaussian.
A quick remark on your condition: Since $x$ and $y$ are not at all random, you compute any covariance between $f(x)$ and $f(y)$ (this matrix always "exists" but may have infinite entries). How should that in general imply that there is some structure on the variables when any variable paired in that way with any other variable has 'some' covariance matrix?
