Definition of Gaussian process Definition of a GP: a stochastic process $\{X_t, t\in T\}$ such that for every finite set of realization times $t_1, \dots, t_k \in T$, the joint $(X_{t_1}, X_{t_2}, \dots, X_{t_k})$ is a multivariate Gaussian random variable.
I am wondering why we don't simply define a GP to be $\{X_t, t\in T\}$ such that for every $t$, $X_t$ is a single-variate Gaussian variable. If this were the case, then $X_t$ would satisfy the above definition (since a joint distribution of Gaussians is a multivariate Gaussian).
I'm guessing this second definition is broader than the true definition. Are there any processes $X_t$ that satisfy the definition I suggested but isn't an actual GP?
Thanks.
 A: Yes, it is possible to construct processes which are marginally normally distributed but not jointly normally distributed.
Take for example two variables that are not jointly normal distributed:
$$
X\sim N(0,1)\\
Y=|U|\cdot \text{sgn}(X), \quad U\sim N(0,1)
$$
and construct the stochastic process
$$
X_t=\begin{cases}
X \quad \text{if } t<1\\
Y \quad \text{if } t\geq 1
\end{cases}
$$
The variable $X_t$ is gaussian distributed for any $t$ but $(X_{0.5},X_2)$ is not gaussian distributed.
A: One of the most important features of GPs is the relationship between its output values, which is characterized by a kernel function $k(x,y) = \mathrm{Cov}(f(x),f(y))$.
If you define $f(x)$ only through marginals as $f(x) \sim \mathcal{N}(\mu(x), \sigma(x))$, you do not define any relationship between $f(x), f(y)$ for $x\neq y$.
A: For my reference and anyone else who might want it:
The best general answer results from whuber's referred post, showing that Gaussian marginals do not generally combine to form a Gaussian joint. This shows that the answer to my posed question is no, in general (not just if variables are discontinuous, as demonstrated by svendvn).
The case where marginal Gaussians do form a Gaussian joint is when the variables are independent or are independent under some linear transformation: as whuber mentioned, any multivariate Gaussian (MVG) is equivalent under linear transformation to a MVG with zero covariances (*see below for explanation).
Another characterization is that the collection of rvs $X_1, \dots, X_n$ is jointly Gaussian iff for any $a_1, \dots, a_n$, the linear combination $\sum_i a_i X_i$ is Gaussian (see here).
*If we have a multivariate $X\sim \mathcal{N}(\mu, \Sigma)$, then a linear transformation $A$ gives $AX \sim \mathcal{N}(A\mu, A\Sigma A^\top)$. Since any covariance matrix $C$ is symmetric, it has the diagonalization $C = Q\Lambda Q^\top$ where $\Lambda$ is diagonal and $Q$ is an orthogonal transformation. This means that our $X$ can be transformed orthogonally to a coordinate system where it is a joint distribution of independent Gaussian marginals.
So in summary, it is necessary to define a GP as having joint Gaussian rather than joint marginals, and an alternate definition along this second route would be require an additional requirement of independence, i.e. "a stochastic process $X_t$ is a GP if for every $t$, $X_t$ is Gaussian distributed and every linear combination of a subset $(X_{t_1}, \dots, X_{t_n})$ is Gaussian.
Note: I'm still not sure how to prove that a set of variables is independent under some linear transformation
