# What does it mean that a Gaussian process is 'infinite dimensional?'

I have glossed over this phrase many times without really understanding what it means. According to Wikipedia - Gaussian process

Gaussian processes can be seen as an infinite-dimensional generalization of multivariate normal distributions

What makes it infinite-dimensional? Just because the kernel function handles an arbitrary number of input dimensions?

• – Sycorax Apr 2 at 3:23
• Any continuous time series that can be indexed at a time $t$ is "infinite-dimensional". If you thought of "building" a multivariate normal time series by considering 2 time points, then 3, then 4, then... suddenly you have an $2^{\aleph_0}$ cardinality infinite time series. – AdamO Apr 2 at 21:59

While a sample from a multivariate Gaussian distribution produces a vector with a discrete number of elements, a sample from a Gaussian Process is a continuous function, which is "infinite-dimensional" in the sense that it is "indexed" by a continuously varying coordinate.

Suppose we have $$X \sim \mathcal N_n(\mu, \Sigma)$$. We can think of $$X$$ as giving us a random function from $$\{1, \dots, n\}$$ to $$\mathbb R$$, which we evaluate by indexing so e.g. $$X(1) = X_1$$. The space of random functions with this domain is $$n$$-dimensional since it is spanned by the functions $$\{e_1, \dots, e_n\}$$ where $$e_i(t) = \mathbf 1_{t=i}$$ are just the standard basis vectors thought of as functions.
The stochastic process view of this is that we have an index set $$T = \{1,\dots, n\}$$ and then we have random variables $$X_t : \Omega\to\mathbb R$$ for each $$t \in T$$. A single realization of this process yields a sequence $$(x_1, \dots, x_n)$$ which can be thought of as a particular random function. More formally, if $$(\Omega, \mathscr F, P)$$ is our probability space, then a single realization of the stochastic process is the function from $$T$$ to $$\mathbb R$$ given by $$t \mapsto X_t(\omega)$$ where $$\omega\in\Omega$$ is the sample outcome.
If we want random functions with an infinite support (so what we more typically think of as functions, like $$f : \mathbb R\to\mathbb R$$ with $$f(x)=x^2$$) we can get those by using stochastic processes with larger index sets like $$T=\mathbb N$$ or $$T = [0,\infty)$$. A single realization of one of these processes gives us a function from $$T$$ to $$\mathbb R$$, but now these functions live in an infinite dimension space (typically). In other words, the space of functions that can be realized by this process is an infinite dimension function space, as opposed to $$\mathcal N_n(\mu,\Sigma)$$ where the space of realizable functions is finite dimensional.