In GP regression we define a prior over functions in the form of a multivariate Gaussian distribution. After observing some data we can apply Bayes rule and we end up with a posterior distribution for predicting new inputs.
The prior distribution captures our initial beliefs via some function that computes the mean of our distribution and a kernel function through which we build the covariance matrix. This covariance matrix will capture any prior beliefs about how the random variables should behave in relation to one another.
How does one determine the dimensionality of this prior distribution? And how will the dimensionality of the prior affect our predictions? Will a larger covariance matrix result in assigning more weight to our prior beliefs since in higher dimensions we have more variables that behave in a similar way (defined via our kernel function)?