# Meaning of "identically distributed" when there's only one variable

From page 8 of Rasmussen's book Gaussian Processes for Machine Learning: This is all pretty basic...but what do they mean by "identically distributed" in the last sentence? I understand if we have a collection of random variables, we refer to them as identically distributed to mean they all have the same distribution. But here $\epsilon$ is just one random variable. So what does "identically distributed" mean?

• This last sentence is definitely a sloppy formulation. There is no such thing as "identically distributed distribution". Mar 2, 2018 at 23:03

You have $n$ observations, $y\in\mathbb{R}^n$. You correspondingly have $n$ noise terms, $\epsilon\in\mathbb{R}^n$. The last sentence means that each separate noise term $\epsilon_i$ is identically distributed and that they are independent.
(In more general situations, the $\epsilon_i$ may not be identically distributed, or dependent, e.g., in time series analysis. You need more complex tools in such a situation.)
A simple random walk where $y_{y} = y_{t-1} + 2\mathcal{Bernouli}\left(0.5\right)-1$ is an example of a variable ($y_{t}$) that is not i.i.d.: each value of $y_{t}$ depends quite directly on it's immediately prior value, and in fact, the process "remembers" all perturbations to it infinitely. More, the variance of $y_{t}$ is a function of $t$ (in fact, $\sigma^{2}_{y_{t}}=t$), meaning that different values in this time series cannot possible have "the same" distribution.