Understanding stationarity in stochastic processes and time series

I am having trouble fully grasping the concept of stationarity in time series.

Here is what I have gathered so far.

1. A stochastic process is a collection of random variables with mean $\mu$ and variance $\sigma^2$.

2. A (A weakly) stationary stochastic process is one whose statistical properties do not change over time. For weak stationarity it is required that the mean and the variance of the process are constant over time and that the autocovariance between lagged values of the process only depends on the lag and not on the time t.

3. A time series is a realization of a stochastic process. It is a sequence of fixed numbers obtained from sampling the underlying stochastic process at discrete time instants.

Questions:

1. If we say weak stationarity requires the mean and the variance to be constant over time, doesn't that mean that the autocovariance of any pair of random variables should just be the variance of the time series, since $autocov(X_t ,X_t) = var(X_t)$? If that is the case, how can there be different values for the autocovariance for different lags?

2. What do we get when we calculate the mean and variance of all the values from one time series (meaning, take all the measured data points, add them up and divide them by the number of data points). How do those statistics relate to the mean and variance of the underlying stochastic process?

3. Regarding my third point: I keep coming across sentences to the extent of: " A time series is a collection of random variables." Can I just assume that people are misusing terminology or am I missing something?

• @Silverfish Thanks for the pointers. Does it make a difference in this case whether the we are considering a continuous or a discrete time series? – TheBaj Mar 15 '16 at 23:34
• Actually I withdraw the previous comment: I think you should have a look at this thread: Is a time series the same as a stochastic process?. You might find that your third question is to some extent a duplicate of the discussion there. (Notwithstanding the discussion on that thread, it isn't unknown for authors to talk of a "continuous time series", e.g. here. But I think "at discrete time instants" is much more common.) – Silverfish Mar 15 '16 at 23:55