Intuitive explanation of stationarity I was wrestling with stationarity in my head for a while... Is this how you think about it? Any comments or further thoughts will be appreciated.

Stationary process is the one which
  generates time-series values such that
  distribution mean and variance is kept
  constant. Strictly speaking, this is
  known as weak form of stationarity or
  covariance/mean stationarity.
Weak form of stationarity is when the
  time-series has constant mean and
  variance throughout the time.
Let's put it simple, practitioners say
  that the stationary time-series is the
  one with no trend - fluctuates around
  the constant mean and has constant
  variance.
Covariance between different lags is
  constant, it doesn't depend on
  absolute location in time-series. For
  example, the covariance between t and
  t-1 (first order lag) should always be
  the same (for the period from
  1960-1970 same as for the period from
  1965-1975 or any other period).
In non-stationary processes there is
  no long-run mean to which the series
  reverts; so we say that non-stationary
  time series do not mean revert. In
  that case, the variance depends on
  absolute position in time-series and
  variance goes to infinity as time goes
  on. Technically speaking,
  auto-correlations to not decay with
  time, but in small samples they do
  disappear - although slowly.
In stationary processes, shocks are
  temporary and dissipate (lose energy)
  over time. After a while, they do not
  contribute to the new time-series
  values. For example, something which
  happened log time ago (long enough)
  such as World War II, had an impact,
  but, it the time-series today is the
  same as if World War II never
  happened, we would say that shock lost
  its energy or dissipated. Stationarity
  is especially important as many
  classical econometric theories are
  derived under the assumptions of
  stationarity.
A strong form of stationarity is when
  the distribution of a time-series is
  exactly the same trough time. In other
  words, the distribution of original
  time-series is exactly same as lagged
  time-series (by any number of lags) or
  even sub-segments of the time-series.
  For example, strong form also suggests
  that the distribution should be the
  same even for a sub-segments
  1950-1960, 1960-1970 or even
  overlapping periods such as 1950-1960
  and 1950-1980. This form of
  stationarity is called strong because
  it doesn't assume any distribution. It
  only says the probability distribution
  should be the same. In the case of
  weak stationarity, we defined
  distribution by its mean and variance.
  We could do this simplification
  because implicitly we assumed normal
  distribution, and normal distribution
  is fully defined by its mean and
  variance or standard deviation. This
  is nothing but saying that probability
  measure of the sequence (within
  time-series) is the same as that for
  lagged/shifted sequence of values
  within same time-series.

 A: First of all, it is important to note that stationarity is a property of a process, not of a time series. You consider the ensemble of all time series generated by a process. If the statistical properties¹ of this ensemble (mean, variance, …) are constant over time, the process is called stationary. Strictly speaking, it is impossible to say whether a given time series was generated by a stationary process (however, with some assumptions, we can take a good guess).
More intuitively, stationarity means that there are no distinguished points in time for your process (influencing the statistical properties of your observation). Whether this applies to a given process depends crucially on what you consider as fixed or variable for your process, i.e., what is contained in your ensemble.
A typical cause of non-stationarity are time-dependent parameters – which allow to distinguish time points by the values of the parameters. Another cause are fixed initial conditions.
Consider the following examples:


*

*The noise reaching my house from a single car passing at a given time is not a stationary process. E.g., the average amplitude² is highest when the car is directly next to my house.

*The noise reaching my house from street traffic in general is a stationary process, if we ignore the time dependency of the traffic intensity (e.g., less traffic at night or on weekends). There are no distinguished points in time anymore. While there may be strong fluctuations of individual time series, these vanish when I consider the ensemble of all realisations of the process.

*If I we include known impacts on traffic intensity, e.g., that there is less traffic at night, the process is non-stationary again: The average amplitude² varies with a daily rhythm. Every point in time is distinguished by the time of the day.

*The position of a single peppercorn in a pot of boiling water is a stationary process (ignoring the loss of water due to evaporation). There are no distinguished points in time.

*The position of a single peppercorn in a pot of boiling water dropped in the exact middle at $t=0$ is not a stationary process, as $t=0$ is a distinguished point in time. The average position of the peppercorn is always in the middle (assuming a symmetric pot without distinguished directions), but at $t=ε$ (with $ε$ small), we can be sure that the peppercorn is somewhere near the middle for every realisation of the process, while at a later time, it can also be closer to the border of the pot.
So, the distribution of positions changes over time. To give a specific example, the standard deviation grows. The distribution quickly converges to the respective distributions of the previous example and if we only take a look at this process for $t>T$ with a sufficiently high $T$, we can neglect the non-stationarity and approximate it as a stationary process for all purposes – the impact of the initial condition has faded away.


¹ For practical purposes, this is sometimes reduced to the mean and the variance (weak stationarity), but I do not consider this helpful to understand the concept. Just ignore weak stationarity until you understood stationarity.
² Which is the mean of the volume, but the standard deviation of the actual sound signal (do not worry too much about this here).

A: For clarity, I would add that any time series where the datapoints are normally distributed through time with a constant mean and variance is considered a strong stationary time series since given the mean and standard deviation the normal distribution will always have the same probability distribution curve (the inputs to the normal equation only depend on the mean and the standard deviation).  
This is not the case with a t-distribution, for example, where an input to the t-distribution equation is gamma which impacts the shape of the distribution curve despite a constant mean and constant standard deviation.      
