Autocorrelation for a continuous time stochastic process I have a question involving this (specifically the auto-correlation part):

My question is: Why is there a dirac delta function instead of an indicator function (i.e. the dirac function is "infinity" when the argument is zero, whereas the indicator function is 1 - for this case - when the argument is zero)? 
Intuitively, it seems that when t and tau are equal, then the expected value of w(t)*w(tau) should just be the variance (or the variance times 1, which would be reflected in the indicator function). Instead, in this example, it's the variance times "infinity". 
 A: The defining equations for the (continuous-time) white noise process are incorrect: the white noise model corresponds to a weakly stationary (or
wide-sense-stationary) process and the autocorrelation function must be
$$E[w(t)w(t+\tau)] = K \delta(\tau), -\infty < t, \tau < \infty
\tag{1}$$ 
where $\delta(\tau)$ is indeed the Dirac delta. Note that I do not
say Dirac delta function because $\delta(\tau)$ is not a function
in the sense that it is not a mapping $\tau \mapsto \delta(\tau)$. Thus, $\delta(1)$ or $\delta(0)$ does not stand for a number, and 
it is not
correct to say that $\delta(1) = 0$ or that $\delta(0)$ is "infinity."
Suffice it to say that for all practical purposes (for engineers
such as myself, at least), the Dirac delta belongs only inside
an integral, and its defining property is
$$\int_{-\infty}^\infty g(t)\delta(t)\,\mathrm dt = g(0)
~ \text{provided that}~ g(t)~ \text{is continuous at}~ t=0.\tag{2}$$
This sifting integral can be manipulated according to the usual
rules regarding change of variables in integrals and thus, subject to the
continuity constraints, we have that 
$$\int_{-\infty}^\infty g(t)\delta(t-\tau)\,\mathrm dt = g(\tau)\tag{3}$$
and
$$\int_{-\infty}^\infty g(t)\delta(at)\,\mathrm dt
= \int_{-\infty}^\infty \frac{1}{|a|}g(\tau/a)\delta(\tau)\,\mathrm d\tau = \frac{g(0)}{|a|}\tag{4}$$
etc. Finally, I will add that the above calculations also
hold when the integrals are not over the entire real line but
over an interval that includes the point where the Dirac delta
argument equals $0$, that is,
$$\int_a^b g(t)\delta(t-\tau)\,\mathrm dt =
\begin{cases} g(\tau), & \tau \in (a,b)\\0, & \tau \notin (a,b)\end{cases}
\tag{5}$$

Turning to the problem at hand, you have a particle that (without loss
of generality) is at the origin at $t=0$ and moves with constant velocity
$v_0$ (which needs to be specified as an initial condition) along
the $x$ axis. This is in the absence of any noise: the acceleration
is defined to be $0$. When the acceleration is a deterministic
function $y(t)$, say, the
velocity at time $t$ is given by
$$v(t) = v_0 + \int_0^t y(\tau)\,\mathrm d\tau \tag{6}$$
If the acceleration is a wide-sense-stationary
random process $y(t)$, then
$(6)$ is a stochastic integral and $v(t)$ is a random variable.
In fact, the ensemble of random variables $\{v(t) \colon t \geq 0\}$
is a random process.  I will simply state (without providing
a justification) that subject to some constraints, it is possible
to interchange the order of taking the expectation and
the integration and write 
$$E[v(t)] = E\left[v_0 + \int_0^t y(\tau)\,\mathrm d\tau\right]
= v_0 + \int_0^t E[y(\tau)]\,\mathrm d\tau = v_0.\tag{7}$$
if $\{y(t)\}$ is a zero-mean process. More generally, for a zero-mean
wide-sense-stationary input process,
\begin{align}
\operatorname{var}(v(t)) &= E[(v(t)-v_0)^2]\\
&= E\left[\int_0^t y(\tau)\,\mathrm d\tau\int_0^t y(\nu)\,\mathrm d\nu\right]\\
&= \int_0^t\int_0^t E[y(\tau)y(\nu)]\,\mathrm d\tau\,\mathrm d\nu\\
&= \int_0^t\int_0^t R_y(\tau-\nu)\,\mathrm d\tau\,\mathrm d\nu\\
&= \int_{-t}^t (t-|\lambda|)R_y(\lambda)\,\mathrm d\lambda\tag{8}
\end{align}
where $R_y(\cdot)$ is the autocorrelation function of the input
process. 
Finally, when the input process is a white noise
process with autocorrelation function $K\delta(\tau)$, we
get from $(5)$ and $(8)$ that the velocity $v(t)$ is a
random variable with mean $v_0$ and variance $Kt$. Note that
the ensemble $\{v(t)\colon t \geq 0\}$ is not a 
wide-sense-stationary process (the variance is an increasing
function of time). If the input process is a Gaussian
white noise process, then the velocity
$v(t)$ is a Gaussian random variable
and $\{v(t)\colon t \geq 0\}$ is a continuous-time random
walk (sometimes called a Brownian motion) that starts at
$v_0$. The location of the particle is the integral of this
Brownian motion.
