why is this key step valid in the derivation of Ito's lemma? I am confused about the validity of a certain step (perhaps the most crucial one) in the derivation of Itô's lemma.
Here's my understanding so far: Itô's lemma deals with transforming a differential equation of a stochastic variable $X(t)$ into a differential equation for a function $F(X, t)$ of that variable. In particular, suppose the variable $X(t)$ satisfies the differential equation
$$
\frac{dX}{dt}=\mu(t)+\sigma(t)\frac{dW}{dt},
$$
where the function $W(t)$ has the property that, given that $W(\tau) = w$, then for any $\Delta > 0$, the value of $W(\tau + \Delta)$ is a random variable that follows the probability distribution
$$
p(x)=\frac{1}{\sqrt{2\pi\Delta}}\exp\left[-\frac{(x-w)^{2}}{2\Delta}\right].
$$
The derivation begins by expanding $F$ to second order in $X$ and first order in $t$:
$$
dF=\frac{\partial F}{\partial X}dX+\frac{1}{2}\frac{\partial^{2}F}{\partial X^{2}}dX^{2}+\frac{\partial F}{\partial t}dt.
$$
Now in order to obtain Itô's lemma, we evaluate $dX^2 = \sigma^2 dW^2 + \mathcal{O}(dtdW) + \mathcal{O}(dt^2)$. Now here is where I am confused: the next step is to take $dW^2 = dt$. The reason I am confused is that $dW^2$ is a random variable; so we cannot say for certain that it is $dt$. In particular, we have
$$
dW=W(t+dt)-W(t).
$$
Hence, the random variable $dW$, evaluated at time $t$ is normally-distributed with mean 0 and variance $dt$:
$$
p_{dW}(x)=\frac{1}{\sqrt{2\pi dt}}\exp\left[-\frac{x^{2}}{2dt}\right].
$$
Now we want the probability distribution of $dW^2$. We have
$$
p_{dW^{2}}(y)=2\left|\frac{d(x^{2})}{dx}\right|^{-1}p_{dW}(x)
$$
where the 2 comes from the fact that there is a positive and negative root. We then have
$$
p_{dW^{2}}(y)=\frac{1}{\sqrt{2\pi ydt}}\exp\left(-\frac{y}{2dt}\right).
$$
This distribution is indeed normalized when integrating from 0 to $\infty$. Now if we compute the first two moments of this distribution, we note that $y = dW^2$ has mean $dt$ and standard deviation $\sqrt{2}dt$. Hence, for infinitesimal $dt$, we are not at all certain that $dW^2 = dt$! I would like to understand why this substitution is legitimate. 
 A: I just realized one possible way to resolve it is to map $W(t)$ back to a discrete random walk, where for each time step $dt$, the walker has equal probability of going left or right by $dx$. The mean displacement in a single step is 0, but the squared displacement is $dx^2$ for certain, because there are only two outcomes: $-dx$ and $dx$. 
Now extend to the continuum limit, the probability distribution is a Gaussian:
$$
p(x) \propto \exp\left(-\frac{x^2}{4Dt} \right),
$$
where $D$ is the continuum limit of the quantity $\frac{dx^2}{2dt}$, as seen in any derivation of the diffusion equation from a discrete random walk. For $W(t)$, we clearly have $D = 1/2$, so we must have $dx^2 = dt$. We concluded earlier in the discrete walk that the squared displacement in a single step is $dx^2$ for certain, so we have $dW^2 = dt$.
A: Heuristically maybe you can think $(dW_t)^2$ as a random variable with mean $dt$ and variance $2(dt)^2$, since $dt$ is infinitesimal, $(dt)^2$ vanishes, so $(dW_t)^2$ degenerates to $dt$.
That said, the reason that we say $(dW_t)^2=dt$ actually goes quite deep: consider a sequence $\{t_i\}_{i=0}^m$ such that $0=t_0<\dotsb<t_m=t$, and let $\Delta t_k=t_{k}-t_{k-1}$. Suppose $X_t$ is a continuous semi-martingale with local martingale part $M_t$, then for $f\in C^2$ we have the following change-of-variable formula (this is a more general version of Ito's formula): $$df(X_t)=f'(X_t)dX_t+\frac{1}{2}f''(X_t)d\langle M\rangle_t.$$ Here $\langle M\rangle_t$ is the quadratic variation of $M_t$, and it appears basically because for some approriately chosen $\eta$s, we have Taylor expansion $$f(X_t)-f(X_0)=\sum_{k=1}^mf'(X_{t_{k-1}})(X_{t_k}-X_{t_{k-1}})+\frac{1}{2}\sum_{k=1}^mf''(\eta_k)(X_{t_k}-X_{t_{k-1}})^2,$$ additionally, one can show that $$\sum_{k=1}^mf''(\eta_k)(X_{t_k}-X_{t_{k-1}})^2\to\int_0^t f''(X_{s})\,d\langle M\rangle_s\approx\sum_{k=1}^mf''(X_{t_{k-1}})(\langle M\rangle_{t_k}-\langle M\rangle_{t_{k-1}})$$ as $\Delta t_k\to 0$. In Ito's formula we replace $M_t$ by a Brownian motion $W_t$; thus $(dW_t)^2$ really is just a shorthand for $d\langle W\rangle_t$, and one can show that almost surely $\langle W\rangle_t=t$, hence the notation $(dW_t)^2=dt$.
In addition, it seems that questions similar to this have been asked several times elsewhere, e.g. you may take a look at this post. Also for reference I think Øksendal's Stochastic Differential Equations and Karatzas & Shreve's Brownian Motion and Stochastic Calculus can be quite helpful.
