To expand my comment, let's also refer to Theorem 12 of lecture 9:
http://www.stat.cmu.edu/~larry/=stat705/Lecture9.pdf
First let us adopt the setting you reference in lecture 16.
In particular $p(\cdot; \theta)$ is your parametric family of distribution and $s(y,\theta)=\frac{\partial}{\partial\theta} p(y;\theta_0) $ is the score function, and
$$
V = Var[s(Y;\theta_0)],
$$
where the data $Y$ is from some density $f$ and (crucially) $\theta_0$ is chosen to minimise the KL distance from $f$ to $p(\cdot,\theta)$.
Now, for the bit that seems to be skipped over in those notes. The minimising $\theta_0$ also minimises
$$
\int f(y) \ln p(y;\theta)dy
$$
and so the derivative at $\theta_0$ is zero also (assuming some regularity conditions). Passing the derivative under the integral sign (++regularity) gives
$$
\int f(y)\frac{\partial}{\partial\theta} p(y;\theta_0) dy = 0,
$$
i.e. $E[s(Y;\theta_0)]=0$.
Now let's go to the proof of Theorem 12.
Taylor-expand the likelihood function around $\theta_0$ (recally that $\hat \theta$ is still the MLE in lecture 16)- the basic difference is that you are using data from $f$ now, but as we've seen the score function at the KL minimiser still has expectation zero. Thus you still get the CLT for $A$ but with $V = Var(s(Y;\theta_0))$ instead of $I(\theta_0)$.
The WLLN still applies for convergence of $B \to -E[H(X;\theta_0)] = J$, and then Slutsky's Theorem still gives the final result. Note that we cannot (generally) simplify $E[H(X;\theta_0)]$ to something like the Fisher information in general. That is why it still appears in the answer.
(I have focused on the univariate case but the generalisation to multivariate should be in the same vein)
To clarify something in the question from my first (deleted) comment- it is not true that $I^{-1} = J^{-1} V J^{-1}$ in general. In the special case that $f$ is in the family you get $I = V = -J$, possibly leading to the confusion.
