How does GARCH compute the realized daily volatility to be compared to the output of the model, to compute in-sample MSE? How do GARCH and GJR-GARCH models (as implemented in rugarch or in EViews) calculate the in-sample MSE if they use the time series of daily returns as the input and don't use a time series of daily variances: How do these models compute the realized volatility every day? Or are they using another measure of the actual volatility on day t that is then compared to the output of the model on day t? 
 A: Recall that a GARCH(1,1) model is a model of the conditional distribution of $x_t$. The model contains equations specifying the conditional mean, conditional variance by conditional standardized distribution:
\begin{aligned}
x_t &= \mu_t+\varepsilon_t, \\
\varepsilon_t &= \sigma_t z_t, \\
\sigma^2_t &= \omega + \alpha \varepsilon^2_{t-1} + \beta\sigma^2_{t-1}, \\
z_t &\sim D(0,1),
\end{aligned}
where $\mu_t$ is the conditional mean of $x_t$ (e.g. a constant or some autoregressive and/or moving-average terms) and $D$ is some distribution with mean $0$ and variance $1$. This can be extended to GARCH(p,q) or GJR-GARCH(p,q) without changing the essence of the argument that follows.
So $x_t$ is the original time series, $\hat x_t$ are fitted values of $x_t$ and (if I understand your question correctly) $\text{MSE}=\frac{1}{n}\sum_{t=1}^n (x_t-\hat x_t)$ (or maybe $\text{MSE}=\frac{1}{n-p}\sum_{t=1}^n (x_t-\hat x_t)$ where $p$ is the number of parameters in the model). $\hat x_t$ are obtained from the fitted model, so calculating the $\text{MSE}$ is not a problem. 
As you can see, there is no realized variance anywhere, while daily conditional variances are part of the model and their fitted values can be obtained if needed. Does this exposition help answer your question?
