Comparing the mean of MSE for two models I've estimated 2 models on simulated data and basically replicate the estimations 100 times. What I want to see is if the models are actually different in terms of their MSE.
What I've done is to keep the MSE1 and MSE2 (for model 1 and 2) for each one of the 100 simulations, and I was thinking of comparing the mean MSE (I am not sure if this is actually the correct way of doing it).
Anyway I don't think I can apply a t-test due to the fact that the variables MSE1 and MSE2 are not part of a normal population (actually they look like a sort of $\chi^2$) because it's truncated ,no negative MSE are allowed.
Should I apply a $t$ test (isn't normality of observations an assumption of this test?) or is there any other method I can use to compare the both models? 
Thanks in advance.
 A: You can use the log likelihood score to compare different nested models, here is an example in R with two polynomial models using an ANOVA for comparison.
> mod1=lm(mpg~poly(disp,2)+hp+cyl,data=mtcars)
> mod2=lm(mpg~poly(disp,3)+hp+cyl,data=mtcars)
> anova(mod1,mod2)

Analysis of Variance Table

Model 1: mpg ~ poly(disp, 2) + hp + cyl
Model 2: mpg ~ poly(disp, 3) + hp + cyl
  Res.Df   RSS Df Sum of Sq      F    Pr(>F)    
1     27 222.5                                  
2     26 106.1  1    116.41 28.526 1.369e-05 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

If these models are not nested than a (repeated) CV procedure could be used to estimate the better model (for ex. using MSE). However, I would not use a statistical test to compare the two models. A subjective decision based on the mean MSE values and considering the simplest model would be my choice.
Nevertheless, if you are so inclined you could use a t-test to compare the two samples with MSE values. If the assumptions do not hold you can still use a non-parametric test, which does not require a specific distribution of the data.
A: For a large population (your 100 iterations are enough in my opinion), sample mean will converge to a normal distribution, even for data that is not normally distributed! You can perform hypothesis testing on it using a T-test
However, the previous solution does all the work for you
