Assessing the accuracy of a deterministic mathematical model How can assess the accuracy of the output of a deterministic mathematical model?
For example, a climate model can predict the mean annual temperature (MAT) for a specific location. I can use the model to predict thirty years of MAT in New York City, $T_\text{model}$. Now let's say I have the observed MAT in New York City for the same 30 years, $T_\text{obs}$.
Is there a statistical test for the hypothesis $T_\text{obs}=T_\text{model}$? Can I assess the accuracy of the mathematical model?
 A: A decent first step might be to compute the correlation between your model's predictions ("data A") and the observed temperatures ("data B"). Correlations range from -1 to +1: 0 indicates no (linear) relationship between the predicted and observed values, while higher values suggest that your model better agrees (up to a scale factor) with the observed data. Correlations can easily be computed in R with the cor function. The cor.test function does some testing of association between two variables. 
You should also just plot the data and take a look at it. Your model might not perform equally well under all conditions (e.g., maybe it breaks down around freezing temperatures)
There are more sophisticated things you could try, but I think these are a reasonable first step.
A: Your question sounds confusing.  When you say accuracy of the model, are you just referring to how well it predicts or do you mean how well it simulates the behavior of weather in New York City?  I don't think you can assess the latter.  As to the former, I would compute the mean square prediction error. By that I mean use the model to predict the mean annual temperature for each of the 30 years (presumably based on available inputs that the mode needs to get the estimates) and take the average squared difference between it and the actual recorded mean annual temperatures.  This gives you an estimate but not the accuracy of the estimate.  So you may have a standard for the accuracy and you want to test the hypothesis that accuracy is better than a certain level.  Now I can give a somewhat vague description of how to do this.  It is admittedly vague because I do not know what inputs go into the model to make the prediction.  The idea would be to make small perturbations to the input and see how these perturbations affect the accuracy of the prediction.  This would give you a distribution of mean square errors from which you could estimate a p-value for your hypothesis.  All this assumes that you have a sensible way to perturb the inputs that would characterize the sampling variability in the inputs.  The resulting estimates would then provide a representation of the variability of the individual predictions and from that the variability in the estimated means square error of prediction.
A: I would suggest two approaches to assessing whether or not a deterministic mathematical model is performing well - neither of which actually involve a statistical test, and which especially do not involve trying to reduce model performance to a p-value.


*

*How well does your model predict parameters? If your model estimates parameters from data, how well does this estimate agree with observed parameters from data other than what you fit the model to?

*Does it generate the correct answer when confronted with parameters that result in a known change. For example, if your model is given all the parameters that occur before a heat wave, does it correctly produce said heat wave?


As someone else has suggested, you could also compare the error between your predicted output and the actual output of the system, though this just gives you a number that you're trying to minimize, not actually a statistical estimate. Designing mathematical models to be tested statistically is very hard to do backwards - the elements you need generally need to be discussed in the model design step, just like with studies.
A: I recently devised a validation frame for deterministic solar irradiance forecasts. It bases on the insight that outcome and prediction of a perfect forecast must be mathematically exchangeable. It is generally applicable for the forecast of continuous stochastic variables.
See https://doi.org/10.1016/j.renene.2021.08.032
