Compare a real-world process with model prediction

Question

I have a real-world time series process with an dependent variable Y responds to multiple independent variables (not necessary a linear response but can assume so if necessary). A non-statistical model was built to predict the process with the same set of the independent variables. I'd like to monitor the model performance over time, specifically whether the modeled Y variable still responds to the independent variables the same way as of the real-world process.

I could build another statistical model for the residuals (real Y - modeled Y) and look at the model coefficients and their statistical significance. But there are some issues associated with the approach. I may not be able to flag model biases or noises. Just wondering whether there is more statistically rigorous approach for this problem? Thanks and any hint may help.

Answer 1

You asked : "I'd like to monitor the model performance over time, specifically whether the modeled Y variable still responds to the independent variables"

I have implemented a test for the constancy of parameters over time (extending the revered Chow Test) to test for break-points in time where the parameters of the model changed significantly.

In this way one can assess the collective homogeneity of model parameters over tine.