1
$\begingroup$

I have a real-world time series process with an dependent variable Y responds to multiple independent variables (not necessary a linear response but can assume so if necessary). A non-statistical model was built to predict the process with the same set of the independent variables. I'd like to monitor the model performance over time, specifically whether the modeled Y variable still responds to the independent variables the same way as of the real-world process.

I could build another statistical model for the residuals (real Y - modeled Y) and look at the model coefficients and their statistical significance. But there are some issues associated with the approach. I may not be able to flag model biases or noises. Just wondering whether there is more statistically rigorous approach for this problem? Thanks and any hint may help.

$\endgroup$

1 Answer 1

1
$\begingroup$

You asked : "I'd like to monitor the model performance over time, specifically whether the modeled Y variable still responds to the independent variables"

I have implemented a test for the constancy of parameters over time (extending the revered Chow Test) to test for break-points in time where the parameters of the model changed significantly.

In this way one can assess the collective homogeneity of model parameters over tine.

$\endgroup$
2
  • $\begingroup$ Telling us you have an answer is not providing an answer! It's just teasing... . $\endgroup$
    – whuber
    Nov 28, 2018 at 20:00
  • 1
    $\begingroup$ I should have referred to stats.stackexchange.com/questions/342512/… has a discussion of the CHOW Test as applied to testing the constancy of parameters for different time slices (groups) $\endgroup$
    – IrishStat
    Nov 28, 2018 at 21:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.