For real datasets, where it's impossible to know the true inverse covariance, what are the methods of evaluating your inverse covariance estimator?

Possible answers:

  1. If the number of features is long enough, randomly separate them into a train and test set. (Followup question: Do you do your own inverse covariance on both of them and compare to see if they're consistent? Or do you try to truly invert the one with more data? In that case, don't you still risk errors as long as p << n?

  2. Calculate the log-likelihood of held out data. I'm wondering for this, what is the common practice for holding out data? For time series: do you hold out contiguous swaths, or randomly pick time points? For features that are not that associated (temperature, pressure, gene experession, etc) do you just randomly pick, or do you try to equally account for categorical, real, integer types of data?

  • $\begingroup$ It would help if you could tell possible answerers to what use you will put your inverse covariance matrix. Answers could depend on that. $\endgroup$ – kjetil b halvorsen May 12 '19 at 11:29
  • 1
    $\begingroup$ Thanks for your comment! Actually this project ended a while ago so I'm no longer searching for any specific answers. But it would be interesting to have any "general wisdom" or tricks-of-the-trade here! $\endgroup$ – Y. S. May 13 '19 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.