I have a dataset with count data predictors and a marker on their reliability or completeness. With 2 options: 0 - Data is not confirmed (could be higher); 1 - data is confirmed to be complete. The expected difference is not much, many 0-s can be complete. Around 2/3 of data are confirmed to be complete.

When I build a model on this - specifically I am building univariate GAMs (year of origin as predictor, expected trends are non-linear), how could I take this best into account?

I see some options:

  1. Include the completeness marker as a random effect. Importantly, the effect should be conditional on the predictor.
  2. Convert the completeness marker into weights on the data points. Say give only half the influence to data points not known to be complete. But the threshold could be too arbitrary?
  3. Include the completeness marker as an interaction term? However, I am interested in the main term only, data completeness is just something that I am hoping to control for, so this is probably not the way.

Are there any guidelines to follow here? I find it difficult to find any advice on this kind of data completeness markers. Is solution (1.) - to use completeness marker as a random effect, ok?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.