0
$\begingroup$

I have what is possible a naive question. I am current comparing various models (i.e. distributions). And the comparisons do not involve different distributions but rather how the model is fed the data.

For example, two models might be 1) a mixture of a gamma + exponential and 2) a mixture of a gamma + exponential wherein the minimum data point is subtracted. With respect to model 2). If I subtract the minimum data then I am left with a data point = 0. The model's fit fine and I get my MLE values.

However, If I am working with just a gamma and in one model I remove the minimum data point, then that data point = 0 causes errors in the MLE estimator. To get past this issue I simply remove the 0 data point and fit the distribution.

My question is, is there a problem with simply removing the data point (my worry is decreasing the size of my data set)? And should I do that for the gamma + exponential as well?

Thanks

Improve this question
$\endgroup$
3
  • $\begingroup$ In the case where the data is shifted exponential, subtracting the smallest data point (and removing the 0) leaves you with exponential. More generally, once you've subtracted, the zero itself won't contain any information - it's a consequence of your transformation and tells you nothing about the distribution. Whether subtracting the smallest is in general a good idea may be another matter, but if you're doing it, I'd suggest throwing out the 0 as a matter of course. It's the subtraction that decreased the effective size of the data set, not tossing the value that shows it. $\endgroup$
    – Glen_b
    Commented Jun 25, 2014 at 4:57
  • $\begingroup$ The minimum is thought to represent the minimum time necessary for the process of interest to complete. And in fact, when we subtract the minimum the Loglikelihood value is much larger than if the minimum remained in the set. However, in discussing this issue with my advisor, removing zeros is problematic not because it reduces sample size but because it biases the sample (because the zeros are systematic and not randomly generated in the data set). To get pass this issue, we subtract all but 0.005 of the minimum. It's a bit arbitrary but does the job and the distributions fit very well. $\endgroup$
    – user40335
    Commented Jun 26, 2014 at 6:49
  • $\begingroup$ If the data were exponential, keeping the zeros is a source of bias in estimation of the exponential mean, while tossing them out doesn't lead to bias. $\endgroup$
    – Glen_b
    Commented Jun 26, 2014 at 9:57

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.