If I estimate the parameters of a distribution $\theta$ using MLE. Can I use the Kolmogorov Smirnov test to check the goodness of the fitted model?
3
-
$\begingroup$ If I understand correctly, the KS test works on samples, where as the MLE is a single estimate. I don't think this is going to work as you expect it to. $\endgroup$– Demetri PananosJun 13, 2020 at 18:11
-
$\begingroup$ @DemetriPananos Do you have a reference? $\endgroup$– VladimirJun 13, 2020 at 18:12
-
$\begingroup$ Can you add some more detail to the procedure you describe first? Are you planning to do MLE, sample from the distirbution, and then compare that sample with your data? $\endgroup$– Demetri PananosJun 13, 2020 at 18:30
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
1
This is an invalid procedure.
https://en.m.wikipedia.org/wiki/Kolmogorov–Smirnov_test
Scroll down to “Test with estimated parameters”. Disappointingly, they do not give much of a reference, but the book referenced in that paragraph might explain. (Cross Validated has many posts on this topic, too, though it would be nice to see it discussed in some primary literature.)
The gist is that you’re giving the reference distribution more similarity to the data than you should.
Yes, it is tempting to ask if your data fit a distribution by estimating the parameters from the data, but this is invalid. That you’re using a maximum likelihood estimator of the parameter(s) is not pertinent; this applies for any estimator. The good news is that this kind of goodness of fit testing is quite unhelpful, as many posts on Cross Validated discuss.
Invalid
Observe data with a mean of $7$ and a variance of $4$, then use KS to test if the data came from $N(7,4)$.
Valid...perhaps not helpful
Speculate that your data come from an exponential distribution with rate parameter $2$, observe data with a sample mean (so rate parameter) of $1$, and test the data against your speculated distribution of $exp(2)$.
(Remember to be careful about the parameter in the exponential distribution, as there is disagreement about whether the exponent is $-\lambda x$ for a mean of $1/\lambda$ or $-x/\beta$ for a mean of $\beta$. In my example, the mean and parameter happen to be $1$ no matter which exponent you’re prefer.)
$\begingroup$
$\endgroup$
1
The standard critical values will be too aggressive as the K-S test doesn’t take into account the sample error in the MLE estimates. You therefore need to bootstrap the critical values to form a valid inference.
