0
$\begingroup$

I am trying to build an estimator based on my distribution that has 1585 values in it. The distribution itself kinda looks normal so I created a KDE and MLE estimators to find the best fit for my distribution.

KDE Estimate:

density = kde.gaussian_kde(odu, bw_method=None)

MLE

alpha = norm.fit(odu)[0]
beta = norm.fit(odu)[1]
e = norm(alpha,beta)

After running KS test for both, I get these values: (Note odu is the dataframe I am comparing too).

KDE KS Test

kstest(odu,density)

output:

KstestResult(statistic=0.9999938489747389, pvalue=0.0)

MLE kstest

kstest(odu,e.cdf)

Output:

KstestResult(statistic=0.11133223603089792, pvalue=3.9032020068459515e-17)

I am confused with the output kstest gives for both values. Lower the p-value means the null hypothesis is false, that means the distribution is a good fit? My professor says D-value close to 1 means its a good fit and the lower p-value means it has a high level of confidence. Is this correct? Thank you The Distribution with KDE and MLE Estimators

Improve this question
$\endgroup$
4
  • $\begingroup$ Neither you nor your professor express the problem in a precisely correct way. But if the p-value is low, below a significance level that you choose means that you reject the null hypothesis that the two distributions are the same. That means that you have a good fit. So you are right in that regard. Your Professor is wrong. A large value of the test statistic D indicates a poor fit. It corresponds to a low p-value. Technically we say that if the p-value is above the significance level we can't reject the null hypothesis. $\endgroup$
    – Michael R. Chernick
    Commented Oct 22, 2019 at 19:05
  • $\begingroup$ Awesome thank you! Quick question, should I use cdf or pdf in kstests? $\endgroup$
    – Tyler Oleson
    Commented Oct 22, 2019 at 19:31
  • 1
    $\begingroup$ The Kolmogorov-Sminrnov test uses the cumulative distribution for a known hypothesized distribution & the empiric cumulative distribution for the sample distribution based on your data. It does not involve probability density functions. $\endgroup$
    – Michael R. Chernick
    Commented Oct 22, 2019 at 19:35
  • $\begingroup$ It's not okay to use a Kolmogorov-Smirnov test to test the fit of a density estimate to the sample you estimated it from; the Kolmogorov-Smirnov test assumes a completely specified distribution, not an estimated one. $\endgroup$
    – Glen_b
    Commented Oct 23, 2019 at 5:12

1 Answer 1

Reset to default
1
$\begingroup$

There are a few problems here some of which are just terminology. You don't have a distribution, you have data. You want to estimate a distribution from your data. An MLE isn't a distribution, it's a type of estimation--the two distributions you are estimating are a KDE and a Gaussian (which you are estimating with MLE).

Using KS hypothesis testing, you cannot prove that a distribution is a good fit; you can only find evidence that it is a bad fit (you cannot accept the null hypothesis, only reject it). That is what has happened here, as Michael pointed out in the comments. This is completely unsurprising, as you have a large amount of data and it is unlikely that the Gaussian or KDE distributions you estimated are the ones that generated the data. The extremely low p-values for the KS test confirm this.

You should not be using a KS test to determine how to model data. The type of distribution you estimate should be chosen scientifically, based on what you know about the way the data were generated/collected.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.