Skip to main content
added 534 characters in body
Source Link
veda Lu
  • 11
  • 1
  • 3

I want to do Kolmogorov -Smirnov test to see whether my data follows a particular distribution or not? When I fitted my data to lognormal distribution it fitted well. But When I am doing ks test, it is rejecting the null hypothesis. Also, my sample size is very large like 1047304 samples. So I took 1000 samples where I thought both empirical and observed data are mostly correlated. The thing is

  1. I haven't studied any statistics. In wiki, they gave that the Kolmogorov–Smirnov statistic quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution. As far as I understood, for each x value, we measure corresponding F(x) or cumulative probability values and these values vary according to the type of distribution we choose. We measure the distance between F(X) values of theoretical distribution and the reference distribution. Am I right?

  2. I have used following code in Matlab for kstest

    cdf2=logncdf(area1out,-3.60186,0.347719); cdfplot(area1out) hold on; cdfplot(cdf2) [h,p,ksstat2]=kstest2(area1out,cdf2);

'area1out' is my data. The values -3.60186 and 0.347719 are the parameters estimated after fitting the data to the lognormal distribution. The result is h=1, p=0, and ksstat=0.9297

  1. I got the output as follows

This figure shows the good fit of the lognormal distribution

enter image description here

The second picture is showing the difference between both CDFs I guess. enter image description here

When I took 1000 samples the output is like this.

I am feeling like I am thinking wrong and making a big blunder. please help me to understand ks test practically. I will read the whole theory behind it but help me to understand this in a simple way as I am very new to this and had to work on this problem.

EDIT: No matter how many different combinations of samples and (with small sample size) I have taken, the distance is very large. When I zoomed my data it is like this and the corresponding test results are in 2nd figure. enter image description here enter image description here

Is there a problem with the parameter values I have taken to estimate the CDF? Or my data really not fitted lognormal distribution?

I want to do Kolmogorov -Smirnov test to see whether my data follows a particular distribution or not? When I fitted my data to lognormal distribution it fitted well. But When I am doing ks test, it is rejecting the null hypothesis. Also, my sample size is very large like 1047304 samples. So I took 1000 samples where I thought both empirical and observed data are mostly correlated. The thing is

  1. I haven't studied any statistics. In wiki, they gave that the Kolmogorov–Smirnov statistic quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution. As far as I understood, for each x value, we measure corresponding F(x) or cumulative probability values and these values vary according to the type of distribution we choose. We measure the distance between F(X) values of theoretical distribution and the reference distribution. Am I right?

  2. I have used following code in Matlab for kstest

    cdf2=logncdf(area1out,-3.60186,0.347719); cdfplot(area1out) hold on; cdfplot(cdf2) [h,p,ksstat2]=kstest2(area1out,cdf2);

'area1out' is my data. The values -3.60186 and 0.347719 are the parameters estimated after fitting the data to the lognormal distribution. The result is h=1, p=0, and ksstat=0.9297

  1. I got the output as follows

This figure shows the good fit of the lognormal distribution

enter image description here

The second picture is showing the difference between both CDFs I guess. enter image description here

When I took 1000 samples the output is like this.

I am feeling like I am thinking wrong and making a big blunder. please help me to understand ks test practically. I will read the whole theory behind it but help me to understand this in a simple way as I am very new to this and had to work on this problem.

I want to do Kolmogorov -Smirnov test to see whether my data follows a particular distribution or not? When I fitted my data to lognormal distribution it fitted well. But When I am doing ks test, it is rejecting the null hypothesis. Also, my sample size is very large like 1047304 samples. So I took 1000 samples where I thought both empirical and observed data are mostly correlated. The thing is

  1. I haven't studied any statistics. In wiki, they gave that the Kolmogorov–Smirnov statistic quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution. As far as I understood, for each x value, we measure corresponding F(x) or cumulative probability values and these values vary according to the type of distribution we choose. We measure the distance between F(X) values of theoretical distribution and the reference distribution. Am I right?

  2. I have used following code in Matlab for kstest

    cdf2=logncdf(area1out,-3.60186,0.347719); cdfplot(area1out) hold on; cdfplot(cdf2) [h,p,ksstat2]=kstest2(area1out,cdf2);

'area1out' is my data. The values -3.60186 and 0.347719 are the parameters estimated after fitting the data to the lognormal distribution. The result is h=1, p=0, and ksstat=0.9297

  1. I got the output as follows

This figure shows the good fit of the lognormal distribution

enter image description here

The second picture is showing the difference between both CDFs I guess. enter image description here

When I took 1000 samples the output is like this.

I am feeling like I am thinking wrong and making a big blunder. please help me to understand ks test practically. I will read the whole theory behind it but help me to understand this in a simple way as I am very new to this and had to work on this problem.

EDIT: No matter how many different combinations of samples and (with small sample size) I have taken, the distance is very large. When I zoomed my data it is like this and the corresponding test results are in 2nd figure. enter image description here enter image description here

Is there a problem with the parameter values I have taken to estimate the CDF? Or my data really not fitted lognormal distribution?

Source Link
veda Lu
  • 11
  • 1
  • 3

Distribution fitted well but kolmogorov -smirnov test not showing right results? Am I doing it right?

I want to do Kolmogorov -Smirnov test to see whether my data follows a particular distribution or not? When I fitted my data to lognormal distribution it fitted well. But When I am doing ks test, it is rejecting the null hypothesis. Also, my sample size is very large like 1047304 samples. So I took 1000 samples where I thought both empirical and observed data are mostly correlated. The thing is

  1. I haven't studied any statistics. In wiki, they gave that the Kolmogorov–Smirnov statistic quantifies a distance between the empirical distribution function of the sample and the cumulative distribution function of the reference distribution. As far as I understood, for each x value, we measure corresponding F(x) or cumulative probability values and these values vary according to the type of distribution we choose. We measure the distance between F(X) values of theoretical distribution and the reference distribution. Am I right?

  2. I have used following code in Matlab for kstest

    cdf2=logncdf(area1out,-3.60186,0.347719); cdfplot(area1out) hold on; cdfplot(cdf2) [h,p,ksstat2]=kstest2(area1out,cdf2);

'area1out' is my data. The values -3.60186 and 0.347719 are the parameters estimated after fitting the data to the lognormal distribution. The result is h=1, p=0, and ksstat=0.9297

  1. I got the output as follows

This figure shows the good fit of the lognormal distribution

enter image description here

The second picture is showing the difference between both CDFs I guess. enter image description here

When I took 1000 samples the output is like this.

I am feeling like I am thinking wrong and making a big blunder. please help me to understand ks test practically. I will read the whole theory behind it but help me to understand this in a simple way as I am very new to this and had to work on this problem.