Skip to main content
Notice removed Canonical answer required by rapaio
Bounty Ended with QuantIbex's answer chosen by rapaio
Notice added Canonical answer required by rapaio
Bounty Started worth 50 reputation by rapaio
simplified a bit the question
Source Link
rapaio
  • 7.1k
  • 31
  • 47

I know about PIT, but this works only when you know the distribution, or at least have a strong hint. What I am trying to achieve is to transform a given sample into an equivalent sample with continuous standard uniform distribution.

My approach is described below:

Denote with $S_n = \{ x_1, x_2, .., x_n\}$ my initialI have a sample, with of size $n$. Now, I choose arbitrary a value $m$, and estimate $m+1$ quantiles from the sample (for example, if $m=4$, I compute quantiles for $\{0.0, 0.25, 0.5, 0.75, 1.0\}$$\{0, .25, .5, .75, 1\}$). I use basically the sameThe procedure is described in Wikipedia.

Using quantiles I transform each $x_i$. If $x_i$ happens to be exactly a computed quantile, than I know precisely its equivalent value. If $x_i$ is between two estimated quantiles, otherwise I interpolate the equivalent value linearly.

The question is: can I use Kolmogorov-Smirnov one sample goodness-of-fit to find a proper value for $m$ (the number computed quantiles)?

I know that I do a lot of estimations, one over the top of other, so I was wondering if will work something like this. The reason why I would want to save some computation time for a give transformation.

I haveI've done a small simulation. I build a random sample from standard normal with $10^6$ values. I applied the described transformation and make KS test for some values of $m$. The results looks like:

  m         D   p-value
100  0.006090   0.000000000000 ***
200  0.003151   0.000000004733 ***
300  0.001991   0.000720875707 ***
400  0.001484   0.024403417075 *
500  0.001057   0.213437843144

It looks like I can do a PIT on sample with only 500 interpolation points.

The question is: can I use Kolmogorov-Smirnov one sample goodness-of-fit to find a proper value for $m$ (the number computed quantiles)?

I know about PIT, but this works only when you know the distribution, or at least have a strong hint. What I am trying to achieve is to transform a given sample into an equivalent sample with continuous standard uniform distribution.

My approach is described below:

Denote with $S_n = \{ x_1, x_2, .., x_n\}$ my initial sample, with size $n$. Now, I choose arbitrary a value $m$, and estimate $m+1$ quantiles from the sample (for example, if $m=4$, I compute quantiles for $\{0.0, 0.25, 0.5, 0.75, 1.0\}$). I use basically the same procedure described in Wikipedia.

Using quantiles I transform each $x_i$. If $x_i$ happens to be exactly a computed quantile, than I know precisely its equivalent value. If $x_i$ is between two estimated quantiles, I interpolate the equivalent value linearly.

The question is: can I use Kolmogorov-Smirnov one sample goodness-of-fit to find a proper value for $m$ (the number computed quantiles)?

I know that I do a lot of estimations, one over the top of other, so I was wondering if will work something like this. The reason why I would want to save some computation time for a give transformation.

I have done a small simulation. I build a random sample from standard normal with $10^6$ values. I applied the described transformation and make KS test for some values of $m$. The results looks like:

  m         D   p-value
100  0.006090   0.000000000000 ***
200  0.003151   0.000000004733 ***
300  0.001991   0.000720875707 ***
400  0.001484   0.024403417075 *
500  0.001057   0.213437843144

I know about PIT, but this works only when you know the distribution, or at least have a strong hint. What I am trying to achieve is to transform a given sample into an equivalent sample with continuous standard uniform distribution.

I have a sample of size $n$. I choose arbitrary a value $m$, and estimate $m+1$ quantiles (for example, if $m=4$, I compute quantiles for $\{0, .25, .5, .75, 1\}$). The procedure is described in Wikipedia.

Using quantiles I transform each $x_i$. If $x_i$ happens to be exactly a computed quantile, than I know precisely its equivalent value, otherwise I interpolate the equivalent value linearly.

I've done a small simulation. I build a random sample from standard normal with $10^6$ values. I applied the described transformation and make KS test for some values of $m$. The results looks like:

  m         D   p-value
100  0.006090   0.000000000000 ***
200  0.003151   0.000000004733 ***
300  0.001991   0.000720875707 ***
400  0.001484   0.024403417075 *
500  0.001057   0.213437843144

It looks like I can do a PIT on sample with only 500 interpolation points.

The question is: can I use Kolmogorov-Smirnov one sample goodness-of-fit to find a proper value for $m$ (the number computed quantiles)?

Source Link
rapaio
  • 7.1k
  • 31
  • 47

PIT on a sample with m bins, and KS test used to estimate a good value for m

I know about PIT, but this works only when you know the distribution, or at least have a strong hint. What I am trying to achieve is to transform a given sample into an equivalent sample with continuous standard uniform distribution.

My approach is described below:

Denote with $S_n = \{ x_1, x_2, .., x_n\}$ my initial sample, with size $n$. Now, I choose arbitrary a value $m$, and estimate $m+1$ quantiles from the sample (for example, if $m=4$, I compute quantiles for $\{0.0, 0.25, 0.5, 0.75, 1.0\}$). I use basically the same procedure described in Wikipedia.

Using quantiles I transform each $x_i$. If $x_i$ happens to be exactly a computed quantile, than I know precisely its equivalent value. If $x_i$ is between two estimated quantiles, I interpolate the equivalent value linearly.

The question is: can I use Kolmogorov-Smirnov one sample goodness-of-fit to find a proper value for $m$ (the number computed quantiles)?

I know that I do a lot of estimations, one over the top of other, so I was wondering if will work something like this. The reason why I would want to save some computation time for a give transformation.

I have done a small simulation. I build a random sample from standard normal with $10^6$ values. I applied the described transformation and make KS test for some values of $m$. The results looks like:

  m         D   p-value
100  0.006090   0.000000000000 ***
200  0.003151   0.000000004733 ***
300  0.001991   0.000720875707 ***
400  0.001484   0.024403417075 *
500  0.001057   0.213437843144