R: How to calculate probabilities for data transformed into normal?

Question

I basically don't know how to proceed with my transformed dataset.

I have a dataset of 500 observations, positive skew. I have used the bestNormalize package to normalise it (it chose Box-Cox, if that matters) and so now I have a nice dataset of new values, normal distribution. (The result is in a bestNormalise object, but I have also made a df from it, which is what you can see below.)

How do I calculate probabilities for new "original values" from this? I.e 113883785 equals 0.17045703 in the transformed data, and I can use pnorm to calculate the probability for 0.17045703, which should be the probability for 113883785 in the original dataset. But how do I go about getting the probability for 130000, or 586, or any number not exactly in the original dataset? Help would be much appreciated. Thank you!

Answer 1

You can use the predict() function from the bestNormalize class to get the corresponding point in the transformed dataset.

Let's demonstrate with a known distribution (Gamma) and transform it to standard normal distribution. The following figure shows that the cumulative probabilities on the transformed data computed with pnorm() match with those computed with pgamma() from the original data, as expected.

library(bestNormalize)
x <- sort(rgamma(100, 1, 1))
BN_obj <- bestNormalize(x)
x.t <- BN_obj$x.t #predict(BN_obj, x)
plot(pgamma(x,1,1), pch=19, xlab='index', ylab='probability')
points(pnorm(x.t), pch=19, col='red')
legend('top', c("P(X<=x)", "P(Xt<=xt)"), col = c(1, 2),
       text.col = "green4", pch = c(19, 19),
       merge = TRUE, bg = "gray90")

Now, let's say we have a new data point x_test from the original dataset and we want to compute the cumulative probability using pnorm() from the transformed dataset.

• You can do it with predict() to apply the transformation to a new test datapoint (taken from the original distribution)

• Then use pnorm() to compute the cumulative probability.

• It will be approximately same as the cumulative probability computed from the original Gamma distribution, as shown in the next code snippet.

  x_test <- 1.2345
  x.t_test <- predict(BN_obj, newdata = x_test)
  pnorm(x.t_test)
  # [1] 0.6891166
  pgamma(x_test, 1, 1)
  # [1] 0.7090198
  all.equal(pnorm(x.t_test), pgamma(x_test, 1, 1), tolerance=0.03)
  # TRUE