Sigmoid transform of Gaussian random variables does not have an easy calculation of covariance.

Specifically, I'm looking for a function transform $$f: \mathbb{R}\rightarrow [0, 1]$$ such that it can transform a Gaussian variable to a probability.

I have two Gaussian random variables $(x_1, x_2)$ with non-zero correlation. After transformation, I want to calculate the correlation of $(f(x_1), f(x_2))$.

UPDATE: I'm fitting a model $f(Gaussian(\mu, \Sigma))$ on some data. After I do model fitting and obtain $\mu$ and $\Sigma$, I need to calculate the covariance of my model, so I need to calculate $cov(f(x_1), f(x_2))$. The function $f$ satisfies the modeling need as long as it has a shape like the sigmoid function. But covariance calculation requires a special form of $f$.

Any hints?

  • $\begingroup$ 1. logit is log(p/(1-p)). That's not going to make sense on a normal; you probably mean the antilogit exp(x)/(1+exp(x)] ... 2. to what end? What's the ultimate aim? Are approximations okay? $\endgroup$ – Glen_b Oct 8 '18 at 2:53
  • $\begingroup$ Yes, I mean antilogit or sigmoid. The ultimate goal is the calculation of the covariance of $(f(x_1), f(x_2))$. $\endgroup$ – Liping Liu Oct 8 '18 at 3:58
  • 1
    $\begingroup$ You said that in your post already -- but if the transformation $f$ is essentially any arbitrary cdf (also as in your post), what purpose does that serve?? What's the point in having the covariance between arbitrarily transformed variates? $\endgroup$ – Glen_b Oct 8 '18 at 5:27
  • $\begingroup$ Perhaps defining a copula might solve your problem? Are you specifically looking for a closed form solution for the covariance? As Glen_b pointed out, it doesn't make sense to find the covariance of $(f_1, f_2)$ after applying any arbitrary transformation $f$. Perhaps what you could do is to apply the appropriate marginal CDFs to transform $$ (\Phi_1(x_1), \Phi_2(x_2)) \mapsto (u_1, u_2) $$. You'd then be able to calculate some measure of correlation like the pearson or the kendall tau $\endgroup$ – InfProbSciX Oct 8 '18 at 10:24
  • 1
    $\begingroup$ No, the normal cdf doesn't have an analytical expression. This exercise seems a bit convoluted, even slightly different-looking sigmoid-like functions can enforce very different behavior. Even if you do find a function, be careful with how you use it. E.g. a "quickly varying" sigmoid (one with a massive slope when it curves up) would basically make a lot of your variables go to 0 or 1, and the concept of correlation might not even make sense when applied to such a dataset $\endgroup$ – InfProbSciX Oct 8 '18 at 19:35

After some thought, I think I find a solution:

The transformation can be:

$f(x) = \left\{\begin{array}{ll} \frac{1}{2}\exp(x) & \mbox{if } x \le 0 \\ 1 - \frac{1}{2} \exp(-x) & \mbox{if } x >0 \end{array} \right.$

One can take the expectation of $f(x_1) \cdot f(x_2)$ with respect to a bi-variate Gaussian distribution of $(x_1, x_2)$.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.