Assuming I have two random non-independent vectors $A,B$ which are within [-1,1]. I want to approximate their sum of product by product of sums (everything is a dot product), i.e.

$\sum_{i=1}^NA_iB_i \approx \sum_{i=1}^NA_i \sum_{i=1}^N B_i$

I can see that this approximation is more and more better if the covariance between the two is closer to zero from the following formula

$Cov(XY) = E(XY) - E(X)E(Y)$

How do I achieve this without destroying any signal in $A,B$? I am only interested in sample estimate of covariance, i.e. even if reducing the sample size or centering and scaling helps to reduce the absolute value of covariance, it will work.


You arrived at the definition of independence of random variables on your own, congratulations! One formulation is $$E[AB]=E[A]E[B]$$ This is true when variables are independent of each other (not the other way around though).

You can't expect your approximation to be good if the variables are not independent or correlated.

| cite | improve this answer | |
  • $\begingroup$ Yes, however, I don't have independent random variables and I need to approximate the sum of product by product of sums, any pointers? $\endgroup$ – lvdp Feb 11 '19 at 15:07
  • $\begingroup$ Moreover, I don't need to exact equivalence, just an approximation (with an error ofcourse) $\endgroup$ – lvdp Feb 11 '19 at 15:14
  • $\begingroup$ @lvdp, you can't always get what you wanted, but if you try, sometimes, you can get what you need. So, why exactly do you need this approximation? $\endgroup$ – Aksakal Feb 11 '19 at 15:16
  • $\begingroup$ I am working on a problem where I need to sum individual vectors first and perturb them before I take their product and compare them to perturbing the sum of product, hope I am making some sense. I was thinking about centering, scaling, and standardizing as that will reduce the covariance without destroying much signal, which something like PCA might do. $\endgroup$ – lvdp Feb 11 '19 at 15:20
  • $\begingroup$ perturbation is from Normal distribution $N(0,\sigma^2)$, where $\sigma$ is always >0. Goal of perturbing is to make the vectors noisy and see how the noise impacts the model outcome (model is a neural network). $\endgroup$ – lvdp Feb 11 '19 at 15:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.