4
$\begingroup$

From what I could gather

  • Mixture: if $X_i\sim^{iid} f_i$, then W is a mixture with $f_W =\sum \frac{f_i}{n}$. This definition could also be for the CDF instead of the density.
  • Convolution: To make it simpler, lets assume if $X_i\sim^{iid} N(\mu_i,\sigma^2_i)$, then $W=\sum X_i\sim N(\sum \mu_i,\sum \sigma^2_i)$. We could write this in terms of densities.

What I don't get is the practical intuition for these definitions.

For example: We have two machines, each producing observations $X_i\sim f_i$.

If there's probability $p$ that machine 1 is chosen, how do we model our final observation $W$? As a convolution or a mixture? And what changes should we do to our problem/situation to model it as the other possibility?

Any help would be appreciated.

$\endgroup$
  • 2
    $\begingroup$ Convolution is for when you add independent variables. ("What's the distribution of the total score or both tests?"). A mixture is for when there's a chance of seeing something from this distribution or that one. Your example at the end is clearly not a sum. $\endgroup$ – Glen_b -Reinstate Monica Aug 1 '16 at 21:35
  • $\begingroup$ @Glen_b by the way, is it usual notation for mixtures to write for example $X=0.5N(\mu_1,\sigma^2_1)+0.5N(\mu_2,\sigma^2_2)$? Is this notation also possible for sum of r.v.? $\endgroup$ – An old man in the sea. Aug 1 '16 at 22:18
  • $\begingroup$ Well, no. X does not equal that mixture, X has that distribution, OR it's the distribution of X.that equals that mixture. So you could write $X\sim ...$ or you might write say $f_X(x)=...$ if you see $N()$ as representing the density function of a normal. For a sum of random variables you write the sum in terms of the random variables. e.g. define independent $X_i\sim N(\mu_i,\sigma_i^2)$, then $Y=X_1+X_2$. In each case you're adding quite different things (a random variable is quite distinct from its density or its distribution). You have to keep straight what it is that you're adding! $\endgroup$ – Glen_b -Reinstate Monica Aug 2 '16 at 1:53
  • $\begingroup$ In short: a linear combination of random variables is very different from a linear combination of densities or a linear combination of distribution functions. [If you wanted to add together something when you need to perform convolution, you would add the cumulant generating functions. If $Y$ and $Y$ are independent and $Z=X+Y$ (so we'd use convolution integral to get the density of $Z$), then $K_Z = K_X+K_Y$ ... as long as the cumulant generating functions exist] $\endgroup$ – Glen_b -Reinstate Monica Aug 2 '16 at 1:55
  • $\begingroup$ @Glen_b my mistake... I meant $X\sim$. Thanks for the help. $\endgroup$ – An old man in the sea. Aug 2 '16 at 12:22
4
$\begingroup$

The mathematical difference is simple (and you probably got that already). A mixture distribution has a density which is a weighted sum of other probability densities (often from the same class) whereas a convolution is a sum of random variables.

The intuition for a mixture can be illustrated (in line with your example) as follows: Let's say you have $k$ sensors each of which draws an independent measurement $X_i\sim f_i$ (for $i=1,\ldots,k$). Furthermore, let's say that you are only observing the measurement $W$ of one of these sensors $s$, i.e. $W=X_s$ by choosing the sensor s randomly (from $1,\ldots,k$) using a discrete uniform distribution. Then, the density of $W$ given that $s$ is known corresponds to $f_s$. Now, as $s$ is not known, we can consider all possible values for s and we obtain for the density a mixture distribution $$f_W(x) = P(s=1)\cdot f_1(x) + \ldots + P(s=k)\cdot f_k(x)=\frac{1}{k}\sum_{i=1}^k f_i(x)$$

In the sensor example you would have a convolution if you would take all measurements (assuming them to be independent) and sum them up,i.e., $W=X_1+\ldots+X_k$. This may happen as part of averaging the sensor measurements. Then, the resulting density is $$f_W(x) = f_{X_1+\ldots+X_k}(x) = (f_1 * f_2 * \ldots*f_k)(x)\ ,$$ where $*$ denotes the convolution operation.

Side note: For the actual averaging procedure, we would have to divide the sum by the number of sensors. That is $W=k^{-1}(X_1+\ldots+X_k)$ and $$f_W(x) = f_{k^{-1}(X_1+\ldots+X_k)}(x) = k^{-1}(f_1 * f_2 * \ldots*f_k)(k\cdot x)\ .$$

$\endgroup$
  • $\begingroup$ Igor, thanks for your answer. How would we change that situation of the sensors to make it a convolution, i.e., a simple adding of r.v.? Would we only take one measurement with prob. $p_i$, instead of the the $k$ measurements in your narrative? $\endgroup$ – An old man in the sea. Aug 1 '16 at 8:14
  • $\begingroup$ I just added that information to my original response. $\endgroup$ – Igor Aug 1 '16 at 9:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.