Difference between a mixture of distributions and a convolution. Interpretation in a applied setting

From what I could gather

• Mixture: if $X_i\sim^{iid} f_i$, then W is a mixture with $f_W =\sum \frac{f_i}{n}$. This definition could also be for the CDF instead of the density.
• Convolution: To make it simpler, lets assume if $X_i\sim^{iid} N(\mu_i,\sigma^2_i)$, then $W=\sum X_i\sim N(\sum \mu_i,\sum \sigma^2_i)$. We could write this in terms of densities.

What I don't get is the practical intuition for these definitions.

For example: We have two machines, each producing observations $X_i\sim f_i$.

If there's probability $p$ that machine 1 is chosen, how do we model our final observation $W$? As a convolution or a mixture? And what changes should we do to our problem/situation to model it as the other possibility?

Any help would be appreciated.

• Convolution is for when you add independent variables. ("What's the distribution of the total score or both tests?"). A mixture is for when there's a chance of seeing something from this distribution or that one. Your example at the end is clearly not a sum. – Glen_b -Reinstate Monica Aug 1 '16 at 21:35
• @Glen_b by the way, is it usual notation for mixtures to write for example $X=0.5N(\mu_1,\sigma^2_1)+0.5N(\mu_2,\sigma^2_2)$? Is this notation also possible for sum of r.v.? – An old man in the sea. Aug 1 '16 at 22:18
• Well, no. X does not equal that mixture, X has that distribution, OR it's the distribution of X.that equals that mixture. So you could write $X\sim ...$ or you might write say $f_X(x)=...$ if you see $N()$ as representing the density function of a normal. For a sum of random variables you write the sum in terms of the random variables. e.g. define independent $X_i\sim N(\mu_i,\sigma_i^2)$, then $Y=X_1+X_2$. In each case you're adding quite different things (a random variable is quite distinct from its density or its distribution). You have to keep straight what it is that you're adding! – Glen_b -Reinstate Monica Aug 2 '16 at 1:53
• In short: a linear combination of random variables is very different from a linear combination of densities or a linear combination of distribution functions. [If you wanted to add together something when you need to perform convolution, you would add the cumulant generating functions. If $Y$ and $Y$ are independent and $Z=X+Y$ (so we'd use convolution integral to get the density of $Z$), then $K_Z = K_X+K_Y$ ... as long as the cumulant generating functions exist] – Glen_b -Reinstate Monica Aug 2 '16 at 1:55
• @Glen_b my mistake... I meant $X\sim$. Thanks for the help. – An old man in the sea. Aug 2 '16 at 12:22

The mathematical difference is simple (and you probably got that already). A mixture distribution has a density which is a weighted sum of other probability densities (often from the same class) whereas a convolution is a sum of random variables.

The intuition for a mixture can be illustrated (in line with your example) as follows: Let's say you have $k$ sensors each of which draws an independent measurement $X_i\sim f_i$ (for $i=1,\ldots,k$). Furthermore, let's say that you are only observing the measurement $W$ of one of these sensors $s$, i.e. $W=X_s$ by choosing the sensor s randomly (from $1,\ldots,k$) using a discrete uniform distribution. Then, the density of $W$ given that $s$ is known corresponds to $f_s$. Now, as $s$ is not known, we can consider all possible values for s and we obtain for the density a mixture distribution $$f_W(x) = P(s=1)\cdot f_1(x) + \ldots + P(s=k)\cdot f_k(x)=\frac{1}{k}\sum_{i=1}^k f_i(x)$$

In the sensor example you would have a convolution if you would take all measurements (assuming them to be independent) and sum them up,i.e., $W=X_1+\ldots+X_k$. This may happen as part of averaging the sensor measurements. Then, the resulting density is $$f_W(x) = f_{X_1+\ldots+X_k}(x) = (f_1 * f_2 * \ldots*f_k)(x)\ ,$$ where $*$ denotes the convolution operation.

Side note: For the actual averaging procedure, we would have to divide the sum by the number of sensors. That is $W=k^{-1}(X_1+\ldots+X_k)$ and $$f_W(x) = f_{k^{-1}(X_1+\ldots+X_k)}(x) = k^{-1}(f_1 * f_2 * \ldots*f_k)(k\cdot x)\ .$$

• Igor, thanks for your answer. How would we change that situation of the sensors to make it a convolution, i.e., a simple adding of r.v.? Would we only take one measurement with prob. $p_i$, instead of the the $k$ measurements in your narrative? – An old man in the sea. Aug 1 '16 at 8:14
• I just added that information to my original response. – Igor Aug 1 '16 at 9:06