Can a sample average be multimodal? Suppose $X_1,\ldots,X_n$ are iid with a well defined expectation. Is it possible for the sample mean $(1/n)\sum_{i=1}^nX_i$ to be multimodal for all $n$? Looking at simple examples I can see that for small $n$ the sample mean can be multimodal, but eventually it goes away. Is this always the case? I thought maybe using the Fourier transform this could be shown, but did not succeed. I don't want to assume the variables have a finite variance but I guess some smoothness-type assumptions on the common density will be necessary.
 A: The answer is yes, it is possible for all the sample mean distributions to be multimodal.
The idea is to exhibit a distribution of the $X_i$ that has rapidly increasing gaps in its support, so that no matter how many of the $X_i$ you might sum, there eventually will be a gap so large that the sum cannot fill it: that is, there will be places within the gap where the sum has zero probability.  That would place some of the probability of the sum to the left of that point (on a number line) and some to the right; and thus the sum's distribution will have at least two modes.  That makes the distribution of the mean at least bimodal.
The rest of this post fleshes out this idea.  If you read it, you should have no trouble finding sequences of weights $p_i$ and displacements $x_i$ for which the $X_i$ have finite variances (and therefore finite expectations, as required in the question): For instance, just pick $x_i$ and for the $p_i$ set $q_i = 2^{-i}/(x_i+1)^2,$ normalizing them to sum to unity.
To illustrate, I set $q_i = 2^{-i}/(x_i+1)$ (so that the component weights don't taper off too quickly, but still ensuring a finite expectation) and graphed the density of $log(X)$ on a logarithmic axis (which is needed to depict the huge range of values in both dimensions).  You can see how the gaps increase faster than exponentially.

Separate components of the distribution are depicted in different colors.  Although the graph eventually looks like a series of spikes, that is only because of the lack of resolution: each apparent spike has the shape of a Beta$(5,5)$ distribution and its width (therefore) is $1.$
Those who rely on the Central Limit Theorem or laws of large numbers to think about this problem might find this example paradoxical, but it isn't: those theorems concern properties of distributions in the large -- global properties -- whereas the number of modes of a distribution is a "hyperlocal" property, typically characterizing the distribution only in an infinitesimal part of its support.  Intuitively, you can take any "nice" density function and put as many tiny bumps on it as you like without appreciably changing any of its moments, its tail behavior, or most other global properties of the distribution.  Those theorems are also concerned about limiting properties, whereas the question is not about a limit at all.

A counterexample must be a distribution (common to all the $X_i$) for which the distributions of infinitely many of the partial sums
$$Y_n = \sum_{i=1}^n X_i$$
are multimodal.  (The division by $n$ to form the means does not change the shape of the distribution and therefore does not change the number of modes.)
To be a satisfactory example -- that is, not some kind of exceptional "pathology" uncharacteristic of the underlying ideas -- we ought to insist that this distribution be continuous with a continuous density so that modes (and therefore the modes of all the $Y_n$) are clearly defined.
One counterexample is determined by three sequences with these properties:

*

*Random variables: Let $Z_i$ be a sequence of iid random variables supported on the interval $[0,1]$ with common distribution function $F$ having at least one mode. Any Beta$(a,b)$ distribution will serve.


*Displacements: Let $x_i$ be a sequence of numbers where $x_1=0$ and for every $i\gt 1,$ $x_i \ge i+i^{i+1}.$


*Weights: Let $p_i,$ $i=1,2,\ldots,$ be a sequence of positive probabilities summing to $1$ (such as a geometric distribution $p_i = 2^{-i}$).
We will shift each variable $Z_i$ to the location $x_i$ and weight it by $p_i$ to form a mixture with distribution function
$$G(x) = \sum_{i=1}^\infty p_i F(x-x_i).$$
Because all the $p_i$ are positive, the support of this distribution is the union of the intervals $[x_i, x_i+1],$ which is non-negative and unbounded.
The counterexample is a sequence of iid variables $X_i$ with common distribution $G.$
We need to investigate its modes. We can identify modes of $G$ by finding two nonempty intervals $(a_0,b_0)$ and $(a_1,b_1)$ with $a_0\lt b_0 \lt a_1\lt b_1$ for which $G$ has no probability, but $G$ has positive probability on the gap between them, $[b_0,a_1]:$ a mode of $G$ must lie in that gap.
Let $F^{(n)}$ be the distribution of the sum of any $n$ distinct $Z_i.$  Note, because this is crucial to the argument, that the support of $F^{(n)}$ is contained in the interval $[0,n].$
As a matter of notation, when $\mathcal{I}$ is any multiset of indexes, let
$$x_{\mathcal I} = \sum_{i\in\mathcal I} x_i$$
be the sum of the displacements indexed by $\mathcal I.$
Because each $X_i$ is a mixture of the $Z_j,$ $Y_n$ is itself a mixture of sums of collections of $n$ of the shifted versions of $Z_j.$  The mixture components of $Y_n$ are thereby determined by multisets of $n$ indexes $\mathcal I$ and each component's distribution function is
$$F^{(n)}\left(x - x_{\mathcal I}\right),$$
whose support is contained in the shifted interval $[x_{\mathcal I}, x_{\mathcal I}+n].$
There are no more than $(n-1)^n$ such multisets corresponding to components with supports in $[0, x_n],$ because only choices from the set $\{x_1,x_2,\ldots, x_{n-1}\}$ yield sums less than $x_n.$
The support of $Y_n$ is contained within the union of supports of these component distributions.  Let's examine the support of $Y_n$ within the interval $[n, x_n).$  Its size (Lebesgue measure) cannot exceed the sum of lengths of the component supports and therefore cannot be greater than $n(n-1)^n \lt n^{n+1}.$  Since we have required $x_n \gt n+n^{n+1}$ for all $n,$ there must be an interval of positive measure in $[n, x_n)$ where $Y_n$ has no probability.
On the other hand, $Y_n$ has zero probability in $(-\infty, 0)$ and, since $x_1=0,$ positive probability in $[0,n]$ (where its weight is $p_1^n\gt 0$).  Moreover, the support of $Y_n$ is unbounded because the (common) support of the $X_i$ is unbounded.  We have thereby located at least one mode in $[0,x_n)$ and another in $[x_n,\infty).$  Consequently,

For all $n$ there must be at least one mode of $Y_n$ in the interval $[0,x_n)$ and there must be at least one mode in the interval $[x_n,\infty):$ that is, $Y_n$ has a multimodal distribution, QED.

Here is the graph corresponding to the initial illustration for sample sizes of $n=10^4:$

The first five components have merged into a density with no gaps (as apparent from the mixing of the colors in that part of the graph), but then--because each apparent spike has a width of only $10^4$--the gaps begin and continue ad infinitum.  This makes it obvious that every one of these sampling distributions has infinitely many modes.
