This question is motivated by a situation that we frequently encounter in economic variables. We believe that the overall distribution of some variable is well approximated by a particular distributional family, but we also think that interesting disjoint subsets of the population, e.g. incomes by state, or stock returns by industry, are well described by the same distributional family, but with varying parameters.
Am I correct in thinking that finite mixtures of distributions drawn from any specified commonly used continuous distributional family can never, strictly speaking, be another distribution from that family, regardless of the choice of underlying distribution and regardless of the mixing distribution (excluding the degenerate case)?
Say we have n distinct draws of parameters of the underlying distribution from the (possibly multivariate) mixing distribution. Are there rules of thumb, or, say, a way of bounding the difference between the finite mixing distribution and the compound distribution (with the same underlying and mixing distribution), or some other wisdom to be had from theory about finite mixtures? Intuitively I would expect a finite mixture to more nearly approximate the compound distribution as n increases, and also as ratio of the variance induced by the mixing distribution draws over the variance of the underlying distribution for fixed parameters becomes small, but I would find it useful to have a more formal statement of these relationships, if such exists.
I have another related question which I am not going to make part of this question because I am not currently able to state it precisely, but if anyone has knowledge or thoughts about it I would welcome them. It is this: It seems to me that if we wish to have some distributional family be a good, and a comparably good, approximation of the observed finite mixture and of the individual component distributions of that mixture, that should constrain either our choice of distributional family, or our choice of mixing distribution, or both. Is that right? If so, do we know what such constraints look like?