For iid normal random variables, $X_i \sim \mathcal{N}(\mu, \sigma^2)$ one can combine them via the standard average, $Y_i = 1/n \sum_i^{n} X_i$, and the likelihood of the data is still the same given an input model, i.e.

\begin{equation} \ln L = \sum_i^n (\mu - X_i)^2 / 2 \sigma^2 \end{equation}

is the same as

\begin{equation} \ln L = (\mu - Y_i)^2 / 2 (\sigma^2/n) \end{equation}

in a statistical sense. This seemingly works simply because the Gaussian formula is inversely proportional to the variance, which for an average reduces by a factor of $n$.

My question is: is there another example in which rv's (from a particular distribution) can be combined in a way that the relevant likelihood is unaffected? I kind of expect there to be an entire family of these, perhaps even a general method for deriving them, but I don't know the relevant terminology to search the web.

The application of course would be that in such cases, for computing likelihoods, it would be much more computationally feasible (and statistically identical) to use the "averaged" form, as we usually do in the Gaussian case. Just wondering if there's a way to generally derive such a case.

EDIT: I think what I'm asking is more succinctly written like this: Given a set of iid variables $X_i \sim f(x|\theta)$, is there a general way to combine these variables, say $Y = C(X_0, X_1, ..., X_n)$ such that the log likelihood is maintained:

\begin{equation} \sum_i \ln f(X_i|\theta) = \ln f'(Y|\theta) + K, \end{equation} where $f'$ is the pdf of $Y$ and $K$ is a constant (wrt $\theta$).

  • 4
    $\begingroup$ It sounds like you might be asking about stable distributions. Would that be correct? $\endgroup$
    – whuber
    Aug 24, 2020 at 16:42
  • $\begingroup$ That looks close to what I'm looking for, thanks! I guess I'd also like to consider non-linear combinations of variables (out of curiosity). Also, it's not clear to me (though maybe it should be) whether being "stable" implies that the corresponding likelihood contains the same information. $\endgroup$ Aug 24, 2020 at 18:20


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.