# Fitting multiple distributions with a joint parameter

I would like to fit multiple distributions that share one of their parameters. As a simple example, let's say I have two different datasets which I know follow a Gaussian distribution. let's say I know $\mu_1$ and $\mu_2$, and now I want to estimate $\sigma_1$ and $\sigma_2$. However, I know that $\sigma_1=\sigma_2$.

Now, I would like to find this joint parameter $\sigma$ that best fits both datasets. The maximum likelihood approach tells us how to find parameters by maximizing the likelihood of one distribution, but how do I maximize the likelihood of both? Perhaps I should maximize the sum of the likelihoods or the product of the likelihoods?

• In your simple normal distribution example, the MLE of $\sigma^2$ is simply $(n_1+n_2)^{-1}(\sum_{i=1}^{n_1}(x_i-\bar{x}_{n_1})^2+\sum_{i=1}^{n_2}(y_i-\bar{y}_{n_2})^2)$. Here $x_1,\ldots,x_{n_1}$ and $y_1,\ldots,y_{n_2}$ are two samples with different means and identical variance. – semibruin Jul 22 '13 at 3:46
• @semibruin thanks, I gave that as an example but I am more interested in a case where the MLE does not have a closed form. – Bitwise Jul 22 '13 at 11:50

In general, for independent samples, maximize the product of both likelihoods, or equivalently the sum of the log-likelihoods, over all parameters. Consider the samples as one data-set with joint probability density $$f_{\vec{X}}(\vec{x})=f(x_{11};\mu_1,\sigma)\cdot f(x_{12};\mu_1,\sigma) \cdot \ldots \cdot f(x_{1n_1};\mu_1,\sigma)\cdot f(x_{21};\mu_2,\sigma)\cdot f(x_{22};\mu_2,\sigma) \cdot \ldots \cdot f(x_{2n_2};\mu_2,\sigma)$$ It's no different from what you do in a regression model where each covariate pattern gives its own expected value for the response. As @semibruin points out, there's a simple analytic solution for the case of two normal distributions with common variance but different means.