Meaning of Gaussian mixture model parameters I came across this question from a tutorial:
Suppose we have observations $x_1$ , $x_2$ , $\ldots$, $x_n$ of a continuous r.v. $X$ known to be drawn from a “mixture” of $k$ Gaussian distributions. Assuming the set of parameters to be $\theta$, give an expression for $p(x_i | \theta)$ (this is the likelihood of $x_i$ given $\theta$)
My question here is: What is $\theta$? Each Guassian in the mixture will have a mean and variance, and a prior probability. What does $\theta$ mean then?
The answer to the question is: $\sum_{j=1}^k P(x_i | j,\theta)P(j | \theta)$. Can someone explain to me how it is calculated? 
 A: Gaussian mixture means, you have a set of $k$ Gaussian distributions that have a parameter $\theta$. We denote them by $P(x|\theta, j)$, where $j \in \{1,...,k\}$ is the index of the Gaussian. Each of these Gaussian distributions - actually each of the indices - has some probability. The probability  of $j \in \{1,....,k\}$ by $P(j|\theta)$.
The model you give here means: First sample $j^* ~ P(\cdot|\theta)$, then sample $x \sim P( \cdot | \theta, j^*)$.
The parameter $\theta$ could include means and covariances of the Gaussians, as well as the probabilities of each Gaussian.
A: In this context, $\theta$ denotes the set of all parameters of the model. These are the means of the individual gaussians ($\mu_j$), their variances ($\sigma_j$), and also the weights of each gaussian ($\pi_j$), i.e. how important is that particular component in the whole mixture. Therefore,
$$\theta = (\mu_1, \sigma_1, \pi_1 \ldots, \mu_k, \sigma_k, \pi_k).$$
When evaluating the probability of a sample $x_i$ under the distribution modeled by the mixture, $P(x_i|\theta)$, you first need to marginalize out the cluster assignment (which is unknown), that is why you sum over all $j$s:
$$P(x_i|\theta) = \sum_{j=1}^k P(x_i|\theta, j) P(j|\theta)$$
Now, $P(j|\theta) = \pi_j$ and $P(x_i|\theta, j)=P(x_i|\mu_j, \sigma_j)$.
